Unable to create markdown doxia… files just ignored.

2014-09-02 Thread Kevin Burton
I’m trying to create a static site using doxia and the markdown plugin and
it seems to be failing.

Basically, apt and xdocs works, but markdown doesn’t.

I’m using the markdown plugin:

plugin
groupIdorg.apache.maven.plugins/groupId
artifactIdmaven-site-plugin/artifactId
version3.4/version
dependencies
dependency
groupIdorg.apache.maven.doxia/groupId
artifactIddoxia-module-markdown/artifactId
version1.5/version
/dependency
/dependencies
/plugin

… and then run

   mvn site

… but no markdown files are picked up.

If I use xdocs or apt they work just fine.

I’m placing my files in src/site/markdown and they have .md extensions.

… anything else I should try?  is there a debug mode?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


packaging site/doxia content into another multi-module?

2014-09-04 Thread Kevin Burton
I have a multi-module maven build.

In module A I have a site that I generate from doxia which is my products
documentation.

I have another mobile B, that hosts my console / web app and I want the
docs to be packaged here and served from a static Jetty container.

What are my options here?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: packaging site/doxia content into another multi-module?

2014-09-04 Thread Kevin Burton
I was thinking about that but it would be nice if I could use a real
dependency...


On Thu, Sep 4, 2014 at 5:30 PM, Dan Tran dant...@gmail.com wrote:

 you may be able to execute a site:jar execution deploy and consume by your
 web module


 On Thu, Sep 4, 2014 at 5:04 PM, Kevin Burton bur...@spinn3r.com wrote:

  I have a multi-module maven build.
 
  In module A I have a site that I generate from doxia which is my products
  documentation.
 
  I have another mobile B, that hosts my console / web app and I want the
  docs to be packaged here and served from a static Jetty container.
 
  What are my options here?
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


js-sequence-diagrams would make a really cool addition to doxia...

2014-09-06 Thread Kevin Burton
I just got this working with doxia:

http://bramp.github.io/js-sequence-diagrams/

Sequence diagrams are great for documenting protocol interaction.

I use them very often but one of the problems is updating the document when
the protocol changes.

With text sequence diagrams you can do that easily…

Here’s an example:

Andrew-China: Says Hello
Note right of China: China thinks\nabout it
China--Andrew: How are you?
Andrew-China: I am good thanks!

… and that can be rendered into an image for every page load.

I have it working in my doxia by just updating head and then using a div
class=‘diagram’

and then:

$( document ).ready(function() {
$(.diagram).sequenceDiagram({theme: 'hand'});
} );


-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Tell maven to not have the parent pom as a dependency?

2014-09-11 Thread Kevin Burton
I have an OSS module in a multi-module maven project.

I want to post this to a public repo… it’s open source.

The problem is that the parent module is not OSS.

When I setup a dependency it pulls in my OSS module just fine, but then
it tries to pull down the parent module, which isn’t in the repo, and
breaks.

The parent pom isn’t really a dependency… so I’d like it to not need it

is this possible?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Maintaining versions in a multi-module project

2014-09-11 Thread Kevin Burton
I use the maven versions plugin for this…

I just run:

mvn versions:set -DnewVersion=0.0.3  mvn versions:commit

and all my pom.s are updated.


On Thu, Sep 11, 2014 at 6:02 AM, Stephen Connolly 
stephen.alan.conno...@gmail.com wrote:

 Use whatever works best for you.

 I have some projects where it is better for me to force them all in
 lock-step

 I have other projects where I let them run free.

 My general rule of thumb: if the modules are in the same GIT repo or the
 same Subversion trunk then they all will end up being released as one
 atomic group, so keep the version numbers in sync. If they are in
 different SCM instances then they will be released independently so let
 them run free

 On 11 September 2014 09:35, Anuj Agrawal anujagrawa...@gmail.com wrote:

  I am working on a multi-module project where I see that some of the
 module
  POMs have versions different from the parent POM. Apparently, developers
  have made changes to some of the modules and updated the version of only
  those modules.
 
  I also see another project where irrespective of which modules have been
  modified, developers modify the versions of all modules using mvn
 versions
  command to keep all of them in-sync.
 
  I am new to maven and have started using it only since last six months.
  Seeing the above mentioned two projects, I am confused what is the right
  path to take - updating all module versions irrespective of where changes
  are, or updating only those module versions where something changed.
 
  Thanks,
  Anuj Agrawal
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Tell maven to not have the parent pom as a dependency?

2014-09-12 Thread Kevin Burton
The problem I have here is that this would definitely fix my problem.
 Breaking it out into another OSS project would be sweet…

BUT… it would introduce its own set of problems.

Now I have two projects to maintain.  And the number is increasing… From an
IDE perspective, I have to have N windows and switch between them, and
remember which file is in which project.

I find it’s 100x easier to just keep everything in one project.

I could use git-submodules… but IDEA breaks on them and they have a few
gotchas.

… but perhaps there’s no perfect solution.  Just a few solutions that are
less horrible than my current solution.

Kevin


On Thu, Sep 11, 2014 at 10:53 PM, Barrie Treloar baerr...@gmail.com wrote:

 On 12 September 2014 12:55, Kevin Burton bur...@spinn3r.com wrote:

  I have an OSS module in a multi-module maven project.
 
  I want to post this to a public repo… it’s open source.
 
  The problem is that the parent module is not OSS.
 
  When I setup a dependency it pulls in my OSS module just fine, but then
  it tries to pull down the parent module, which isn’t in the repo, and
  breaks.
 
  The parent pom isn’t really a dependency… so I’d like it to not need it
 
  is this possible?


 As Dan says, make it a stand alone project.
 i.e. Dont make it a module.

 Being a module has a special meaning - treat this as part of a bigger
 whole.
 It also help with syntatic sugar by allowing you to run one command at the
 top and have it propogate into all the modules.

 To be complete a module has nothing to do with dependencies or dependency
 management.

 The reason your OSS module is pulling in the parent is not because of
 dependency, but because of inheritance of the parent hierarchy.

 Usually all modules are released together and will share version
 identifiers.
 If they are released independently then you normally wont make them
 modules, and their version identifiers can do their own thing.
 There is a recent post Maintaining versions in a multi-module project
 that Stephen answers, you might also want to search the archives on this
 topic as well.

 A parent pom can be used in two ways; 1) to share common information i.e.
 inheritance 2) keep related artifacts together to make working on a bug
 that traverses artifacts easier i.e aggregation

 In your case I dont think you need to use aggregation, you just need to
 pull out the OSS artifact into its own stand alone location and then
 include it as a normal dependency in your non-OSS project.

 If you find that you are also fixing bugs in the OSS project at the same
 time you are working on the non-OSS one, then you might want to create an
 aggregate pom that has two modules (one OSS, the other non-OSS) so that you
 can run maven commands in one place against both projects. Stephen Connolly
 has some stuff somewhere about that I think.
 The freely availble Maven books might also go into this in more detail, but
 it tends to be a more advanced feature not well described.

 Cheers
 Barrie




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Tell maven to not have the parent pom as a dependency?

2014-09-12 Thread Kevin Burton
 In other words, you can keep using your toplevel pom.xml as an
_aggregator_
without it being the _parent_ of your OSS module.

Ha.  That works.  But the versions plugin now won’t update the version of
the sub-module…

This is super fun! :)


On Fri, Sep 12, 2014 at 9:33 AM, Curtis Rueden ctrue...@wisc.edu wrote:

 Hi Kevin,

  I want to post this to a public repo… it’s open source.
 
  The problem is that the parent module is not OSS.

 Child modules of a multi-module build do not need to use the toplevel
 module as parent.

 In other words, you can keep using your toplevel pom.xml as an _aggregator_
 without it being the _parent_ of your OSS module.

 http://rostislav-matl.blogspot.com/2011/12/maven-aggregator-vs-parent.html

 Regards,
 Curtis

 On Thu, Sep 11, 2014 at 10:25 PM, Kevin Burton bur...@spinn3r.com wrote:

  I have an OSS module in a multi-module maven project.
 
  I want to post this to a public repo… it’s open source.
 
  The problem is that the parent module is not OSS.
 
  When I setup a dependency it pulls in my OSS module just fine, but then
  it tries to pull down the parent module, which isn’t in the repo, and
  breaks.
 
  The parent pom isn’t really a dependency… so I’d like it to not need it
 
  is this possible?
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Tell maven to not have the parent pom as a dependency?

2014-09-12 Thread Kevin Burton
OK… so this definitely works and the aggregation model compiles my module.

But I need a way to bump the version number because I have complex
dependencies and updating 10 places every time I change the project version
isn’t going to be fun.


On Fri, Sep 12, 2014 at 10:03 AM, Kevin Burton bur...@spinn3r.com wrote:

  In other words, you can keep using your toplevel pom.xml as an
 _aggregator_
 without it being the _parent_ of your OSS module.

 Ha.  That works.  But the versions plugin now won’t update the version of
 the sub-module…

 This is super fun! :)


 On Fri, Sep 12, 2014 at 9:33 AM, Curtis Rueden ctrue...@wisc.edu wrote:

 Hi Kevin,

  I want to post this to a public repo… it’s open source.
 
  The problem is that the parent module is not OSS.

 Child modules of a multi-module build do not need to use the toplevel
 module as parent.

 In other words, you can keep using your toplevel pom.xml as an
 _aggregator_
 without it being the _parent_ of your OSS module.

 http://rostislav-matl.blogspot.com/2011/12/maven-aggregator-vs-parent.html

 Regards,
 Curtis

 On Thu, Sep 11, 2014 at 10:25 PM, Kevin Burton bur...@spinn3r.com
 wrote:

  I have an OSS module in a multi-module maven project.
 
  I want to post this to a public repo… it’s open source.
 
  The problem is that the parent module is not OSS.
 
  When I setup a dependency it pulls in my OSS module just fine, but
 then
  it tries to pull down the parent module, which isn’t in the repo, and
  breaks.
 
  The parent pom isn’t really a dependency… so I’d like it to not need it
 
  is this possible?
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Tell maven to not have the parent pom as a dependency?

2014-09-12 Thread Kevin Burton
Thanks…

I just want to bump the version number on the release.

I was using the auto increment fork of the versions number to make it
easier… but I can use the versions plugin manually.

I mean the general use case is I have about 20 modules and mildly complex
inheritance and I don’t want to have to update 30-40 places when I bump a
version number :-(


On Fri, Sep 12, 2014 at 10:22 AM, Puncel, Robert (393J) 
robert.pun...@jpl.nasa.gov wrote:

 You mentioned you were using the versions plugin and it wasn¹t working-
 did you use it before? Can you describe your use case in terms of the
 command line you are using, and what versions you want to upgrade?

 On 9/12/14, 10:16 AM, Kevin Burton bur...@spinn3r.com wrote:

 OKŠ so this definitely works and the aggregation model compiles my module.
 
 But I need a way to bump the version number because I have complex
 dependencies and updating 10 places every time I change the project
 version
 isn¹t going to be fun.
 
 
 On Fri, Sep 12, 2014 at 10:03 AM, Kevin Burton bur...@spinn3r.com
 wrote:
 
   In other words, you can keep using your toplevel pom.xml as an
  _aggregator_
  without it being the _parent_ of your OSS module.
 
  Ha.  That works.  But the versions plugin now won¹t update the version
 of
  the sub-moduleŠ
 
  This is super fun! :)
 
 
  On Fri, Sep 12, 2014 at 9:33 AM, Curtis Rueden ctrue...@wisc.edu
 wrote:
 
  Hi Kevin,
 
   I want to post this to a public repoŠ it¹s open source.
  
   The problem is that the parent module is not OSS.
 
  Child modules of a multi-module build do not need to use the toplevel
  module as parent.
 
  In other words, you can keep using your toplevel pom.xml as an
  _aggregator_
  without it being the _parent_ of your OSS module.
 
 
 
 http://rostislav-matl.blogspot.com/2011/12/maven-aggregator-vs-parent.ht
 ml
 
  Regards,
  Curtis
 
  On Thu, Sep 11, 2014 at 10:25 PM, Kevin Burton bur...@spinn3r.com
  wrote:
 
   I have an OSS module in a multi-module maven project.
  
   I want to post this to a public repoŠ it¹s open source.
  
   The problem is that the parent module is not OSS.
  
   When I setup a dependency it pulls in my OSS module just fine, but
  then
   it tries to pull down the parent module, which isn¹t in the repo, and
   breaks.
  
   The parent pom isn¹t really a dependencyŠ so I¹d like it to not need
 it
  
   is this possible?
  
   --
  
   Founder/CEO Spinn3r.com
   Location: *San Francisco, CA*
   blog: http://burtonator.wordpress.com
   Š or check out my Google+ profile
   https://plus.google.com/102718274791889610666/posts
   http://spinn3r.com
  
 
 
 
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  Š or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 
 
 
 
 --
 
 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 Š or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com


 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Improving the speed of ‘deploy’ ?

2014-09-12 Thread Kevin Burton
mvn package” on my code , without tests, is actually pretty fast.  Takes
about one minute.  (I don’t need the tests btw as I’m using continuous
integration and the code is already tested).

One minute is pretty darn good…

The problem is that ‘deploy’ takes about 7 minutes.

That’s kind of not fun.

I think the issue is that I”m using scpexe for my distributionManagement
repository.

Any options for improving the performance here?

I think the issue is creating new SSH connections each time.  I could use
ControlPersist I think… which is an SSH feature to keep the SSH connection
in the background and re-use it.

Downside: This will require me to use a custom SSH config everywhere.

Another option is to use a file:// repository which is local and then rsync
the binaries over in one pass.

Downside: kind of jacky and requires an additional step.

Any other options ?



-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


forking plugin execution?

2014-09-12 Thread Kevin Burton
Is it possible to fork plugin execution so that it executes in a separate
process?

I think doxia+markdown has a threading bug with peg down.

I get the attached exception.

Works fine with default options but -T 8 breaks it.

I assume if I just have the site plugin in a separate proc that it will
probably work since I won’t have two site builds stepping on each other.


java.lang.ClassCastException: java.lang.Integer cannot be cast to
org.pegdown.ast.SuperNode

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:353)

at org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.OneOrMoreMatcher.match(OneOrMoreMatcher.java:38)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.FirstOfMatcher.match(FirstOfMatcher.java:41)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.FirstOfMatcher.match(FirstOfMatcher.java:41)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.ZeroOrMoreMatcher.match(ZeroOrMoreMatcher.java:39)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.matchers.SequenceMatcher.match(SequenceMatcher.java:46)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

at org.parboiled.parserunners.BasicParseRunner.run(BasicParseRunner.java:72)

at
org.parboiled.parserunners.ReportingParseRunner.runBasicMatch(ReportingParseRunner.java:86)

at
org.parboiled.parserunners.ReportingParseRunner.run(ReportingParseRunner.java:66)

at
org.parboiled.parserunners.AbstractParseRunner.run(AbstractParseRunner.java:81)

at org.pegdown.Parser.parseToParsingResult(Parser.java:1329)

at org.pegdown.Parser.parseInternal(Parser.java:1316)

at org.pegdown.Parser.parse(Parser.java:78)

at org.pegdown.PegDownProcessor.parseMarkdown(PegDownProcessor.java:137)

at
org.apache.maven.doxia.module.markdown.MarkdownParser.parse(MarkdownParser.java:170)

at org.apache.maven.doxia.DefaultDoxia.parse(DefaultDoxia.java:65)

at
org.apache.maven.doxia.siterenderer.DefaultSiteRenderer.renderDocument(DefaultSiteRenderer.java:398)

at
org.apache.maven.doxia.siterenderer.DoxiaDocumentRenderer.renderDocument(DoxiaDocumentRenderer.java:51)

at
org.apache.maven.doxia.siterenderer.DefaultSiteRenderer.renderModule(DefaultSiteRenderer.java:311)

at
org.apache.maven.doxia.siterenderer.DefaultSiteRenderer.render(DefaultSiteRenderer.java:129)

at
org.apache.maven.plugins.site.render.SiteMojo.renderDoxiaDocuments(SiteMojo.java:261)

at
org.apache.maven.plugins.site.render.SiteMojo.renderLocale(SiteMojo.java:172)

at org.apache.maven.plugins.site.render.SiteMojo.execute(SiteMojo.java:141)

at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:132)

... 14 more

Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast
to org.pegdown.ast.SuperNode

at org.pegdown.Parser.addAsChild(Parser.java:1251)

at org.pegdown.Action$mNyPdk1ujbVDZrIU.run(Unknown Source)

at org.parboiled.matchers.ActionMatcher.match(ActionMatcher.java:96)

at
org.parboiled.parserunners.BasicParseRunner.match(BasicParseRunner.java:77)

at org.parboiled.MatcherContext.runMatcher(MatcherContext.java:338)

... 65 more


-- 

Founder/CEO Spinn3r.com
Location: *San 

Re: Improving the speed of ‘deploy’ ?

2014-09-12 Thread Kevin Burton
Worth a shot.. I’m using scpexe which forks OpenSSH as it’s easier to use
when I forward SSH auth… this way I’m not constantly entering passwords for
machines I’ve already authenticated to...


On Fri, Sep 12, 2014 at 11:31 AM, Curtis Rueden ctrue...@wisc.edu wrote:

 Hi Kevin,

  I think the issue is that I”m using scpexe for my
  distributionManagement repository.

 Is there some reason you can't just use wagon-ssh?

 http://stackoverflow.com/a/14404990

 Regards,
 Curtis

 On Fri, Sep 12, 2014 at 1:11 PM, Kevin Burton bur...@spinn3r.com wrote:

  mvn package” on my code , without tests, is actually pretty fast.  Takes
  about one minute.  (I don’t need the tests btw as I’m using continuous
  integration and the code is already tested).
 
  One minute is pretty darn good…
 
  The problem is that ‘deploy’ takes about 7 minutes.
 
  That’s kind of not fun.
 
  I think the issue is that I”m using scpexe for my distributionManagement
  repository.
 
  Any options for improving the performance here?
 
  I think the issue is creating new SSH connections each time.  I could use
  ControlPersist I think… which is an SSH feature to keep the SSH
 connection
  in the background and re-use it.
 
  Downside: This will require me to use a custom SSH config everywhere.
 
  Another option is to use a file:// repository which is local and then
 rsync
  the binaries over in one pass.
 
  Downside: kind of jacky and requires an additional step.
 
  Any other options ?
 
 
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Tell maven to not have the parent pom as a dependency?

2014-09-13 Thread Kevin Burton
I think I could probably migrate it down …  Maybe cut my dependencies in
1/2 .. However, the version idea is probably the right way to go here :)

Thanks for the recommendation!

Kevin

On Fri, Sep 12, 2014 at 6:31 PM, Barrie Treloar baerr...@gmail.com wrote:

 On 13 September 2014 02:46, Kevin Burton bur...@spinn3r.com wrote:

  OK… so this definitely works and the aggregation model compiles my
 module.
 
  But I need a way to bump the version number because I have complex
  dependencies and updating 10 places every time I change the project
 version
  isn’t going to be fun.
 

 If you have 10 places where the dependencies need changing you are doing it
 wrong.

 You should have zero version definitions in your modules.
 All version definitions are in the topmost parent pom in the
 dependencyManagement section.
 If you have a bunch of things that share versions, you can also create a
 property and then use something like ${mygroup.version} so that you only
 need to update the property section once.

 Have you read the freely available Maven books?
 It's been long while since I've read them but I think this advice is in
 there.




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


retrying flakey tests?

2014-09-19 Thread Kevin Burton
Is there a way to retry a flakey test?

I’d basically like to have a flag that retries a failing test 2 or 3 times.

Flakey and non deterministic tests are a fact of life in more complicated
systems.

The main problem being that they are impossible to setup again because
they’re usually race conditions.

If I could just retry the test a second time , that would solve this issue.

Would be nice to have the build warned that the test is failing though..

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: retrying flakey tests?

2014-09-19 Thread Kevin Burton
The loop idea is a good one.. I’d have to implement that on my own but it’s
not the end of the world.

And I’d love to live in a world you present above where all test cases work
and can be isolated :)

… but it’s not practical. Google has a big problem with this in their unit
testing.  They have millions of unit tests and they can’t fail a whole
build because 1/5000 of their tests are flakey.  At 1M tests you’re going
to end up with a lot failing.

In my situations I have weird race conditions with ActiveMQ having a JMX
bug where there’s a race from the GC time of the queue and the time I read
the JMX value.

I have about 2-3 of these.. I CAN NOT reproduce these at all.. only during
testing. It’s amazingly annoying :-(

On Fri, Sep 19, 2014 at 10:18 AM, Martin Todorov carlspr...@gmail.com
wrote:

 I agree with Curtis about the possible approach of doing a for loop. I'm
 also aware of the fact that every corner case is a case of it's own and
 that you sometimes need to be able to do things quick and dirty.

 I had a manager who insisted I should simply ignore one of the test cases
 he'd written, which was failing regularly only during releasing on our CI
 server. He refused to fix the test for about a year and a half, as it was
 a flake. Eventually, it started hitting people big time, especially when
 we moved to the cloud, and it turned out to be something which shouldn't
 have been ignored at all...

 In the end, it's all up to you, but I'd recommend playing it as safe as
 possible! :-)

 On Fri, Sep 19, 2014 at 6:08 PM, Curtis Rueden ctrue...@wisc.edu wrote:

  Hi Kevin,
 
   Is there a way to retry a flakey test?
 
  In general I agree with Martin Todorov that tests should be small and
  atomic, and flakiness is a sign of larger problems. However, I also agree
  with you that sometimes flaky tests are a reality: my group has run into
  this with behavior of the JVM garbage collector differing on Windows, for
  example, in a rather nondeterministic way. In that case, retrying tests
  ~5-10 times was a good solution for us.
 
  But rather than configuring Surefire or Failsafe to do this, I'd suggest
  (if possible) to write it into the test itself. Put the test in a for
 loop,
  and then assert at the end that numSuccesses  0 or whatever. You then
 also
  issue a warning if numSuccesses != numAttempts, as you requested. In
 short:
  it gives you the flexibility you need.
 
  Regards,
  Curtis
 
  On Fri, Sep 19, 2014 at 11:24 AM, Kevin Burton bur...@spinn3r.com
 wrote:
 
   Is there a way to retry a flakey test?
  
   I’d basically like to have a flag that retries a failing test 2 or 3
  times.
  
   Flakey and non deterministic tests are a fact of life in more
 complicated
   systems.
  
   The main problem being that they are impossible to setup again because
   they’re usually race conditions.
  
   If I could just retry the test a second time , that would solve this
  issue.
  
   Would be nice to have the build warned that the test is failing
 though..
  
   --
  
   Founder/CEO Spinn3r.com
   Location: *San Francisco, CA*
   blog: http://burtonator.wordpress.com
   … or check out my Google+ profile
   https://plus.google.com/102718274791889610666/posts
   http://spinn3r.com
  
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: retrying flakey tests?

2014-09-20 Thread Kevin Burton
NICE!!! This is awesome.  I’m definitely going to be using this :)

On Sat, Sep 20, 2014 at 8:44 AM, Andreas Gudian andreas.gud...@gmail.com
wrote:

 Hi there,

 We had a pull request for such a feature and it was merged. It will be part
 of our next release: https://jira.codehaus.org/browse/SUREFIRE
 https://jira.codehaus.org/browse/SUREFIRE-1087-1087

 I think the feature makes expecially sense for tracking flaky tests over a
 longer period of time, as those flaky tests show up more nicely on the
 surefire tester port xml files and can thus be tracked specifically in the
 CI system.

 Surefire 2.18 should be out in the next couple of weeks.

 Hth,
 Andreas

 Am Samstag, 20. September 2014 schrieb Kevin Burton :

  The loop idea is a good one.. I’d have to implement that on my own but
 it’s
  not the end of the world.
 
  And I’d love to live in a world you present above where all test cases
 work
  and can be isolated :)
 
  … but it’s not practical. Google has a big problem with this in their
 unit
  testing.  They have millions of unit tests and they can’t fail a whole
  build because 1/5000 of their tests are flakey.  At 1M tests you’re going
  to end up with a lot failing.
 
  In my situations I have weird race conditions with ActiveMQ having a JMX
  bug where there’s a race from the GC time of the queue and the time I
 read
  the JMX value.
 
  I have about 2-3 of these.. I CAN NOT reproduce these at all.. only
 during
  testing. It’s amazingly annoying :-(
 
  On Fri, Sep 19, 2014 at 10:18 AM, Martin Todorov carlspr...@gmail.com
  javascript:;
  wrote:
 
   I agree with Curtis about the possible approach of doing a for loop.
 I'm
   also aware of the fact that every corner case is a case of it's own and
   that you sometimes need to be able to do things quick and dirty.
  
   I had a manager who insisted I should simply ignore one of the test
 cases
   he'd written, which was failing regularly only during releasing on our
 CI
   server. He refused to fix the test for about a year and a half, as it
 was
   a flake. Eventually, it started hitting people big time, especially
  when
   we moved to the cloud, and it turned out to be something which
 shouldn't
   have been ignored at all...
  
   In the end, it's all up to you, but I'd recommend playing it as safe as
   possible! :-)
  
   On Fri, Sep 19, 2014 at 6:08 PM, Curtis Rueden ctrue...@wisc.edu
  javascript:; wrote:
  
Hi Kevin,
   
 Is there a way to retry a flakey test?
   
In general I agree with Martin Todorov that tests should be small and
atomic, and flakiness is a sign of larger problems. However, I also
  agree
with you that sometimes flaky tests are a reality: my group has run
  into
this with behavior of the JVM garbage collector differing on Windows,
  for
example, in a rather nondeterministic way. In that case, retrying
 tests
~5-10 times was a good solution for us.
   
But rather than configuring Surefire or Failsafe to do this, I'd
  suggest
(if possible) to write it into the test itself. Put the test in a for
   loop,
and then assert at the end that numSuccesses  0 or whatever. You
 then
   also
issue a warning if numSuccesses != numAttempts, as you requested. In
   short:
it gives you the flexibility you need.
   
Regards,
Curtis
   
On Fri, Sep 19, 2014 at 11:24 AM, Kevin Burton bur...@spinn3r.com
  javascript:;
   wrote:
   
 Is there a way to retry a flakey test?

 I’d basically like to have a flag that retries a failing test 2 or
 3
times.

 Flakey and non deterministic tests are a fact of life in more
   complicated
 systems.

 The main problem being that they are impossible to setup again
  because
 they’re usually race conditions.

 If I could just retry the test a second time , that would solve
 this
issue.

 Would be nice to have the build warned that the test is failing
   though..

 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com

   
  
 
 
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


wagon ssh corrupts known_hosts?

2014-09-20 Thread Kevin Burton
I have a step release process.

Step 1 uses maven and the wagon-ssh plugin to push .debs and .jars to my
maven and debian repositories via SSH and the scp:// wagon which uses jSch.

Step 2 uses ansible and SSH via OpenSSH… this is the step that breaks now
because jSch writes a new known_hosts file.  SSH then prompts me every time
to accept the new host key.

This is prone to error and I  want to fix it.. I do releases in ‘screen’
and if I forget to come back 5 minutes later to type ‘yes’ for all my hosts
it breaks the release.

Plus I want it automated for obvious reasons.



-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Maven site:deploy hosting providers??

2014-09-26 Thread Kevin Burton
I don’t want to necessarily host our own repositories.

They’re just static file repositories.

Are there any hosting providers out there that support dav, SCP or HTTP PUT
that would work with wagon?

Required features are:

- SSL

- redundant copies on multiple servers (CDN)

- dav/scp/ftp/ upload.. anything supported by wagon.

- HTTP auth

- domain masking

… I just want something easy to setup and then never worry about it again.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Maven site:deploy hosting providers??

2014-09-27 Thread Kevin Burton
This is such an obviously problem I’m amazed there isn’t an easier
solution.  Bintray looks cool. I just asked them how they feel about static
site hosting?

I want something that works with mvn site:deploy

On Fri, Sep 26, 2014 at 10:49 PM, Manfred Moser manf...@mosabuam.com
wrote:

 If you just want to have the sites hosted any hosting solution will do it.
 CDN will be more expensive but I would really wonder if you need that for a
 static site. You would have to get a LOT of traffic to make it worth it.

 Bintray focuses on binary distribution and not site hosting.

 If you want both you could even just run Nexus OSS on any server and host
 the binaries and the site using a Maven repository for the binaries and a
 site repository for the site..

 manfred

 Bernd wrote on 26.09.2014 17:41:

  Have a look at bintray
 
  https://bintray.com/docs/uploads/uploads_uploads.html
 
  Greetings
  Bernd
  Am 26.09.2014 21:39 schrieb Kevin Burton bur...@spinn3r.com:
 
  I don’t want to necessarily host our own repositories.
 
  They’re just static file repositories.
 
  Are there any hosting providers out there that support dav, SCP or HTTP
 PUT
  that would work with wagon?
 
  Required features are:
 
  - SSL
 
  - redundant copies on multiple servers (CDN)
 
  - dav/scp/ftp/ upload.. anything supported by wagon.
 
  - HTTP auth
 
  - domain masking
 
  … I just want something easy to setup and then never worry about it
 again.
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 
 


 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Maven site:deploy hosting providers??

2014-09-27 Thread Kevin Burton
I don't think github would work but would love to be wrong.

Their static page hosting  is acceptable but it's tricky to make sure all
pages get added and removed properly.

Their binary hosting uses their own proprietary api which isn't supported
by wagon.
On Sep 27, 2014 4:49 AM, Jason van Zyl ja...@takari.io wrote:

 I believe there are tools that will allow you to publish your static site
 to Github pages.

 On Sep 27, 2014, at 2:21 AM, Kevin Burton bur...@spinn3r.com wrote:

  This is such an obviously problem I’m amazed there isn’t an easier
  solution.  Bintray looks cool. I just asked them how they feel about
 static
  site hosting?
 
  I want something that works with mvn site:deploy
 
  On Fri, Sep 26, 2014 at 10:49 PM, Manfred Moser manf...@mosabuam.com
  wrote:
 
  If you just want to have the sites hosted any hosting solution will do
 it.
  CDN will be more expensive but I would really wonder if you need that
 for a
  static site. You would have to get a LOT of traffic to make it worth it.
 
  Bintray focuses on binary distribution and not site hosting.
 
  If you want both you could even just run Nexus OSS on any server and
 host
  the binaries and the site using a Maven repository for the binaries and
 a
  site repository for the site..
 
  manfred
 
  Bernd wrote on 26.09.2014 17:41:
 
  Have a look at bintray
 
  https://bintray.com/docs/uploads/uploads_uploads.html
 
  Greetings
  Bernd
  Am 26.09.2014 21:39 schrieb Kevin Burton bur...@spinn3r.com:
 
  I don’t want to necessarily host our own repositories.
 
  They’re just static file repositories.
 
  Are there any hosting providers out there that support dav, SCP or
 HTTP
  PUT
  that would work with wagon?
 
  Required features are:
 
  - SSL
 
  - redundant copies on multiple servers (CDN)
 
  - dav/scp/ftp/ upload.. anything supported by wagon.
 
  - HTTP auth
 
  - domain masking
 
  … I just want something easy to setup and then never worry about it
  again.
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 
 
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
  For additional commands, e-mail: users-h...@maven.apache.org
 
 
 
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com

 Thanks,

 Jason

 --
 Jason van Zyl
 Founder,  Apache Maven
 http://twitter.com/jvanzyl
 http://twitter.com/takari_io
 -

 the course of true love never did run smooth ...

  -- Shakespeare












Re: Maven site:deploy hosting providers??

2014-09-29 Thread Kevin Burton
I got that working.. but I think it would not work with multi-modules?

I have it working in a module… but it writes to / … I would assume it would
write to /$module_name … or something along those lines.

So I think later modules would overwrite previous modules.  But perhaps I’m
wrong.

On Sun, Sep 28, 2014 at 2:10 PM, Manfred Moser manf...@mosabuam.com wrote:

 If you are looking for a working example setup, the android maven plugin
 uses Github pages..

 http://jayway.github.io/maven-android-plugin/
 https://github.com/jayway/maven-android-plugin

 Mirko Friedenhagen wrote on 28.09.2014 02:18:

  I havr used scm-publish together with gh-pages branch as well and as
  Karl-Heinz already stated, it works like a charm for me as well.
 
  Regards
  Mirko
  --
  Sent from my mobile
  On Sep 27, 2014 8:05 PM, Karl Heinz Marbaise khmarba...@gmx.de
 wrote:
 
  Hi,
 
   we have scm-publish plugin for that
 
  http://maven.apache.org/plugins/maven-scm-publish-plugin/
 
 
  I've already used that for GitHub pages...works like a charm...
 
  https://github.com/khmarbaise/smpp
  https://github.com/khmarbaise/uptodate-maven-plugin etc.
 
  which makes it very ease...
 
  apart from that you can use github-site-plugin which works as well...
 
  https://github.com/github/maven-plugins
 
 
  Kind regards
  Karl-Heinz Marbaise
 
 
  Regards,
 
  Hervé
 
  Le samedi 27 septembre 2014 10:09:35 Kevin Burton a écrit :
 
  I don't think github would work but would love to be wrong.
 
  Their static page hosting  is acceptable but it's tricky to make sure
 all
  pages get added and removed properly.
 
  Their binary hosting uses their own proprietary api which isn't
 supported
  by wagon.
 
  On Sep 27, 2014 4:49 AM, Jason van Zyl ja...@takari.io wrote:
 
  I believe there are tools that will allow you to publish your static
  site
  to Github pages.
 
  On Sep 27, 2014, at 2:21 AM, Kevin Burton bur...@spinn3r.com
 wrote:
 
  This is such an obviously problem I’m amazed there isn’t an easier
  solution.  Bintray looks cool. I just asked them how they feel about
 
 
  static
 
   site hosting?
 
  I want something that works with mvn site:deploy
 
  On Fri, Sep 26, 2014 at 10:49 PM, Manfred Moser 
 manf...@mosabuam.com
 
  wrote:
 
  If you just want to have the sites hosted any hosting solution
 will do
 
 
  it.
 
   CDN will be more expensive but I would really wonder if you need
 that
 
 
  for a
 
   static site. You would have to get a LOT of traffic to make it worth
  it.
 
  Bintray focuses on binary distribution and not site hosting.
 
  If you want both you could even just run Nexus OSS on any server
 and
 
 
  host
 
   the binaries and the site using a Maven repository for the binaries
 and
 
 
  a
 
   site repository for the site..
 
  manfred
 
  Bernd wrote on 26.09.2014 17:41:
 
  Have a look at bintray
 
  https://bintray.com/docs/uploads/uploads_uploads.html
 
  Greetings
  Bernd
 
  Am 26.09.2014 21:39 schrieb Kevin Burton bur...@spinn3r.com:
 
  I don’t want to necessarily host our own repositories.
 
  They’re just static file repositories.
 
  Are there any hosting providers out there that support dav, SCP
 or
 
 
  HTTP
 
   PUT
 
   that would work with wagon?
 
  Required features are:
 
  - SSL
 
  - redundant copies on multiple servers (CDN)
 
  - dav/scp/ftp/ upload.. anything supported by wagon.
 
  - HTTP auth
 
  - domain masking
 
  … I just want something easy to setup and then never worry about
 it
 
 
  again.
 
   --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 
 
 
 -
  To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
  For additional commands, e-mail: users-h...@maven.apache.org
 
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 
 
  Thanks,
 
  Jason
 
  --
  Jason van Zyl
  Founder,  Apache Maven
  http://twitter.com/jvanzyl
  http://twitter.com/takari_io
  -
 
  the course of true love never did run smooth ...
 
-- Shakespeare
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
  For additional commands, e-mail: users-h...@maven.apache.org
 
 
 

 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https

Best practice for integration tests (vs unit tests)

2014-10-07 Thread Kevin Burton
Right now I have an overlap of unit and integration tests and I want to
tear out my integration tests and keep them separate.  This way our unit
tests execute fast and integration testing takes a bit longer.

… and now I’m trying to figure out the best strategy for that.

Reading this:

http://docs.codehaus.org/display/MAVENUSER/Maven+and+Integration+Testing

seems to not solve the problem.

Seems to me the easiest would be to just have src/main/integration-tests

… but would I have to use failsafe or surefire?  I’m not sure why I would
need another plugin for this.

Maybe someone can enlightenment me with some maven wisdom?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


How do I test client and server since they form circular dependencies?

2014-10-08 Thread Kevin Burton
I have a client and server in a multi-module.

How do I test them ? I’d like to start the server in a test, then run the
client against it.

Then I’d like to run the client in a test, and have to test a server…

maybe one idea is to build a meta-module like foo-client-server-test which
as a dependency on foo-client and foo-server.  then they could each test
each other?  Seems a bit of a hack but it should work ..

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


best strategy for starting daemons and forking procs during maven integration tests?

2014-10-14 Thread Kevin Burton
I want to run more integration tests as my daemons and command line apps
sometimes have bugs that I’d like to catch before a release.

9/10 these are Guice bindings and pretty easy to figure out what’s
happening quickly.

I would like to actually create new processes as a lot of Java code just
doesn’t do a good job at cleaning up after itself.  Mostly this is due to
leaving Threads around and other incomplete shutdown issues.

my biggest concern is running maven, and then accidentally leaving a
process around afterwards.  I’d like to avoid this problem entirely…

One idea is to run everything in a Docker container, then kill the
container after.

Thoughts? would love to see what others have done here.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


computing rank on the top maven modules by using the pom.xml files in maven central?

2014-10-18 Thread Kevin Burton
I had a though today that it might be valuable to compute a rank (either
pagerank or just raw in degree) on the top modules in maven central.

But perhaps someone has already done this?  I’m lazy. :)

The goal here , for me at least, is to fine the top libraries that everyone
is using based on rank.  If the community is adopting a certain module I’d
like to yield to wisdom of the crowds and assume it’s decent and maybe
something I should also consider using.


-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: computing rank on the top maven modules by using the pom.xml files in maven central?

2014-10-18 Thread Kevin Burton
Ah.. yes… caveats included, this is really helpful!  Thanks!

On Sat, Oct 18, 2014 at 12:58 PM, Bernd Eckenfels e...@zusammenkunft.net
wrote:

 Hello,

 some of the Maven search engines like mvnrepository.com do that.
 However the metrics are a bit of a problem, as a lot of those are
 forced into the dependency tree and others might be very popular in
 a small field (science) but never be able to dominate the general
 purpose ones.

 http://mvnrepository.com/popular

 Bernd

 Am Sat, 18 Oct 2014 10:22:48 -0700 schrieb Kevin Burton
 bur...@spinn3r.com:

  I had a though today that it might be valuable to compute a rank
  (either pagerank or just raw in degree) on the top modules in maven
  central.
 
  But perhaps someone has already done this?  I’m lazy. :)
 
  The goal here , for me at least, is to fine the top libraries that
  everyone is using based on rank.  If the community is adopting a
  certain module I’d like to yield to wisdom of the crowds and assume
  it’s decent and maybe something I should also consider using.
 
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


fetching a tar.gz URL / dependency via HTTP during a build?

2014-11-08 Thread Kevin Burton
I’m trying to build .debs using jdeb and maven for various projects.

For example, there is no .deb for spark.  So I’m just taking the tar.gz and
making a deb that installs to /usr/share/apache-spark.

The problem is I don’t want to put the full binary into git as it’s about
200MB.

I’d rather have the build fetch the tar.gz directly from Apache during the
build, untar it, then build the deb from the output.

Is there and way to do this or should I just write a bash script which uses
wget to fetch the URL?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: fetching a tar.gz URL / dependency via HTTP during a build?

2014-11-08 Thread Kevin Burton
oh .. gotcha.  So I would/could take the tar.gz and put it in a maven
repo.  I don’t think there is one for the tar.gzs though.



On Sat, Nov 8, 2014 at 10:45 AM, Bernd e...@zusammenkunft.net wrote:

 You could use the dependency-plugin with copy dependencies to get the
 binary artifact from a maven repo (if there is one).
 Am 08.11.2014 19:43 schrieb Kevin Burton bur...@spinn3r.com:

  I’m trying to build .debs using jdeb and maven for various projects.
 
  For example, there is no .deb for spark.  So I’m just taking the tar.gz
 and
  making a deb that installs to /usr/share/apache-spark.
 
  The problem is I don’t want to put the full binary into git as it’s about
  200MB.
 
  I’d rather have the build fetch the tar.gz directly from Apache during
 the
  build, untar it, then build the deb from the output.
 
  Is there and way to do this or should I just write a bash script which
 uses
  wget to fetch the URL?
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: fetching a tar.gz URL / dependency via HTTP during a build?

2014-11-08 Thread Kevin Burton
Nice.. it works actually.  the only problem is that it downloads the file
every time and it’s 180MB… there’s no way to skip it if the file already
exists.  hm..

On Sat, Nov 8, 2014 at 11:00 AM, Karl Heinz Marbaise khmarba...@gmx.de
wrote:

 Hi Kevin,

 to download a an file from somewhere you could use the wagon-maven-plugin
 like this:

 plugin
   groupIdorg.codehaus.mojo/groupId
   artifactIdwagon-maven-plugin/artifactId
   version1.0-beta-5/version
   executions
 execution
   iddownload-test-data/id
   phaseprepare-package/phase
   goals
 goaldownload-single/goal
   /goals
   configuration
 urlhttp://archive.apache.org/dist/abdera/1.1.2//url
 fromFileapache-abdera-1.1.2-src.tar.gz/fromFile
 toDir${project.build.directory}/environment/qa/toDir
   /configuration
 /execution
   /executions
 /plugin



 For unpacking of the tar.gz you
 On 11/8/14 7:40 PM, Kevin Burton wrote:

 I’m trying to build .debs using jdeb and maven for various projects.

 For example, there is no .deb for spark.  So I’m just taking the tar.gz
 and
 making a deb that installs to /usr/share/apache-spark.

 The problem is I don’t want to put the full binary into git as it’s about
 200MB.

 I’d rather have the build fetch the tar.gz directly from Apache during the
 build, untar it, then build the deb from the output.

 Is there and way to do this or should I just write a bash script which
 uses
 wget to fetch the URL?


 Kind regards
 Karl Heinz Marbaise

 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Sorting javadoc output by importance of documentation and length of docs?

2014-11-16 Thread Kevin Burton
This is slightly off topic so I apologize.

Javadoc is great for the developer but the main problem is that important
documents can be hidden in a sea of classes.

My Javadoc tends to have say 20-40 important documents on the design of a
system but the rest of the Javadoc just tends to have an overview.

I’d like a way to organize it more as a book, maybe even by the length of
the documentation instead of being organized as a table of contents.

This way my ops guys and other members of the team can find the
documentation they’re looking for and not necessarily how a specific
component behaves.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Maven locking up downloading file over scp in deploy plugin.

2015-02-08 Thread Kevin Burton
Nice.  Thanks for the FYI.  Any ETA on when 2.9 will be released?

For now I can remove the maven-metadata.xml files on release but it’s
somewhat annoying.

On Mon, Feb 2, 2015 at 11:11 AM, Robert Scholte rfscho...@apache.org
wrote:

 Hi,

 I think you either changes Maven or its wagon-library.
 See https://jira.codehaus.org/browse/MNG-5605 for details.
 Root cause is https://jira.codehaus.org/browse/WAGON-429

 thanks,
 Robert

 Op Sat, 31 Jan 2015 21:05:25 +0100 schreef Kevin Burton 
 bur...@spinn3r.com:

  I’m really confused over this issue.

 I’m running the maven deploy plugin and sending files over scp.

 It’s been working fine for MONTHS, then all of a sudden it started to
 fail.

 It uploads the files fine. but then on download it just freezes here:

 Downloading: scp://
 util0017.wdc.sl.spinn3r.com/var/www/maven-repo/private/
 com/spinn3r/metrics/kairosdb/metrics-kairosdb/maven-metadata.xml
 327/326 B

 and locks up forever.

 dumping the stack it looks like it’s just reading the response.

 If I cat the file on the remote end, it reads it just fine.  The box has
 no
 disk issues, no networking issues, no CPU, no obvious bottlenecks.  dmesg
 doesn’t show any filesystem errors, etc.

 Any advice?


 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Maven locking up downloading file over scp in deploy plugin.

2015-02-08 Thread Kevin Burton
I’m curious why this is happening to me. We just upgraded our version of
Maven.  We were on 3.2.x before and now we’re on 3.2.5.  Can I downgrade
versions to fix this?

I’m unsure what the previous version of Maven we were on as we’ve given
that box back now :-(

Kevin

On Sat, Feb 7, 2015 at 9:55 AM, Kevin Burton bur...@spinn3r.com wrote:

 Nice.  Thanks for the FYI.  Any ETA on when 2.9 will be released?

 For now I can remove the maven-metadata.xml files on release but it’s
 somewhat annoying.

 On Mon, Feb 2, 2015 at 11:11 AM, Robert Scholte rfscho...@apache.org
 wrote:

 Hi,

 I think you either changes Maven or its wagon-library.
 See https://jira.codehaus.org/browse/MNG-5605 for details.
 Root cause is https://jira.codehaus.org/browse/WAGON-429

 thanks,
 Robert

 Op Sat, 31 Jan 2015 21:05:25 +0100 schreef Kevin Burton 
 bur...@spinn3r.com:

  I’m really confused over this issue.

 I’m running the maven deploy plugin and sending files over scp.

 It’s been working fine for MONTHS, then all of a sudden it started to
 fail.

 It uploads the files fine. but then on download it just freezes here:

 Downloading: scp://
 util0017.wdc.sl.spinn3r.com/var/www/maven-repo/private/
 com/spinn3r/metrics/kairosdb/metrics-kairosdb/maven-metadata.xml
 327/326 B

 and locks up forever.

 dumping the stack it looks like it’s just reading the response.

 If I cat the file on the remote end, it reads it just fine.  The box has
 no
 disk issues, no networking issues, no CPU, no obvious bottlenecks.  dmesg
 doesn’t show any filesystem errors, etc.

 Any advice?


 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Avoiding situations where duplicate classnames are imported?

2015-01-18 Thread Kevin Burton
It looks like I’ve accidentally ended up in a situation where I have
duplicate class names in separate .jars with different versions.

So I’ll have Foo.class in foo-1.0.0.jar and foo-3.0.0.jar …

Shouldn’t Maven fail in this scenario?  IE assert that you have conflicting
dependencies and that you should resolve them by hand.

This usually happens with two projects (usually an older one) depends on an
older version of a library.

I may be foolishly using a few older libs which, while stable, don’t really
work with some of their more modern dependencies.



-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Avoiding situations where duplicate classnames are imported?

2015-01-19 Thread Kevin Burton
The main situation I see is when the artifact and group ID differ …

so  an older version of jdom used just jdom and the newer version used
org.jdom and then I ended up with both :)

On Sun, Jan 18, 2015 at 10:31 PM, Adrien Rivard adrien.riv...@gmail.com
wrote:

 Hi,

 if youre artifacts have the same artifactId/groupId/type/classifier  it
 should'nt happen,Maven should pick only one of the two. If one of
 the artifactId/groupId/type/classifier is not the same, it could happen
 because it's not the same jar for maven.


 On Mon, Jan 19, 2015 at 4:32 AM, Steven Schlansker 
 stevenschlans...@gmail.com wrote:

  Check out
  https://github.com/basepom/duplicate-finder-maven-plugin
 
  On Jan 18, 2015, at 6:12 PM, Kevin Burton bur...@spinn3r.com wrote:
 
   It looks like I’ve accidentally ended up in a situation where I have
   duplicate class names in separate .jars with different versions.
  
   So I’ll have Foo.class in foo-1.0.0.jar and foo-3.0.0.jar …
  
   Shouldn’t Maven fail in this scenario?  IE assert that you have
  conflicting
   dependencies and that you should resolve them by hand.
  
   This usually happens with two projects (usually an older one) depends
 on
  an
   older version of a library.
  
   I may be foolishly using a few older libs which, while stable, don’t
  really
   work with some of their more modern dependencies.
  
  
  
   --
  
   Founder/CEO Spinn3r.com
   Location: *San Francisco, CA*
   blog: http://burtonator.wordpress.com
   … or check out my Google+ profile
   https://plus.google.com/102718274791889610666/posts
   http://spinn3r.com
 
 
  -
  To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
  For additional commands, e-mail: users-h...@maven.apache.org
 
 


 --
 Adrien Rivard




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Question about next maven-fluido-skin release

2015-02-17 Thread Kevin Burton
It’s a bit disturbing that this bug took so long to fix. I had to work
around it and was expecting a resolution.

It was a 1 line fix that took 10 months for a resolution, and it’s been 2
months since then and there hasn’t been a release yet.

On Sun, Feb 15, 2015 at 11:39 PM, Baptiste Mathus m...@batmat.net wrote:

 Hi better ask for a release in the dev list. If the code is actually
 already fixed and just needs be released there's generally committers
 willing to do it.

 Cheers
 Le 14 févr. 2015 02:23, Mikko Kuokkanen mikko.kuokka...@gmail.com a
 écrit :

  Hi.
 
  We are prototyping following combination for our web site generation:
  * maven-site-plugin 3.4
  * doxia-module-markdown 1.6
  * maven-fluido-skin 1.3.1
 
  According to fluido documentation it has nice source code prettify
  feature with line numbers. Alas, I didn't get it to work. Then I found
  this bug report http://jira.codehaus.org/browse/MSKINS-86. It seems
  that it is a bug and will be fixed in the next versions 1.3.2 and 1.4.
 
  I did not see any release date attached to those versions. Is there
  any timetable when they could be released?
 
 
  Thanks,
  Mikko Kuokkanen
 
  -
  To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
  For additional commands, e-mail: users-h...@maven.apache.org
 
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


I created Debian packages for maven 3.2.5..

2015-01-27 Thread Kevin Burton
Yo dawg.  I heard you like maven. So I used maven to build debian packages
of maven so you can use maven while you build maven. ….

Debian only has an old/stale version of Debian so I created my own packages

https://github.com/spinn3r/apache-maven-deb

You an just run mvn package to build these.

I haven’t tested them extensively but they work fine for me so far.

I didn’t do much with the dependencies but you have to have Java installed.

Feel free to submit a pull request if you would like any changes.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Maven locking up downloading file over scp in deploy plugin.

2015-01-31 Thread Kevin Burton
I’m really confused over this issue.

I’m running the maven deploy plugin and sending files over scp.

It’s been working fine for MONTHS, then all of a sudden it started to fail.

It uploads the files fine. but then on download it just freezes here:

Downloading: scp://
util0017.wdc.sl.spinn3r.com/var/www/maven-repo/private/com/spinn3r/metrics/kairosdb/metrics-kairosdb/maven-metadata.xml
327/326 B

and locks up forever.

dumping the stack it looks like it’s just reading the response.

If I cat the file on the remote end, it reads it just fine.  The box has no
disk issues, no networking issues, no CPU, no obvious bottlenecks.  dmesg
doesn’t show any filesystem errors, etc.

Any advice?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Resolving classpath hell issues by allowing libraries to only use the version they request?

2015-01-05 Thread Kevin Burton
Breaking your application into smaller independent services may give you a
 way to reduce the number of conflicts related to these different transitive
 dependencies but that may not be as easy as testing your app with the
 updated versions or doing some research with the teams that are building
 with the old libraries to see if they will upgrade and run their acceptance
 tests within a time frame that meets your project's timeline requirements.


I agree.  in this case the issue is testing. I need to embed cassandra so I
can test it but it’s conflicting with spark.

Ideally I could just start a container and I think that’s the way we’re
going to evolve things.  Essentially, start a container for the daemons I
need to test against and then make sure all my components work.  Then just
kill the containers.

—

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Easy way to test maven / surefire with only changed classes? javac dependencies?

2015-02-10 Thread Kevin Burton
Is there an easy way to build the Java dependency tree from the compiler?

I was thinking that if you can get the Java dependency tree built, then you
take take a look at a diff and look at which files have changed.

Then from there you could take say 1000 test and reduce that to only 10
test if only those ten had their dependencies changed.

The theory being that if the previous commit already tested the previous
990, why test them again?

The epiphany I had was that one could EASILY integrate this into maven by
just passing a list of which tests to skip.

This could dramatically improve the speed of continuous integration systems

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Easy way to test maven / surefire with only changed classes? javac dependencies?

2015-02-15 Thread Kevin Burton
I agree that dependency insanity could cause problems , as well as
resources.

However, the maven dependencies can be partly resolved because we can also
just analyze the pom to look at those dependencies.

I imagine as long as resources are loaded from /src/test/resources then we
can make ANY change here trigger a full re-test.  It’s not like this
happens often.

And in spare CPU time you can re-run FULL tests (and maybe we should do
this before a release).

But at least this way, if something DEFINITELY fails, you can resolve it
somewhat quickly.

On Wed, Feb 11, 2015 at 4:52 AM, Ron Wheeler rwhee...@artifact-software.com
 wrote:

 How do you manage/allow dependency changes?
 If you are using a parent pom with a dependency management section, you
 have a single pom to watch.

 We adopted a policy that developers are not able to change dependencies
 during a release cycle.
 The versions of dependencies are a team decision under the direct control
 of the project manager and a meeting is held at the beginning of the
 release cycle to review the dependencies and settle on a stable set.
 Of course, there are emergencies from time to time but the decision to
 change is a team decision.

 This increase the stability of the development environment and makes the
 testing of upgraded dependencies happen at the start of the release cycle
 so that we know at the start that the initial code still works with
 upgraded dependencies.

 Nothing is worse or more time-consuming that have a test failure appear in
 the midst of your changing code but caused by a dependency change.

 A developer will go crazy trying to figure out why a small change to their
 code caused such a problem.

 I am not sure how you can be sure that a change in a transitive dependency
 will not cause errors higher up in the stack or create bad data structures
 that only show up later in code that has no dependency on the original
 culprit.

 Ron



 On 11/02/2015 2:57 AM, Andreas Gudian wrote:

 Hi,

 You can't do that with javac, but the takari-plugins maintain a fine
 grained dependency graph in order to do incremental builds.

 With tests, it is a different thing, though. Their runtime behaviour may
 depend on more than their class dependency might tell you: property/xml
 files, dependency injection - stuff like that.
 I think clover (code coverage tool for test) has a feature to run only
 tests for which any code has changed that has been recorded to be used in
 the previous run.

 So for real incremental tests out of the box, we'd have to support
 different strategies: compile-time dependencies, resource dependencies,
 runtime dependencies. That's quite an undertaking. For Surefire 3 we want
 to open up the API to allow attaching stuff like that from the outside.

 Andreas

 Am Mittwoch, 11. Februar 2015 schrieb Kevin Burton :

  Is there an easy way to build the Java dependency tree from the compiler?

 I was thinking that if you can get the Java dependency tree built, then
 you
 take take a look at a diff and look at which files have changed.

 Then from there you could take say 1000 test and reduce that to only 10
 test if only those ten had their dependencies changed.

 The theory being that if the previous commit already tested the previous
 990, why test them again?

 The epiphany I had was that one could EASILY integrate this into maven by
 just passing a list of which tests to skip.

 This could dramatically improve the speed of continuous integration
 systems

 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com



 --
 Ron Wheeler
 President
 Artifact Software Inc
 email: rwhee...@artifact-software.com
 skype: ronaldmwheeler
 phone: 866-970-2435, ext 102


 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Resolving classpath hell issues by allowing libraries to only use the version they request?

2015-01-05 Thread Kevin Burton
I spent a ton of time tonight in classpath hell.

Basically, Apache Spark, Cassandra, and Cassandra Unit, and Guava, Jackson
JSON, and Jetty have an INSANE dependency graph.  They're all trampling on
each other with broken
dependencies.  This results in a lot of exclusion work to get them to use
sane bindings.

And it’s not necessarily anyone’s fault.  Library A will use library B
which depend on library C version 1.1 but then library D will use library C
which depends on 1.2…

I *mostly* end up with NoClassDefFoundException.

But I’m worried about other smaller bugs.  For example, if my library
wasn’t tested with a later version of a library it isn’t necessarily as
reliable out of the box since it wasn’t tested with that config.

Has anyone done any work with only allowing a library to use the version of
a class it requested?

If you release your library fully tested on version 1.1.1 of some library,
and
someone accidentally starts using 1.1.2 becuase of dependency hell, then
we're
going to break the build and introduce bugs.

So what would happen is every dependency would only be sourced from the
version you require.

This could be done via a hierarchical class loader. Of course this would
probably end up burning up some memory but probably worth it.

The main issue, is it WORTH it.. would this introduce new bugs?

For example, it’s possible that libraries could step on each other by
creating duplicate threads with the same names, or messing up system
properties.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Property substitution in /project/name?

2015-03-17 Thread Kevin Burton
Is it ${parent.name} ?

Kevin

On Tue, Mar 17, 2015 at 3:31 AM, Pop Qvarnström pop.qvarnst...@gmail.com
wrote:

 For anyone searching for this in the archives, I created an SO question and
 a JIRA issue was promptly created.

 JIRA issue: https://jira.codehaus.org/browse/MNG-5784
 SO question:

 http://stackoverflow.com/questions/29013184/maven-support-for-property-substitution-in-project-name

 Cheers,
 Pop

 2015-03-12 14:19 GMT+01:00 Pop Qvarnström pop.qvarnst...@gmail.com:

  Hi,
 
  In a multi module project, I want to reuse the project name from the
  parent pom as part of the project name for each module. It seemed
  reasonable to use ${project.parent.name}, but it seems that this
 property
  is never substituted, i.e. anything that refers to the project name gets
  the literal string '${project.parent.name} - Some Module'.
 
  ${project.parent.name} resolves just fine in other places and with mvn
  help:evaluate.
 
  The only info I have managed to find says that property substitution is
  not allowed in /project/(groupId|artifactId|version). To me, that says
  property substitution should be allowed (and happening) for
 /project/name.
 
  Am I missing or misunderstanding something, or have I hit a bug?
 
  Maven version: 3.2.5
 
  Kind regards,
  Pop Qvarnström
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Disabling surefire error printing to console?

2015-03-08 Thread Kevin Burton
Is there away to get surefire to ONLY show the tests which failed, and NONE
of their output?

I read through the configuration and all the options that would seem to do
this are already enabled (defaults) … and I can’t find anything that would
disabled it.

I tried redirectTestOutputToFile and that doesn’t seem to solve the
situation.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Disabling surefire error printing to console?

2015-03-09 Thread Kevin Burton
It still prints the full exception to the console.

All I want is a list of the tests which failed.

On Sun, Mar 8, 2015 at 11:08 PM, Baptiste Mathus bmat...@batmat.net wrote:

 What do you mean doesn't seem to solve the situation? Doesn't it work?

 2015-03-08 20:22 GMT+01:00 Kevin Burton bur...@spinn3r.com:

  Is there away to get surefire to ONLY show the tests which failed, and
 NONE
  of their output?
 
  I read through the configuration and all the options that would seem to
 do
  this are already enabled (defaults) … and I can’t find anything that
 would
  disabled it.
 
  I tried redirectTestOutputToFile and that doesn’t seem to solve the
  situation.
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 



 --
 Baptiste Batmat MATHUS - http://batmat.net
 Sauvez un arbre,
 Mangez un castor !




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Do people actually use markdown or should I abandon it and migrate to xdocs?

2015-03-12 Thread Kevin Burton
I really wanted to live in the future but I think Markdown support in Maven
is really lacking.

1. it’s super slow.  I have a moderate sized project (10 small files) and
it takes like 10 minutes to generate my site.

2.  It’s buggy.  If you have two projects and a multi-module build you
can’t use threaded builds.

Now it looks as if my documentation generation has completely locked up.
I’m not sure what change I made but 9/10 times I run it locks up in an
infinite loop.

I imagine if no one else is actually using it then I should probably just
use what the community is running.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Do people actually use markdown or should I abandon it and migrate to xdocs?

2015-03-12 Thread Kevin Burton
Thanks.  IF these are used in big projects like this I’ll probably just
migrate.

It’s good to be pragmatic sometimes!

KEvin

On Thu, Mar 12, 2015 at 2:58 PM, Manfred Moser manf...@mosabuam.com wrote:

 I find markdown as a format just too restricted and painful to use. We had
 good success with using asciidoc for the Android Maven Plugin for site
 generation. It uses the very active asciidoctor project for rendering in
 the site.

 http://simpligility.github.io/android-maven-plugin/


 The Maven and Nexus books are also using asciidoc as a format btw.

 https://github.com/sonatype/maven-reference-en
 https://github.com/sonatype/nexus-book
 https://github.com/sonatype/maven-example-en

 Hope that helps.

 Manfred

 Kevin Burton wrote on 12.03.2015 14:11:

  I really wanted to live in the future but I think Markdown support in
 Maven
  is really lacking.
 
  1. it’s super slow.  I have a moderate sized project (10 small files) and
  it takes like 10 minutes to generate my site.
 
  2.  It’s buggy.  If you have two projects and a multi-module build you
  can’t use threaded builds.
 
  Now it looks as if my documentation generation has completely locked up.
  I’m not sure what change I made but 9/10 times I run it locks up in an
  infinite loop.
 
  I imagine if no one else is actually using it then I should probably just
  use what the community is running.
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 

 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Determining the directory of test/resources from multi-modules?

2015-03-02 Thread Kevin Burton
I have a bunch of side-files I need to maintain for my unit tests.  They’re
the results of an text extraction / mining operation.  So I need a portable
way to learn the directory of the current module from a test.

Is there an easy way to find that out?

Kevin


-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


printing the maven parallel dependency graph...

2015-02-24 Thread Kevin Burton
The documentation on maven parallelism shows a parallel dependency graph:

https://cwiki.apache.org/confluence/display/MAVEN/Parallel+builds+in+Maven+3

here is the image:

https://cwiki.apache.org/confluence/download/attachments/18153538/PastedGraphic-6.png?version=1modificationDate=1379608014000api=v2

Is it possible to get this dumped as text ?

My thinking is that I could use that to automatically configure something
like drone.io or codeship with the dependencies.

If certain modules take too long to test I can just distribute the tests
among test modules.

This would dramatically speed my testing time and I think I could keep it
at a constant time duration just by purchasing more parallelism.

Which would make my year!

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


bulk updating test output / corpus ?

2015-02-25 Thread Kevin Burton
I have a number of algorithms that output complicated data structures and I
have tests verifying the output.

Often, I’ll make one small change, which will then cascade and invalidate
all the tests causing them to fail.  The NEW output is correct but now I
need to go through and update all my tests which can take 30-50 minutes.

Is there a way to store this data in a side file and then set a flat to
reburn the side files?  This way I can validate the git diff and if it’s
correct I can commit in bulk.

Kevin

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Force a project in a multi-module build to build single threaded

2015-02-23 Thread Kevin Burton
I want to use parallelism to improve our build time … however one of our
modules uses the site plugin which has a threading issue in markdown
support.

I’d like to just force it to go single threaded since it builds in about 30
seconds.

I could of course rip it off and move it into it’s own projects but keeping
things centrally organized is one of the advantages of multi-modules.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Any work to use maven with docker or linux containers?

2015-02-24 Thread Kevin Burton
And for us this might be a week project to resolve.  We’re using embedded
zookeeper, activemq, cassandra, etc in a number of our unit tests and the
embedding frameworks don’t always have a way to change the port number.

On Tue, Feb 24, 2015 at 10:12 AM, Kevin Burton bur...@spinn3r.com wrote:

 Yes.  But while this solves this problem for this one situation,
 singletons pop up everywhere.  It might not be MY library but it might be
 something else.

 File names, database table names, port numbers.  I’m sure I’m missing
 something.

 With containers, all of this stuff is solved and I don’t need to code
 around these issues.

 On Tue, Feb 24, 2015 at 10:08 AM, Aldrin Leal ald...@leal.eng.br wrote:

 If you set the port number to zero, you'd get a random one. Try looking at
 some tests out there to find out how

 for reference:

 http://stackoverflow.com/questions/2231467/dynamically-choosing-port-number


 --
 -- Aldrin Leal, ald...@leal.eng.br
 Master your EC2-fu! Get the latest ekaterminal public beta
 http://www.ingenieux.com.br/products/ekaterminal/

 On Tue, Feb 24, 2015 at 3:05 PM, Kevin Burton bur...@spinn3r.com wrote:

  The ‘singleton’ problem with maven tests around port numbers, file
 names,
  is a big problem that’s bitten me over the years.
 
  I’d love if Maven could fork tests or with parallelism, run tests in a
  container.
 
  Right now I have two modules running tests and they are conflicting on
  ports.
 
  If they were run in containers the ports wouldn’t conflict.
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Any work to use maven with docker or linux containers?

2015-02-24 Thread Kevin Burton
The ‘singleton’ problem with maven tests around port numbers, file names,
is a big problem that’s bitten me over the years.

I’d love if Maven could fork tests or with parallelism, run tests in a
container.

Right now I have two modules running tests and they are conflicting on
ports.

If they were run in containers the ports wouldn’t conflict.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Any work to use maven with docker or linux containers?

2015-02-24 Thread Kevin Burton
Yes.  But while this solves this problem for this one situation, singletons
pop up everywhere.  It might not be MY library but it might be something
else.

File names, database table names, port numbers.  I’m sure I’m missing
something.

With containers, all of this stuff is solved and I don’t need to code
around these issues.

On Tue, Feb 24, 2015 at 10:08 AM, Aldrin Leal ald...@leal.eng.br wrote:

 If you set the port number to zero, you'd get a random one. Try looking at
 some tests out there to find out how

 for reference:
 http://stackoverflow.com/questions/2231467/dynamically-choosing-port-number


 --
 -- Aldrin Leal, ald...@leal.eng.br
 Master your EC2-fu! Get the latest ekaterminal public beta
 http://www.ingenieux.com.br/products/ekaterminal/

 On Tue, Feb 24, 2015 at 3:05 PM, Kevin Burton bur...@spinn3r.com wrote:

  The ‘singleton’ problem with maven tests around port numbers, file names,
  is a big problem that’s bitten me over the years.
 
  I’d love if Maven could fork tests or with parallelism, run tests in a
  container.
 
  Right now I have two modules running tests and they are conflicting on
  ports.
 
  If they were run in containers the ports wouldn’t conflict.
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Any work to use maven with docker or linux containers?

2015-02-24 Thread Kevin Burton
And also, you’re right about surefire… each individual test (or chunks of
say 5-10 at a time) could be run on containers too.

On Tue, Feb 24, 2015 at 10:31 AM, Kevin Burton bur...@spinn3r.com wrote:

 On Tue, Feb 24, 2015 at 10:16 AM, Aldrin Leal ald...@leal.eng.br wrote:

 Not quite - but I've been using Drone http://www.drone.io/ (which is
 wrapped around Docker) with Maven with little to no problems, although
 building up a baseline takes a little practice.


 Meaning you run maven WITHIN a container?

 I’m using Codeship.. they have this new ParallelCI which works (in theory)
 with multiple modules but I’d need to first decompose and present the
 dependencies to codeship so it could do the work.  But I think maven could
 probably provide that with about a day of work.


 Perhaps a better question would be, say, how to make Surefire/Failsafe
 fork
 not across a machine, but instead across a fleet of Docker containers.
 Viable? Yes, of course.


 YES. This is what I meant. But I actually think it’s Maven because what I
 want/need to start with is for the -T parallel option to , instead of
 forking, create a new container.


 That is a question better addressed by Yarn and Mesosphere (Kubernetes
 perhaps?)


 Yes.  Ideally each container would be on a separate piece of hardware so
 you could use 5-10 boxes to complete the tests quickly.

 Kevin

 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts
 http://spinn3r.com




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Any work to use maven with docker or linux containers?

2015-02-24 Thread Kevin Burton
On Tue, Feb 24, 2015 at 10:16 AM, Aldrin Leal ald...@leal.eng.br wrote:

 Not quite - but I've been using Drone http://www.drone.io/ (which is
 wrapped around Docker) with Maven with little to no problems, although
 building up a baseline takes a little practice.


Meaning you run maven WITHIN a container?

I’m using Codeship.. they have this new ParallelCI which works (in theory)
with multiple modules but I’d need to first decompose and present the
dependencies to codeship so it could do the work.  But I think maven could
probably provide that with about a day of work.


 Perhaps a better question would be, say, how to make Surefire/Failsafe fork
 not across a machine, but instead across a fleet of Docker containers.
 Viable? Yes, of course.


YES. This is what I meant. But I actually think it’s Maven because what I
want/need to start with is for the -T parallel option to , instead of
forking, create a new container.


 That is a question better addressed by Yarn and Mesosphere (Kubernetes
 perhaps?)


Yes.  Ideally each container would be on a separate piece of hardware so
you could use 5-10 boxes to complete the tests quickly.

Kevin

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Any work to use maven with docker or linux containers?

2015-02-24 Thread Kevin Burton
I agree.. my problem is I have a backlog of about 1000 weekend hacks :-P

On Tue, Feb 24, 2015 at 10:40 AM, Aldrin Leal ald...@leal.eng.br wrote:

 that requires resource coordination. Not a huge problem at all, given the
 right tools. I'd say it is worthy a weekend hack

 --
 -- Aldrin Leal, ald...@leal.eng.br
 Master your EC2-fu! Get the latest ekaterminal public beta
 http://www.ingenieux.com.br/products/ekaterminal/

 On Tue, Feb 24, 2015 at 3:31 PM, Kevin Burton bur...@spinn3r.com wrote:

  On Tue, Feb 24, 2015 at 10:16 AM, Aldrin Leal ald...@leal.eng.br
 wrote:
 
   Not quite - but I've been using Drone http://www.drone.io/ (which is
   wrapped around Docker) with Maven with little to no problems, although
   building up a baseline takes a little practice.
  
  
  Meaning you run maven WITHIN a container?
 
  I’m using Codeship.. they have this new ParallelCI which works (in
 theory)
  with multiple modules but I’d need to first decompose and present the
  dependencies to codeship so it could do the work.  But I think maven
 could
  probably provide that with about a day of work.
 
 
   Perhaps a better question would be, say, how to make Surefire/Failsafe
  fork
   not across a machine, but instead across a fleet of Docker containers.
   Viable? Yes, of course.
  
  
  YES. This is what I meant. But I actually think it’s Maven because what I
  want/need to start with is for the -T parallel option to , instead of
  forking, create a new container.
 
 
   That is a question better addressed by Yarn and Mesosphere (Kubernetes
   perhaps?)
  
 
  Yes.  Ideally each container would be on a separate piece of hardware so
  you could use 5-10 boxes to complete the tests quickly.
 
  Kevin
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
  http://spinn3r.com
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Running ALL tests in a multi-module , then failing the entire build.

2015-04-22 Thread Kevin Burton
For our continuous integration setup, I want each module to run tests, and
continue even if the tests failed.

This way I can see the list of ALL test failures , across all modules, then
fix them all at once.

My understanding , is that I can enable this, but then the result of the
run is a successful build?  Is that true?

what’s the easiest way to set this up?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Maven assembly pulls in FAR more dependencies than shown by dependency:test ?

2015-05-14 Thread Kevin Burton
(sorry, hit send by accident on that other one)

totally stumped here and was wondering if you guys had some feedback.

I’m building all the dependency jars for my project by using the assembly
plugin.

dependencySets
dependencySet
outputDirectorylib/outputDirectory
useProjectArtifacttrue/useProjectArtifact
scoperuntime/scope
/dependencySet
/dependencySets

which works.

the problem is that I added a new dependency, and now it’s pulling in a LOT
more than I had anticipated. FAR more than is listed in dependency:tree

More importantly, I’m getting conflicting packages.  Like Jetty 6 and Jetty
9, and conflicting servlet-api .jars and things I don’t want in the
classpath.

I can’t figure out how it’s magically doing this… I would assume these
dependencies are going to show up in the dependency tree - but they’re not.



-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Maven assembly pulls in FAR more dependencies than shown by dependency:test ?

2015-05-14 Thread Kevin Burton
totally stumped here and was wondering if you guys had some feedback.

I’m building all the dependency jars for my project by using the assembly
plugin.


-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Re: Maven assembly pulls in FAR more dependencies than shown by dependency:test ?

2015-05-14 Thread Kevin Burton
I think I figured it out… I think the dependencySet is ALWAYS using test.
Even if I set it to runtime.

I looked in the pom for the project that I’m including and the dependencies
its pulling in are from it’s test scope.

On Thu, May 14, 2015 at 1:43 PM, Kevin Burton bur...@spinn3r.com wrote:

 (sorry, hit send by accident on that other one)

 totally stumped here and was wondering if you guys had some feedback.

 I’m building all the dependency jars for my project by using the assembly
 plugin.

 dependencySets
 dependencySet
 outputDirectorylib/outputDirectory
 useProjectArtifacttrue/useProjectArtifact
 scoperuntime/scope
 /dependencySet
 /dependencySets

 which works.

 the problem is that I added a new dependency, and now it’s pulling in a
 LOT more than I had anticipated. FAR more than is listed in dependency:tree

 More importantly, I’m getting conflicting packages.  Like Jetty 6 and
 Jetty 9, and conflicting servlet-api .jars and things I don’t want in the
 classpath.

 I can’t figure out how it’s magically doing this… I would assume these
 dependencies are going to show up in the dependency tree - but they’re not.



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Re: Maven assembly pulls in FAR more dependencies than shown by dependency:test ?

2015-05-14 Thread Kevin Burton
Yes. IT seems I’m a weird type of dependency hell.  The dependencySet is
using test… and if I manually add exclusions to some of these, it
overwrites the exclusion used in the original POM.

But at least I know there’s a core bug causing all this insanity.  (I hope)

On Thu, May 14, 2015 at 1:51 PM, Kevin Burton bur...@spinn3r.com wrote:

 I think I figured it out… I think the dependencySet is ALWAYS using test.
 Even if I set it to runtime.

 I looked in the pom for the project that I’m including and the
 dependencies its pulling in are from it’s test scope.

 On Thu, May 14, 2015 at 1:43 PM, Kevin Burton bur...@spinn3r.com wrote:

 (sorry, hit send by accident on that other one)

 totally stumped here and was wondering if you guys had some feedback.

 I’m building all the dependency jars for my project by using the assembly
 plugin.

 dependencySets
 dependencySet
 outputDirectorylib/outputDirectory
 useProjectArtifacttrue/useProjectArtifact
 scoperuntime/scope
 /dependencySet
 /dependencySets

 which works.

 the problem is that I added a new dependency, and now it’s pulling in a
 LOT more than I had anticipated. FAR more than is listed in dependency:tree

 More importantly, I’m getting conflicting packages.  Like Jetty 6 and
 Jetty 9, and conflicting servlet-api .jars and things I don’t want in the
 classpath.

 I can’t figure out how it’s magically doing this… I would assume these
 dependencies are going to show up in the dependency tree - but they’re not.



 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts




 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Re: Maven assembly pulls in FAR more dependencies than shown by dependency:test ?

2015-05-15 Thread Kevin Burton
Yeah.  Agreed.  I’ll try to get some time to do this..

On Thu, May 14, 2015 at 5:09 PM, Karl Heinz Marbaise khmarba...@gmx.de
wrote:

 Hi Kevin,

 if you have really the assumption seeing a bug than it would be helpful to
 create a test project or offer the project where you can observe the
 problem so we can take a deeper look into it?

 Kind regards
 karl Heinz Marbaise


 On 5/14/15 11:22 PM, Kevin Burton wrote:

 Yes. IT seems I’m a weird type of dependency hell.  The dependencySet is
 using test… and if I manually add exclusions to some of these, it
 overwrites the exclusion used in the original POM.

 But at least I know there’s a core bug causing all this insanity.  (I
 hope)

 On Thu, May 14, 2015 at 1:51 PM, Kevin Burton bur...@spinn3r.com wrote:

  I think I figured it out… I think the dependencySet is ALWAYS using test.
 Even if I set it to runtime.

 I looked in the pom for the project that I’m including and the
 dependencies its pulling in are from it’s test scope.

 On Thu, May 14, 2015 at 1:43 PM, Kevin Burton bur...@spinn3r.com
 wrote:

  (sorry, hit send by accident on that other one)

 totally stumped here and was wondering if you guys had some feedback.

 I’m building all the dependency jars for my project by using the
 assembly
 plugin.

  dependencySets
  dependencySet
  outputDirectorylib/outputDirectory
  useProjectArtifacttrue/useProjectArtifact
  scoperuntime/scope
  /dependencySet
  /dependencySets

 which works.

 the problem is that I added a new dependency, and now it’s pulling in a
 LOT more than I had anticipated. FAR more than is listed in
 dependency:tree

 More importantly, I’m getting conflicting packages.  Like Jetty 6 and
 Jetty 9, and conflicting servlet-api .jars and things I don’t want in
 the
 classpath.

 I can’t figure out how it’s magically doing this… I would assume these
 dependencies are going to show up in the dependency tree - but they’re
 not.



 --


 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


easiest way to create debian packages using jdeb/assembly?

2015-04-12 Thread Kevin Burton
I’m currently using jdeb to create debian packages.

I use the assembly plugin to create a dependency set of .jars, then I use
jdeb to package everything together.

The problem is that it’s a HUGE chunk of boilerplate that I have to keep
copying.  We have about 10 daemons right now in a micro services
environment so it’s becoming onerous.

I tried to create a parent pom to inherit from but that doesn’t seem to
work as the assembly there doesn’t have a dependency set since the parent
doesn’t have any dependencies.

Is there a better way to go about this.  Using .dpkg is way easier in our
environment.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Computing the maven dependency graph at runtime for unit tests?

2015-04-06 Thread Kevin Burton
I have a few modules that I want to lock down so that I can easily keep
track of dependencies over time.

This way if a developer adds a new dependency, the test will immediately
break and someone will have to approve the change.

Is this possible? Could I embed this in a unit test or does it have to be a
plugin?  ideally something easy…

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Re: Easy way to tell if I have .class file conflicts between two jars?

2015-04-05 Thread Kevin Burton
Looks like it works but it’s a little too strict. I just manually resolved
some of the dependencies for one of my projects and it’s complaining about
conflicts even though the effective class path doesn’t have those class
conflicts.  hm.

On Sun, Apr 5, 2015 at 11:11 AM, Robert Scholte rfscho...@apache.org
wrote:

 Hi,

 take a look at this enforcer rule:
 http://mojo.codehaus.org/extra-enforcer-rules/banDuplicateClasses.html

 Robert

 Op Sun, 05 Apr 2015 19:57:07 +0200 schreef Kevin Burton 
 bur...@spinn3r.com:

  Im looking for a somewhat easy way to tell if a directory of .jar files
 contains class reference which conflict.

 For example,org/xml/sax/XMLReader.class from two incompatible versions.

 I wrote a simple command that does a find and then a jar tf to list the
 contents, then I sort by the filename, but this creates false positives in
 the case where the files are actually the same .class file version.

 I think I could just use SHA1 to detect this … if the SHA1 is identical I
 can just ignore it.

 Of course a WAY better way to do this would be some sort of maven plugin
 that looked at all dependencies and generated errors/warnings if you add a
 dependency with class conflicts.  That would be sweet.


 -
 To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
 For additional commands, e-mail: users-h...@maven.apache.org




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Easy way to tell if I have .class file conflicts between two jars?

2015-04-05 Thread Kevin Burton
Im looking for a somewhat easy way to tell if a directory of .jar files
contains class reference which conflict.

For example,org/xml/sax/XMLReader.class from two incompatible versions.

I wrote a simple command that does a find and then a jar tf to list the
contents, then I sort by the filename, but this creates false positives in
the case where the files are actually the same .class file version.

I think I could just use SHA1 to detect this … if the SHA1 is identical I
can just ignore it.

Of course a WAY better way to do this would be some sort of maven plugin
that looked at all dependencies and generated errors/warnings if you add a
dependency with class conflicts.  That would be sweet.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


assembly plugin dependencySet not factoring in exclusions in my dependencies.

2015-04-05 Thread Kevin Burton
I have some issues with dependency conflicts in my dependencies section in
my pom.

I need to have a way to compute a directory of .jar files for use with
creating debian packages

I’ve been using the assembly package do this this via:

dependencySets
dependencySet
outputDirectorylib/outputDirectory
useProjectArtifacttrue/useProjectArtifact
scoperuntime/scope
/dependencySet
/dependencySets

… this build a directory of .jar files I can use.

The problem is that one of my dependencies pulled in an optional .jar which
has conflicting dependencies on OLD version of a library we use.

So I have this:

dependency
groupIdorg.apache.activemq/groupId
artifactIdactivemq-leveldb-store/artifactId
version${activemq.version}/version

exclusions
exclusion
groupIdorg.mortbay.jetty/groupId
artifactId*/artifactId
/exclusion
/exclusions

/dependency

… which specifies an inclusion to not fetch this older dependency.

and if I do a dependency:tree it shows that it’s properly excluded.

BUT , once I try to run the assembly, that’s ignored and all the transitive
dependencies are pulled in.

This breaks my class path and my app now breaks because it tries to load
old versions of classes.

Any advice here?  Maybe there’s a better way to build the .jar directory
instead of the assembly plugin?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts
http://spinn3r.com


Bug with Maven 3.2.5 and threaded builds and dependencies? Corrupt maven cache.

2015-08-17 Thread Kevin Burton
Hey guys.

I think there’s a bug with Maven 3.2.5 and threaded builds and dependencies.

We have a LOT of dependencies.  I think 400 or so external .jars (not sure
if we’re an unusual case or not).

Anyway. If we have an empty cache, and do a threaded build, what happens is
there’s a race around downloading a dependency.

Thread A will try to download foo.jar and so will thread B…

There’s no mutual exclusion locking present

This means we see the following issue:

- zero byte .jars with no data (not sure how that ones possible but we see
it).
- file not found when trying to copy the .part to the final .jar because
another thread has done so already.

I haven’t tested 3.3.3 because we’re in a CI environment and I can’t
upgrade maven easily.

This will be hard to write a test for I think.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Running jdeb plugin by itself .. not attached to the package lifecycle.

2015-08-02 Thread Kevin Burton
The Maven jdeb plugin (for building debian packages) recommends you set it
up like:

executions
  execution
phasepackage/phase
goals
  goaljdeb/goal
/goals

… but this means that if I do an

mvn install

that packages are built and installed into the maven repo.

1. this doesn’t make a ton of sense.  No one is going to install our .debs
from a maven repo.

2. it significantly slows down the build.  I imagine our build could be
from 30-200% faster if I could remove this.  It’s slowing down our
continuous integration system.

What I’d like to do is run it like

mvn jdeb:jdeb

… but it doesn’t seem to have any goals associated.

Any thoughts?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Unsupported protocol: ‘http’” during maven threaded build.

2015-07-21 Thread Kevin Burton
We’re using maven 3.2.5 and recently migrated to a threaded build and I now
get this error:

[ERROR] Unsupported protocol: 'http'
[ERROR] Unsupported protocol: 'https'
[ERROR] Unsupported protocol: 'http'
[ERROR] Unsupported protocol: 'http'
[ERROR] Unsupported protocol: 'https'
[ERROR] Unsupported protocol: 'http'
[ERROR] Unsupported protocol: 'http'

I get about 40 of those and the build fails. It only happens intermittently
so I think on of the plugins isn’t thread safe.  Any recommendation of a
fix and how to resolve this?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Fetch all remote dependencies NOT in my pom?

2015-07-22 Thread Kevin Burton
I need a way to fetch all external dependencies that aren’t just a
submodule in my current project.

So things like junit, mockito, log4j, jetty, etc. I want to pull down all
that stuff.

However, I do NOT want it to include dependencies from the project.  Those
aren’t built yet so they’re not in any repo.

This is important for our continuous integration system. I want to
pre-cache these so that they’re local. It will shave 2-5 minutes off our
build time.  We have about 200 external dependencies that we pull in so
this init process takes a long time (and wastes resources).

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Re: Fetch all remote dependencies NOT in my pom?

2015-07-22 Thread Kevin Burton
I tried the following.. but that doesn’t work.  hm… stumped on this one.

mvn -DexcludeGroupIds=com.spinn3r.artemis -DexcludeTransitive=true
dependency:resolve
mvn -DexcludeArtifactIds=artemis -DexcludeTransitive=true dependency:resolve

On Wed, Jul 22, 2015 at 8:13 PM, Kevin Burton bur...@spinn3r.com wrote:

 I need a way to fetch all external dependencies that aren’t just a
 submodule in my current project.

 So things like junit, mockito, log4j, jetty, etc. I want to pull down all
 that stuff.

 However, I do NOT want it to include dependencies from the project.  Those
 aren’t built yet so they’re not in any repo.

 This is important for our continuous integration system. I want to
 pre-cache these so that they’re local. It will shave 2-5 minutes off our
 build time.  We have about 200 external dependencies that we pull in so
 this init process takes a long time (and wastes resources).

 --

 Founder/CEO Spinn3r.com
 Location: *San Francisco, CA*
 blog: http://burtonator.wordpress.com
 … or check out my Google+ profile
 https://plus.google.com/102718274791889610666/posts




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Re: Fetch all remote dependencies NOT in my pom?

2015-07-22 Thread Kevin Burton
What I want to do is just download all artifacts not in com.spinn3r.* “
group ID.

They aren’t built yet or in any repositories yet.  This is for a continuous
integration systems so I want to pre-cache them.  After this they’re just
restored from cache to build faster.


 Is your repo on the same computer as the CI system?


No.. different.


 Maven does cache dependencies locally after they are downloaded from your
 repo.


Yes.  But this is a hosted CI build system (CircleCI) so everything is a
container and wiped out each time.


-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Re: Fetch all remote dependencies NOT in my pom?

2015-07-23 Thread Kevin Burton
I didn’t try go-offline.. worth a shot!

On Wed, Jul 22, 2015 at 8:57 PM, Curtis Rueden ctrue...@wisc.edu wrote:

 Hi Kevin,

  What I want to do is just download all artifacts not in com.spinn3r.* “
  group ID.

 Did you try:

 mvn dependency:go-offline -DexcludeGroupIds=com.spinn3r ...

 ?

 Regards,
 Curtis

 On Wed, Jul 22, 2015 at 10:50 PM, Kevin Burton bur...@spinn3r.com wrote:

  What I want to do is just download all artifacts not in com.spinn3r.* “
  group ID.
 
  They aren’t built yet or in any repositories yet.  This is for a
 continuous
  integration systems so I want to pre-cache them.  After this they’re just
  restored from cache to build faster.
 
 
   Is your repo on the same computer as the CI system?
  
  
  No.. different.
 
 
   Maven does cache dependencies locally after they are downloaded from
 your
   repo.
  
 
  Yes.  But this is a hosted CI build system (CircleCI) so everything is a
  container and wiped out each time.
 
 
  --
 
  Founder/CEO Spinn3r.com
  Location: *San Francisco, CA*
  blog: http://burtonator.wordpress.com
  … or check out my Google+ profile
  https://plus.google.com/102718274791889610666/posts
 




-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Locking down dependency versions...

2015-11-12 Thread Kevin Burton
Is there a maven module that can lock down dependency versions?

I have a custom / in house script we wrote that writes a .dependencies file
with the jar dependencies.

If we commit without updating it, CI will fail with an error because you
didn't manually approve the change by regenerating the .dependencies file.

This way we don't have to worry about a radical dependency change due to a
new dependency breaking our tree.

The problem is I'm starting to break off our code into sub-projects and I'd
like to use this everywhere.

Kevin

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



mvn printing 'Running TestSuite' instead of the names of individual tests

2015-11-12 Thread Kevin Burton
I can't for the life of me figure this one out.

We did a module refactor recently and didn't change any dependencies.

Now instead of seeing something like:

Running com.spinn3r.artemis.rest.RESTServiceReferencesTest
> Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 3.58 sec -
> in com.spinn3r.artemis.rest.RESTServiceReferencesTest
> Running com.spinn3r.artemis.rest.render.RenderServletTest
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 8.675 sec
> - in com.spinn3r.artemis.rest.render.RenderServletTest
>

(AKA each individual test printed after its finished and the elapsed time.)

... now we just see:

"Running TestSuite".

... with no individual tests printed after they are finished.

This causes a problem for our CI server as it sees no output in ten minutes
and marks the build as timed out.

I tried forcing printSummary to true but that doesn't work.

We also made NO changes to use test suites...

It also seems to impact only a 1/2 of our modules.  The other ones print
each test like it did before.


-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Re: Locking down dependency versions...

2015-11-12 Thread Kevin Burton
Just regular dependency versions.

So if we're using 1.0.1 of library A I don't want adding adding library B
to transitively change our dependency on library A...

This has happened to us before and caused problems.

On Thu, Nov 12, 2015 at 1:40 PM, Karl Heinz Marbaise <khmarba...@gmx.de>
wrote:

> Hi Kevin,
>
> On 11/12/15 10:22 PM, Kevin Burton wrote:
>
>> Is there a maven module that can lock down dependency versions?
>>
>
> Are you talking about SNAPSHOT's or something different?
>
>
>> I have a custom / in house script we wrote that writes a .dependencies
>> file
>> with the jar dependencies.
>>
>> If we commit without updating it, CI will fail with an error because you
>> didn't manually approve the change by regenerating the .dependencies file.
>>
>> This way we don't have to worry about a radical dependency change due to a
>> new dependency breaking our tree.
>>
>> The problem is I'm starting to break off our code into sub-projects and
>> I'd
>> like to use this everywhere.
>>
>> Kevin
>>
>>
>
> Kind regards
> Karl Heinz Marbaise
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>


-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>


How easy/reliable is a maven repo hosted on webdav?

2015-11-04 Thread Kevin Burton
I need to setup a new maven repo due to some changes we're making in our CI
framework.

Basically, I want something simple.. I was looking at bintray and
artifactory and I think they are overkill for what we need and could be
rather expensive per year.

We already have plenty of hardware so I just want something simple.

We used to use SCP but I don't want to give our CI hosting provider / SAAS
the ability to auth into our cluster.

WebDAV seems ideal because it could just work with plain old apache.

However, I can't find any documentation for how to set this up.

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Copying resources from another .jar with assembly to build a deb?

2015-11-05 Thread Kevin Burton
I have init scripts and various other scripts I want to share between
projects.

The MAIN one is our init script for our daemons.

so if I have a project I want to depend on artemis-daemon which has our
daemon scripts.

Then I want to copy some resources from that .jar while building a dpkg via
jdeb...

In the past I've just done a copy ../ and used a parent directory to read a
module.

But I can't do that now since I'm breaking projects apart.

Is there an easy way to handle this?

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Re: Copying resources from another .jar with assembly to build a deb?

2015-11-05 Thread Kevin Burton
I'm trying this plugin:

https://maven.apache.org/plugins/maven-remote-resources-plugin/examples/sharing-resources.html

... and seems to work well!

On Thu, Nov 5, 2015 at 11:29 AM, Wayne Fay <wayne...@gmail.com> wrote:

> I could be wrong, but I think you are looking for
> maven-dependency-plugin:unpack.
>
> Wayne
>
> On Thu, Nov 5, 2015 at 12:42 PM, Kevin Burton <bur...@spinn3r.com> wrote:
> > I have init scripts and various other scripts I want to share between
> > projects.
> >
> > The MAIN one is our init script for our daemons.
> >
> > so if I have a project I want to depend on artemis-daemon which has our
> > daemon scripts.
> >
> > Then I want to copy some resources from that .jar while building a dpkg
> via
> > jdeb...
> >
> > In the past I've just done a copy ../ and used a parent directory to
> read a
> > module.
> >
> > But I can't do that now since I'm breaking projects apart.
> >
> > Is there an easy way to handle this?
> >
> > --
> >
> > We’re hiring if you know of any awesome Java Devops or Linux Operations
> > Engineers!
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile
> > <https://plus.google.com/102718274791889610666/posts>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>


-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>


Re: How easy/reliable is a maven repo hosted on webdav?

2015-11-05 Thread Kevin Burton
On Wed, Nov 4, 2015 at 10:36 PM, Anders Hammar  wrote:

> A real repository manager is not overkill but rather the basic foundation
> in a Maven-based dev environment. And there are free versions of them:
> nexus oss, artifactory and archiva.
> I strongly advice against trying to do this with your home-brewed solution!
>


I'm starting to appreciate that perspective but then again it's another
component in our infra that can fail, requires maintenance, etc.

Additionally there's replication , multiple datacenters, etc.


> --
>
We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Submodule without parent can't be version:set in version plugin 2.2

2015-10-11 Thread Kevin Burton
I have a multi-module project where some of my modules don't have parents
referenced.

if I change to the directory of that module and then run:

mvn versions:set -DnewVersion=5.1.508

I get an exception (see below).

It seems to construct the path wrong.  It uses the name of the submodule
twice (you can see in the path below).

I think this is a bug... I think the issue is that I don't have it setup as
a normal submodule with a referenced 

This has worked in the past and I think it's a supported configuration.

For now I'm just using search/replace to bump my module versions but it's
somewhat annoying.

Is there a workaround or is this just a bug?

java.io.FileNotFoundException:
/home/burton/projects/parent_module/submodule/submodule (No such file or
directory)
at java.io.FileInputStream.open0(Native Method)
at java.io.FileInputStream.open(FileInputStream.java:195)
at java.io.FileInputStream.(FileInputStream.java:138)
at org.codehaus.plexus.util.xml.XmlReader.(XmlReader.java:124)
at
org.codehaus.plexus.util.xml.XmlStreamReader.(XmlStreamReader.java:67)
at
org.codehaus.plexus.util.ReaderFactory.newXmlReader(ReaderFactory.java:118)
at org.codehaus.mojo.versions.api.PomHelper.readXmlFile(PomHelper.java:1613)
at
org.codehaus.mojo.versions.AbstractVersionsUpdaterMojo.process(AbstractVersionsUpdaterMojo.java:319)
at org.codehaus.mojo.versions.SetMojo.execute(SetMojo.java:251)
at
org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:134)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:208)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153)
at
org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:116)
at
org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:80)
at
org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build(SingleThreadedBuilder.java:51)
at
org.apache.maven.lifecycle.internal.LifecycleStarter.execute(LifecycleStarter.java:128)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:307)
at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:193)
at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:106)
at org.apache.maven.cli.MavenCli.execute(MavenCli.java:862)
at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:286)
at org.apache.maven.cli.MavenCli.main(MavenCli.java:197)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:497)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Launcher.java:289)
at
org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.java:229)
at
org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(Launcher.java:415)
at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java:356)


-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



publishing sub-modules without requiring the parent module to be published?

2015-10-11 Thread Kevin Burton
I'm trying to open source a bunch of code in our stack.

Is it possible to have a parent module private, and the child module public?

The problem I have now is that if our customers or the open source
community imports our modules, it needs to fetch the parent, which is
private.

I assume this is required due to maven inheritance because it needs to
determine dependencies.

I can just not specify a  but that causes other problems like the
maven versions problem skipping it for some reason (though maybe this is a
bug).



-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Re: publishing sub-modules without requiring the parent module to be published?

2015-10-12 Thread Kevin Burton
That's a good point.  It's a tough call because we're trying to move as
fast as possible but this means that the code can bitrot and not get
updates from the community.

The other thing I was thinking about was using the maven updates plugin to
automatically detect when new versions are released.

The problem with this is that releases get cascaded as libs move up the
dependency tree.  We have about 150 modules so this could get confusing  if
we're not careful.

On Mon, Oct 12, 2015 at 5:00 AM, Ron Wheeler <rwhee...@artifact-software.com
> wrote:

> You are probably going to want to make your open source project completely
> separate with its own versions and parent.
>
> If third parties are actually going to file issues,  provide updates and
> create new minor versions as they fix bugs, you are going to either be
> forced to release versions of your private project before you are ready or
> have to fork your own open source project to maintain a consistent version
> between the 2 parts of your product.
>
> Having a successful open source project requires a new way of thinking
> about your baby.
>
> Ron
>
>
>
> On 12/10/2015 2:49 AM, Baptiste Mathus wrote:
>
>> I'm pretty sure you can't. At least on Central who IIRC checks that kind
>> of
>> things.
>> It's indeed needed for the dependency resolution mechanism to work.
>> Cheers
>> Le 11 oct. 2015 11:50 PM, "Kevin Burton" <bur...@spinn3r.com> a écrit :
>>
>> I'm trying to open source a bunch of code in our stack.
>>>
>>> Is it possible to have a parent module private, and the child module
>>> public?
>>>
>>> The problem I have now is that if our customers or the open source
>>> community imports our modules, it needs to fetch the parent, which is
>>> private.
>>>
>>> I assume this is required due to maven inheritance because it needs to
>>> determine dependencies.
>>>
>>> I can just not specify a  but that causes other problems like the
>>> maven versions problem skipping it for some reason (though maybe this is
>>> a
>>> bug).
>>>
>>>
>>>
>>> --
>>>
>>> We’re hiring if you know of any awesome Java Devops or Linux Operations
>>> Engineers!
>>>
>>> Founder/CEO Spinn3r.com
>>> Location: *San Francisco, CA*
>>> blog: http://burtonator.wordpress.com
>>> … or check out my Google+ profile
>>> <https://plus.google.com/102718274791889610666/posts>
>>>
>>>
>
> --
> Ron Wheeler
> President
> Artifact Software Inc
> email: rwhee...@artifact-software.com
> skype: ronaldmwheeler
> phone: 866-970-2435, ext 102
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>


-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>


Best way to lock in .jar versions between releases.

2015-10-13 Thread Kevin Burton
I want to lock in the effective classpath for our releases.

So if extra .jars or versions of .jars are changed, I want the build to
fail until I manually approve it.

For some reason, our version of cassandra regressed and broke on release.

Still trying to track this down but in the future it would be nice to just
flat out prevent this from happening.

Thoughts?

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Re: Best way to lock in .jar versions between releases.

2015-10-13 Thread Kevin Burton
Wow. Lots of questions!

I ended up working around this by just writing a post build script that
generates a build fingerprint of our .jars

I then commit the fingerprint.  If a build changes the fingerprint then we
can't push.

It's kind of ugly but we unfortunately have a post build script anyway

On Tue, Oct 13, 2015 at 1:15 PM, Wayne Fay <wayne...@gmail.com> wrote:

> Forget transitive dependencies. Pretend you're back on Ant. All
> artifacts must be declared. Leave nothing to chance.
>
> Declare all dependencies, including the ones you are currently
> bringing in transitively, in your project's pom.
>
> Set all versions with [1.2.3] to "lock" them down.
>
> There may be another way to do this, but this is what I'd suggest to
> start...
>
> Wayne
>
> On Tue, Oct 13, 2015 at 1:08 PM, Kevin Burton <bur...@spinn3r.com> wrote:
> > I want to lock in the effective classpath for our releases.
> >
> > So if extra .jars or versions of .jars are changed, I want the build to
> > fail until I manually approve it.
> >
> > For some reason, our version of cassandra regressed and broke on release.
> >
> > Still trying to track this down but in the future it would be nice to
> just
> > flat out prevent this from happening.
> >
> > Thoughts?
> >
> > --
> >
> > We’re hiring if you know of any awesome Java Devops or Linux Operations
> > Engineers!
> >
> > Founder/CEO Spinn3r.com
> > Location: *San Francisco, CA*
> > blog: http://burtonator.wordpress.com
> > … or check out my Google+ profile
> > <https://plus.google.com/102718274791889610666/posts>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>


-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>


forking between flakey tests retries ?

2015-08-30 Thread Kevin Burton
Is there a way to fork between flakey tests?

I think the new surefire plugin changes retry the test in the same process.

I’m using surefire to fork each test so I can release daemon resources as
part of integration tests.

We have integration / unit tests for zookeeper, cassandra, elastic search,
etc.

Sometimes they don’t startup due to sockets/ports/etc…

I *though* the flakey test rerunning would fix it but it won’t reform I
think.

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
https://plus.google.com/102718274791889610666/posts


Best way to keep test resources with junit / surefire /maven.

2015-10-03 Thread Kevin Burton
I'm trying to figure out if there's a better way to do this.

Java doesn't support "here files" or embedded files so you have to keep
your resources external.  Usually in 'resources',

I don't want to have things like HTML code and other things in my tests as
it makes the tests ugly.

But the problem is that once I create the resources, they can't participate
in refactoring.

At least in intellij... even if it's *OBVIOUS* that the resource is
associated with a given test.

Is there a cleaner way to do this?

Either this whole thing is broken or I'm doing it wrong.

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Implementing “distcc” (distributed builds) for Maven.

2015-09-20 Thread Kevin Burton
I’m curious if anyone has worked on distributed builds and testing for
maven.

It seems like a lot of the framework is there.

At the very minimum the parallel builds framework could by used by
implementing a maven backend daemon system and build protocol.

I think at a minimum it would work with the version control plugins to pull
on a worker node.. then have maven copy any locally changed files, then do
the build remotely, then transfer the resulting artifacts back to the local
machine.

For small builds I don’t think anyone would notice any performance change.
For large builds the performance could be substantial.

I think the trick would be to make it work with tools like IntelliJ so that
the remote builds would just be triggered and automagically work.

Thoughts?

-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Re: Best strategy for using lots of modules/projects with some private and some OSS

2015-09-22 Thread Kevin Burton
Weird. Looks like the blog.artifact-software.com isn’t responding to DNS .
At least on our network..

On Tue, Sep 22, 2015 at 11:29 AM, Ron Wheeler <
rwhee...@artifact-software.com> wrote:

> You have clearly given this a lot of thought and your project is just a
> bit less than twice as large as ours so I am not sure about giving advice.
>
> http://blog.artifact-software.com/tech/?p=84 is an article that I wrote a
> few years ago dealing with our approach to the same problem.
>
> Our solution to the version management problem may not be to your tastes
> but it did save us a lot of grief and served us well once we got through
> the first few releases where we were touching every piece of source code on
> each release.
>
> We essentially started treating our own code as if it was OSS.
> This made us think about modularity and functional separation into
> projects and changed our ideas about versions.
>
> Ron
>
>
>
> On 22/09/2015 1:47 PM, Kevin Burton wrote:
>
>> We have a multi-module setup whereby we have about 150 independent
>> modules.
>>
>> Our build takes a long time and actually slows down development as we have
>> to do a compile of a LOT of source code to rebuild the project.
>>
>> Additionally, we have a lot of code that we want to Open Source.
>>
>> This has meant git submodules the IMO git submodules really don’t work
>> when
>> using branches.  They break and require a whole bunch of custom works and
>> hack and when they DO break it’s confusing how to resolve them.
>>
>> This has meant that we’ve not really done a good job of OSSing our code
>> base as its just too hard.
>>
>> What we’ve done to date is just have one major version number across all
>> our projects.  So upgrading them and fixing their dependencies means that
>> I
>> just have to change a version number everywhere and I’m done.
>>
>> What I was thinking of is changing this strategy to use the maven
>> "versions:use-latest-versions” plugin.
>>
>> What i would do is have a parent directory named ‘spinn3r’ which just has
>> a
>> bunch of git submodules.  We NEVER branch in this directory.
>>
>> It also means that any of our developers can check it out so that they
>> have
>> all of our source code.
>>
>> At this point I can use a normal development strategy for each project.
>> They don’t use submodules which enables us to branch/merge easily.
>>
>> I can also have a dedicated IntelliJ or Eclipse project for each one and
>> switch between them.
>>
>> Now the main issue I have is how do I bump releases easily and make sure
>> all my code is using the latest version of its sibling projects.
>>
>> In the parent directory I can just run versions:use-latest-versions … on
>> each one of the projects so that it automatically pulled in the latest
>> version after a release.
>>
>> The only problem here is that there’s a dependency graph that needs to be
>> considered.
>>
>> for example, if project A depends on project B, then we have to bump the
>> version and push project B into maven before we upgrade dependencies on
>> project A.
>>
>> This is a frustrating issue…
>>
>>
>
> --
> Ron Wheeler
> President
> Artifact Software Inc
> email: rwhee...@artifact-software.com
> skype: ronaldmwheeler
> phone: 866-970-2435, ext 102
>
>
> -
> To unsubscribe, e-mail: users-unsubscr...@maven.apache.org
> For additional commands, e-mail: users-h...@maven.apache.org
>
>


-- 

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile
<https://plus.google.com/102718274791889610666/posts>


Best strategy for using lots of modules/projects with some private and some OSS

2015-09-22 Thread Kevin Burton
We have a multi-module setup whereby we have about 150 independent modules.

Our build takes a long time and actually slows down development as we have
to do a compile of a LOT of source code to rebuild the project.

Additionally, we have a lot of code that we want to Open Source.

This has meant git submodules the IMO git submodules really don’t work when
using branches.  They break and require a whole bunch of custom works and
hack and when they DO break it’s confusing how to resolve them.

This has meant that we’ve not really done a good job of OSSing our code
base as its just too hard.

What we’ve done to date is just have one major version number across all
our projects.  So upgrading them and fixing their dependencies means that I
just have to change a version number everywhere and I’m done.

What I was thinking of is changing this strategy to use the maven
"versions:use-latest-versions” plugin.

What i would do is have a parent directory named ‘spinn3r’ which just has a
bunch of git submodules.  We NEVER branch in this directory.

It also means that any of our developers can check it out so that they have
all of our source code.

At this point I can use a normal development strategy for each project.
They don’t use submodules which enables us to branch/merge easily.

I can also have a dedicated IntelliJ or Eclipse project for each one and
switch between them.

Now the main issue I have is how do I bump releases easily and make sure
all my code is using the latest version of its sibling projects.

In the parent directory I can just run versions:use-latest-versions … on
each one of the projects so that it automatically pulled in the latest
version after a release.

The only problem here is that there’s a dependency graph that needs to be
considered.

for example, if project A depends on project B, then we have to bump the
version and push project B into maven before we upgrade dependencies on
project A.

This is a frustrating issue…

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Re: Best strategy for using lots of modules/projects with some private and some OSS

2015-09-24 Thread Kevin Burton
Pretty impressive.  I think another issue I hadn't thought of is the
overhead of having a CI server/project for each project.

If anything, you have to enumerate each project so you can see inside of it
and what's happening.

Having one macro one means you can see everything in one place but of
course that has drawbacks too.

On Tue, Sep 22, 2015 at 3:28 PM, Ben Podgursky <bpodgur...@gmail.com> wrote:

> +1 for aggressive SNAPSHOT use
>
> Our setup.  YMMV:
>
> - We have ~75 independent projects and each of them is versioned and
> deployed independently.  Few million lines of code.
> - We have CI jenkins builds + unit test suites for every project, and we
> deploy SNAPSHOT artifacts as soon as a build succeeds.
> - Our OSS projects follow the same pattern (https://github.com/liveramp/).
> We do releases every so often for external users, but internally we use
> snapshots which build for every new commit.
> - Every internal dependency is on a SNAPSHOT version.
> - All developers have all code checked out in intellij, source linked
> - If you push changes to a library, you update all downstream usages.  No
> exceptions.  Since you have all source in intellij, this usually isn't
> hard.
>
> Our use-case is a bit different in that we don't really have "application
> versions", since everything is back-end data pipeliens.  But when we
> deploy, we build an artifact with locked SNAPSHOT versions which passes
> integration tests and deploy that artifact.  We deploy many projects
> several times a day.
>
> It takes a bit of diligence, but the payoff of not having to manage
> internal versioning and have formal releases is huge.
>
> On Tue, Sep 22, 2015 at 1:14 PM, Ron Wheeler <
> rwhee...@artifact-software.com
> > wrote:
>
> > +1 (probably better and more complete than my description)
> >
> > Has anyone else looked at using an installer like izPack for assembling
> > test setups?
> > It integrates with Maven and will pick up all the right versions of jars
> > and configuration files and build an installer that will drop the whole
> set
> > where ever you want to test.
> > You can build an installer that will install on different OSs so you can
> > take the same test jar and run it on Windows, Linux or a Mac and get the
> > "right" configuration files for the target OS.
> >
> > Uses maven dependency management to get the jars from your repo and uses
> a
> > command language in XML to assemble the rest of your supporting data and
> > configuration files.
> >
> > A bit smarter than Ant about building application run-time structures but
> > that is all it does.
> >
> > Ron
> >
> >
> > On 22/09/2015 3:43 PM, Curtis Rueden wrote:
> >
> >> Hi Kevin,
> >>
> >> My projects opt for independent versioning of modules to facilitate
> >> "release early, release often." To do this for large sets of components
> >> like yours requires a Bill of Materials -- i.e., common parent POM with
> >> dependencyManagement section.
> >>
> >> FWIW, the docs we have about our projects that work this way are at:
> >> * http://imagej.net/Architecture
> >>
> >> And in particular:
> >> * http://imagej.net/Architecture#Bill_of_Materials
> >> *
> http://imagej.net/Architecture#How_SciJava_achieves_reproducible_builds
> >> * http://imagej.net/Philosophy#Release_early.2C_release_often
> >>
> >> And the BOM stuff is at:
> >> *
> >>
> >>
> https://github.com/scijava/pom-scijava/blob/pom-scijava-8.3.0/pom.xml#L103-L819
> >>
> >> The downside, as you point out, of all components being release version
> >> coupled is that it is annoying to have to do a "release cascade" to
> >> propagate a bug fix from the lowest level components to the highest
> level
> >> ones. We have some tooling to make that easier (I personally live in the
> >> "releases should be as easy/automated/fast as possible" camp), but the
> >> modularity does cost time sometimes. Hopefully a lot less time than
> >> building your huge multi-module project from scratch every time, though!
> >>
> >> I also recently wrote a "melting pot" script to do end-to-end testing of
> >> large component collections:
> >>
> >>
> >>
> https://github.com/scijava/scijava-scripts/blob/d892adc0092c220ee1e597b9fb5a1fb067e4509b/melting-pot.sh
> >>
> >> This script builds and runs unit tests for all components of a large
> >> collection at their respective versions, all in the sa

Dealing with .jars that have overlapping .classes

2016-04-22 Thread Kevin Burton
We have a rather complex classpath in our app... sometimes maven computes
the wrong classpath because one package changes classpath order and has a
dependency on an earlier jar.

To avoid this we usually analyze our classpath between builds and our
builds break if a classpath change isn't approved.

One of the BIG problems here is when we get overlapping classes.

Right now this is happening with joda time and joda convert.  WE have about
10 classes that are in both .jars.

First. It seems irresponsible for project maintainers to release artifacts
like this. If there are shared .classes just publish a new -core artifact.

Just being pragmatic.. I'm not sure the BEST way to handle this.

One strategy is to make sure that each lib is the LATEST .. but sometimes
that's not possible.

How do you guys handle this?

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Ability to run all tests in all modules, and fail if ANY test fails.

2016-08-02 Thread Kevin Burton
I have a COMPLEX multi module build.

Like 250 sub-modules.

If I do a big refactor the build stops after testing say 20 modules, then I
have to fix the build again, then it might get through 5 more modules, I
have to keep repeating this process.

What I want to do is to run tests on ALL modules and then show me which
fail AT THE END.

I also want to the build to FAIL at the end.

There is -fn , but this cause the build to SUCCEED with failures and a zero
(success) exit code.

Is there a way I can make it run ALL tests , across ALL modules, and then
fail if ANY of the tests failed?

Right now our CI system won't support maven failing with a zero exit code.

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Plugin specific command line options?

2017-01-27 Thread Kevin Burton
Is there a way to pass command line options but only have one plugin see it?

My understanding is that plugins just read from system properties.

So to pass 'skip' to a plugin I would use -Dskip but what if two plugins
use the same 'skip' ?

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Re: Using maven profiles for easily parallel testing?

2016-11-06 Thread Kevin Burton
>
>
> I'm thinking of a clunky approach of including tests with names starting
> with [A-Ma-m] in one build configuration, and those with names starting
> with [^A-Ma-m] in another build configuration. These build configuration
> could have most of their actual configuration in a build template, and only
> specify the differences separately, which would be the regex.
>
>
Yeah.. I've thought about that too.

The big thing you have to worry about is if all your tests end with or
start with "Test" which is why I was thinking of the hashing approach.

But I guess this will work as long as you're careful.


> My organization is also using TeamCity and Maven, and is about to embark
> on using (again) build chains in TeamCity, which would allow to define
> these two (or whichever many you desire) build configurations, and pull
> them together somehow.
>
>
I think this is the hard part.  One would needs to merge all the test
reports in Teamcity. Not sure if it supports this already.

How do you use chains to get them to work in parallel?  I thought that this
would only work if you do them sequentially - which won't yield much of a
speedup.

But I do think the 'dependent build configuration' thing might work.

Right now we first run an incremental build, and that then fires up a full
build.

Maybe one solution is to first compile all the code, and then have that
trigger N chained builds at the same time.

This way you quickly confirm that the code compiles, and then that can
trigger all the secondary builds.

Of course that might be redundant and just waste 1-2 minutes.  Might be
better to just have a quick 'return true' script that always works and then
have that kick off your two parallel builds.

Maybe aggregating the tests output isn't strictly necessary since Teamcity
shows them in the UI...

We would be using this approach to split up, and parallelize unit tests vs.
> (module) integration tests vs. some end to end tests (cross-module
> integration tests). Some of these tests live in a separate Maven module
> already, which I'd call a best practice for cross-module integration tests.


Yeah.. we do the same thing.


> I'm curious to hear what you come up with and/or settle on. Please report
> back!
>

Will do.. actually I think talking this through made it seem a lot easier.



-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Re: Using maven profiles for easily parallel testing?

2016-11-06 Thread Kevin Burton
>
>
> Is this really the time for the tests only or is this the whole build time
> including the test time? If it is only the time for running the tests it
> means those tests are slow...
>
>
Yeah.  I agree.  The slow ones are all essentially integration tests.

Some take 20-30 seconds as they involve things like stopping daemons like
zookoeeper / cassandra which just take a while.

At the 95th percentile all of our tests are <1ms


> Having a log file of the maven run available you can summarize the run
> time of the tests only by using something like this (on a linux like
> system):
>
> This will summarize the time for the tests only...
>
>
>
Yet another reason I love Teamcity... it does this automatically and has an
easy report :)


>
>> We use -T 16 on our tests and our boxes have 8 cores so this allows some
>> tests to block on IO while others execute.
>>
>
>
> -T 16 tries to parallellize the module build and not parallize the
> tests...furthermore I have my doubts that using -T 16 is faster than -T 4
> etc..
>
>
Yeah.. but the modules are running parallel so  you have as many parallel
tests as modules in that iteration.

Looking at our CPU we're pretty much at 100% across all our tests for the
entire time.


> I don't know how many modules you have in your build. If you have really
> many modules like 100+ it might be a good idea to consider using Smart
> Builder[1].
>
> If you like to parallize the unit tests themself you have to configure
> maven-surefire-plugin accordingly...I recommend reading the doc about that
> subject[2].
>

Ah.. interesting. Those changes should be incorporated back into maven
proper...

But we will try this out.  I think though in practice it won't make much of
a change for us.

We have like 200 modules and our build says nearly 100% the entire time.

Kevin

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Using maven profiles for easily parallel testing?

2016-11-05 Thread Kevin Burton
We use TeamCity internally (which is great btw) for Maven testing.

We have about 2000 tests which we continually integrate on every commit.

The problem is that testing takes about 15 minutes from start to end.

We use -T 16 on our tests and our boxes have 8 cores so this allows some
tests to block on IO while others execute.

However, I'm not happy with this.  I want our tests to finish in 2 minutes.

The only way this could happen is for us to hand parallelize everything (no
fun), or do it automatically, or buy REALLY expensive hardware (no fun).

I think one could do this with JUnit categories and maven profiles.

I think what one could do is hash the name of the test and then based on
the hash prefix, create N buckets.

Right now we have 4 CI boxes so we would run 4 profiles, one per box.  Then
TeamCity could integrate the results.

I think this is technically possible now but I think a lot of plugins would
need to cooperate here.

It would be better if maven didn't need to explicitly have to split these
out.

Maybe update surefire to have a way to only run 1/Nth of the tests and make
the set determinstic?

This way you can use two build boxes and each have two sets of tests with
no intersection.

Thoughts on this approach?  I think it would be pretty awesome.

I'm sure someone here knows the gang at TeamCity and could recommend this
to the proper decision makers.

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



Are there any standard junit categories / projects?

2016-11-07 Thread Kevin Burton
I want to migrate to using junit categories in our build.

Are there any projects/modules that implement standard categories like
Online, Integration, etc.

I don't see any in junit directly...

Otherwise I have to build a new module and it has to be outside our main
pipeline which is no fun.

-- 

We’re hiring if you know of any awesome Java Devops or Linux Operations
Engineers!

Founder/CEO Spinn3r.com
Location: *San Francisco, CA*
blog: http://burtonator.wordpress.com
… or check out my Google+ profile



  1   2   >