[JIRA] (JENKINS-39932) Support more arbitrary Pipeline definitions
Title: Message Title
Andrew Melo created an issue
Jenkins / JENKINS-39932
Support more arbitrary Pipeline definitions
Issue Type:
Bug
Assignee:
Andrew Bayer
Components:
pipeline-model-definition-plugin
Created:
2016/Nov/22 8:54 AM
Priority:
Minor
Reporter:
Andrew Melo
I think the idea of a declarative (and not imperative) pipeline is a lot easier on the writers end (and I assume the implementers end as well), but I think the implementation of several serial stages each fanning out to a number of parallel stages is limiting to the overall description of the dependencies between different steps/nodes. In my case (simplifying, please excuse pseudo-code), I'd like to do something like this:
stage("checkout") { node () { git XXX } }
stage("build") {
parallel ("ubuntu") { node("ubuntu") { make XXX }},
("centos") { node("centos") { make YYY }},
// tons more distributions here
}
stage ("test") {
node ("ubuntu") { test-ubuntu ZZZ }
}
stage ("publish") { node() { publish.sh }
Since I know the "test" stage only depends on the "ubuntu" part of the build stage, I'd like the test stage to fire without waiting on the whole "build" stage completes. From what I see, the parallel block waits for everything inside to complete, but I know the dependencies and can speed it up (in my case, there are a lot more nodes who match the testing than who don't). If I were to write it in bash, I would do something like:
git checkout XXX
build ubuntu &
TEST_PID=$!
build centos &
[JIRA] (JENKINS-33185) Visualize parallel steps within a Pipeline Stage
Title: Message Title Andrew Melo commented on JENKINS-33185 Re: Visualize parallel steps within a Pipeline Stage Yessir, Java's not one of my proficiencies, and I failed at even trying to build Jenkins on my local (OSX) machine, but I'm willing to try again to help scratch my itch. I'll write a JIRA against the above component and give it another shot to try and help. Cheers! Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-33185) Visualize parallel steps within a Pipeline Stage
Title: Message Title Andrew Melo commented on JENKINS-33185 Re: Visualize parallel steps within a Pipeline Stage Sorry, James Durmay – I'm not super familiar with JIRA quoting. I can rewrite/edit if it makes it clearer. Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-33185) Visualize parallel steps within a Pipeline Stage
Title: Message Title Andrew Melo commented on JENKINS-33185 Re: Visualize parallel steps within a Pipeline Stage I think the visualization is a second-order optimization, but at least firing off the subordinate nodes would speed things up a lot. And ,as a script writer, if I could hint to you who goes where visually, that would simplify your job Add Comment This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-33185) Visualize parallel steps within a Pipeline Stage
Title: Message Title
Andrew Melo commented on JENKINS-33185
Re: Visualize parallel steps within a Pipeline Stage
I like it! My only hitch would be trying to describe the dependency that wasn't a series of stages (forgive my pseudo-code) where things would block unnecessarily: stage("checkout") { git checkout XXX } node("linux-ubuntu") { stage("compile") { XXX} } node("linux-centos") { stage ("compile") { XXX } ; stage ("test") { YYY } } It's a contrived example, but I currently do something like: state("checkout") { git checkout XXX } stage("compile") { parallel { node ("linux-ubuntu") { XXX } node ("linux-centos") { XXX } } stage ("test") { node ("linux-centos") { YYY } } which means every part of "compile" has to finish before the "test" phase happens, even if there are available executors. If there's a rewrite in a declarative language, I'd hope that the "test" parts could at least start before all the "compile" stages start. I know it's not what's exposed in the current or blue ocean implementations, but it would be nice to fire off additional nodes/stages without waiting on unnecessary parallel branches.
Add Comment
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] (JENKINS-33185) Visualize parallel steps within a Pipeline Stage
Title: Message Title
Andrew Melo commented on JENKINS-33185
Re: Visualize parallel steps within a Pipeline Stage
I understand completely it's hard to try and arrange a display for the results of a turing-complete programming language, but perhaps the compromise is allowing the programmer define where things should go, instead of having the language try to divine out the positioning post-facto. For instance (pseudo code, please be gentle) stage("checkout",0,0) { git checkout } stage("build",1,0) { build windows} stage("build",1,1) { build linux} stage("deploy",2,0) { deploy windows} stage("deploy",2,1) { deploy linux} Since the person writing the code knows (ostensibly) where things should go, you could just trust them to make the right grid-like display of each node/step
Add Comment
This message was sent by Atlassian JIRA (v7.1.7#71011-sha1:2526d7c)
--
You received this message because you are subscribed to the Google Groups "Jenkins Issues" group.
To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/d/optout.
[JIRA] [workflow-plugin] (JENKINS-31831) Allow Workflow Snippet Generator to be used outside of a configuration page
Title: Message Title Andrew Melo commented on JENKINS-31831 Re: Allow Workflow Snippet Generator to be used outside of a configuration page I hit the same issue as squalou jenkins. It's nice to have the snippet generator outside of a configuration page, but having it next to the actual configuration box was much more convenient to use. Is there any reason it can't be at both locations? Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [github-branch-source-plugin] (JENKINS-34966) GH branch source uses wrong commit
Title: Message Title Andrew Melo created an issue Jenkins / JENKINS-34966 GH branch source uses wrong commit Issue Type: Bug Assignee: Jesse Glick Attachments: jenkins_indexing.pdf, merge.pdf Components: github-branch-source-plugin, workflow-multibranch-plugin, workflow-plugin Created: 2016/May/20 3:49 AM Environment: Jenkins master using the official docker image updated to 2.5, all plugins updated to latest version as of 22:30CDT, May 19th. GitHub Branch Source Plugin 1.7 GitHub Organization Folder Plugin 1.3 Pipeline 2.1 Labels: 2.0 workflow Priority: Major Reporter: Andrew Melo This looks similar to, but distinct from #34727. Somehow the branch source plugin gets confused about what is the "most recent" commit for a PR, and
[JIRA] [pipeline-stage-view-plugin] (JENKINS-33185) Visualize parallel steps within a Pipeline Stage
Title: Message Title Andrew Melo commented on JENKINS-33185 Re: Visualize parallel steps within a Pipeline Stage How would that visualization look if you have tons of things running in parallel that aren't necessarily related? For instance, I run the packaging step immediately, with the tests in parallel (to reduce the overall latency). Would I end up with one very tall column? Add Comment This message was sent by Atlassian JIRA (v6.4.2#64017-sha1:e244265) -- You received this message because you are subscribed to the Google Groups "Jenkins Issues" group. To unsubscribe from this group and stop receiving emails from it, send an email to jenkinsci-issues+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
[JIRA] [dockerhub-notification-plugin] (JENKINS-34063) java.lang.ClassCastException in DockerHubTrigger Plugin
Title: Message Title Andrew Melo created an issue Jenkins / JENKINS-34063 java.lang.ClassCastException in DockerHubTrigger Plugin Issue Type: Bug Assignee: Unassigned Components: dockerhub-notification-plugin Created: 2016/Apr/06 5:32 PM Environment: Jenkins 2.0-beta1 from dockerhub Labels: docker Priority: Minor Reporter: Andrew Melo Trying to enable the "trigger on docker hub updates" box in a multibranchproject yields: javax.servlet.ServletException: java.lang.ClassCastException: org.jenkinsci.plugins.workflow.multibranch.WorkflowMultiBranchProject cannot be cast to hudson.model.Job at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:796) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at org.kohsuke.stapler.MetaClass$5.doDispatch(MetaClass.java:233) at org.kohsuke.stapler.NameBasedDispatcher.dispatch(NameBasedDispatcher.java:58) at org.kohsuke.stapler.Stapler.tryInvoke(Stapler.java:746) at org.kohsuke.stapler.Stapler.invoke(Stapler.java:876) at
[JIRA] (JENKINS-13430) JUnitParses blows up on directories with cycles
Andrew Melo created JENKINS-13430: - Summary: JUnitParses blows up on directories with cycles Key: JENKINS-13430 URL: https://issues.jenkins-ci.org/browse/JENKINS-13430 Project: Jenkins Issue Type: Bug Components: core Reporter: Andrew Melo Hello, Our workspace has various symlinks going in different directions, so simple directory scanning ends up in infinite recursion. I get the following output at the end of all my test jobs: Build step 'Execute shell' marked build as failure Recording test results ERROR: Failed to archive test reports hudson.util.IOException2: remote file operation failed: /jenkins/workspace/WMCore-UnitTests-try/jobSlice/7/label/wmcore-unit-test-slaves at hudson.remoting.Channel@1e4d9c84:slc5-64-02 at hudson.FilePath.act(FilePath.java:828) at hudson.FilePath.act(FilePath.java:814) at hudson.tasks.junit.JUnitParser.parse(JUnitParser.java:83) at hudson.tasks.junit.JUnitResultArchiver.parse(JUnitResultArchiver.java:122) at hudson.tasks.junit.JUnitResultArchiver.perform(JUnitResultArchiver.java:134) at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19) at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:705) at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:680) at hudson.model.AbstractBuild$AbstractRunner.performAllBuildSteps(AbstractBuild.java:658) at hudson.model.Build$RunnerImpl.post2(Build.java:162) at hudson.model.AbstractBuild$AbstractRunner.post(AbstractBuild.java:627) at hudson.model.Run.run(Run.java:1438) at hudson.matrix.MatrixRun.run(MatrixRun.java:146) at hudson.model.ResourceController.execute(ResourceController.java:88) at hudson.model.Executor.run(Executor.java:238) Caused by: java.io.IOException: Remote call on slc5-64-02 failed at hudson.remoting.Channel.call(Channel.java:655) at hudson.FilePath.act(FilePath.java:821) ... 14 more Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded at java.util.Arrays.copyOf(Arrays.java:2894) at java.lang.AbstractStringBuilder.expandCapacity(AbstractStringBuilder.java:117) at java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:589) at java.lang.StringBuilder.append(StringBuilder.java:220) at java.io.UnixFileSystem.resolve(UnixFileSystem.java:108) at java.io.File.init(File.java:329) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1256) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) at org.apache.tools.ant.DirectoryScanner.scandir(DirectoryScanner.java:1287) Finished: FAILURE In my (limited) ability to poke around ant it seems like a slowScan() needs to be performed, but I can't figure out
[JIRA] (JENKINS-13334) Matrix builds should alphabatize their test result failures
Andrew Melo created JENKINS-13334: - Summary: Matrix builds should alphabatize their test result failures Key: JENKINS-13334 URL: https://issues.jenkins-ci.org/browse/JENKINS-13334 Project: Jenkins Issue Type: Bug Components: matrix Reporter: Andrew Melo Priority: Trivial Hello, Matrix builds concatenate their subbuild test failures (between different subbuilds) but don't alphabatize them, which makes it hard to find affected test-cases. See: http://cl.ly/0n3V1O0b2A2F322v180W Thanks, Andrew -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-13335) Matrix builds should alphabatize their test result failures
Andrew Melo created JENKINS-13335: - Summary: Matrix builds should alphabatize their test result failures Key: JENKINS-13335 URL: https://issues.jenkins-ci.org/browse/JENKINS-13335 Project: Jenkins Issue Type: Bug Components: matrix Reporter: Andrew Melo Priority: Trivial Hello, Matrix builds concatenate their subbuild test failures (between different subbuilds) but don't alphabatize them, which makes it hard to find affected test-cases. See: http://cl.ly/0n3V1O0b2A2F322v180W Thanks, Andrew -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-13335) Matrix builds should alphabatize their test result failures
[ https://issues.jenkins-ci.org/browse/JENKINS-13335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Melo resolved JENKINS-13335. --- Resolution: Duplicate sorry, duplicate ticket Matrix builds should alphabatize their test result failures --- Key: JENKINS-13335 URL: https://issues.jenkins-ci.org/browse/JENKINS-13335 Project: Jenkins Issue Type: Bug Components: matrix Reporter: Andrew Melo Priority: Trivial Hello, Matrix builds concatenate their subbuild test failures (between different subbuilds) but don't alphabatize them, which makes it hard to find affected test-cases. See: http://cl.ly/0n3V1O0b2A2F322v180W Thanks, Andrew -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
[JIRA] (JENKINS-6747) Support concurrent runs of matrix builds
[ https://issues.jenkins-ci.org/browse/JENKINS-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=159645#comment-159645 ] Andrew Melo commented on JENKINS-6747: -- I'm in the same boat as the others. We actually have access to a giant VM farm where we can spin up (nearly) unbounded numbers of VMs, but the matrix builds don't take advantage of that. I'm looking to see if I can't at least make the flyweight build run concurrently. Support concurrent runs of matrix builds Key: JENKINS-6747 URL: https://issues.jenkins-ci.org/browse/JENKINS-6747 Project: Jenkins Issue Type: Improvement Components: concurrent-build Reporter: Eric Smalling Assignee: Kohsuke Kawaguchi I'd like to be able to run matrix builds in parallel, currently that option doesn't appear in the configuration so I'm assuming it's not implemented yet. :) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators: https://issues.jenkins-ci.org/secure/ContactAdministrators!default.jspa For more information on JIRA, see: http://www.atlassian.com/software/jira
