I think would be "easy" in a scripted pipeline compared to a declarative pipeline. The reason I say that is because you can use loops directly in your pipeline, so if you knew the number of frames, you could iterate over the number of frames with 4 as an increment and create parallel steps for each iteration (https://jenkins.io/doc/pipeline/examples/#parallel-from-list). You'd use the iterator variable as the start index and the increment as the number of frames. You could then even make the number of frames a parameter to your job. I am not sure how this could be done with declarative, I am still new to using declarative.
On Tue, Dec 18, 2018 at 1:56 PM 'Mark Raynsford' via Jenkins Users < [email protected]> wrote: > Hello! > > I've been using Jenkins to build code from hundreds of git repositories > using declarative pipelines. In the setup I'm using, I have a small > number of nodes, and all nodes must successfully build and run all > tests in order for the job as a whole to have been considered > successful. > > I'd now like to try using Jenkins to render animations using Blender[0]. > > The problem that I'm actually trying to solve is that Blender (like all > 3D renderers) generally takes a long time to render images. When > rendering an animation, each frame is rendered individually and saved > to a PNG file. When all frames have been rendered, the PNG files can be > stitched together with something like ffmpeg to produce a video file. > Given that there's no data dependency between frames, it's possible to > distribute the workload over many machines by giving each machine a set > of frame numbers to render and then collecting the images from all > machines when they've all finished rendering. > > Blender has a full command-line interface, so the 'sh' declaration > in the pipelines will be enough to launch instances of Blender for > rendering. I can place the data to be rendered in a Git repository so > that all nodes can pull from that repository to start building. I would > include a pipeline script in the repository that each node can use to > build. This is how I've been handling building code, and it's been > working very well. > > The problem: It's not clear to me how I could assign a set of frame > numbers to each node in the pipeline. I could hard code the numbers > into something that the pipeline code reads in the first stage of > building, but it seems like to do that effectively, I'd need to know > all of the nodes ahead of time (meaning I'd have to rewrite the data if > I add or remove a node later). > > What I would really like is for the pipeline in each node to be told > "You are node 1 of 4" "You are node 3 of 4", and so on. Node 3 could > then, for example, use this information to tell Blender "Start > rendering at frame 3, and step forward by 4 frames each time" (4 > because there are 4 nodes in total). If each node did this, the entire > range of frames would be covered and the work would be split equally > between the four nodes. The nodes I have are homogeneous, so there's no > need to try to give powerful nodes more work, and weaker nodes less. > > Any advice would be appreciated! > > [0] https://www.blender.org/ > > -- > Mark Raynsford | http://www.io7m.com > > -- > You received this message because you are subscribed to the Google Groups > "Jenkins Users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/jenkinsci-users/20181218205530.767b8097%40almond.int.arc7.info > . > For more options, visit https://groups.google.com/d/optout. > -- Website: http://earl-of-code.com -- You received this message because you are subscribed to the Google Groups "Jenkins Users" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/jenkinsci-users/CAPiUgVdbvm8oz%2BWt5k1KuJtKcH8J3LfFw2q5Hd0bXr20rXt4zA%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
