Hi, I'm afraid there's a lot to consider. > That would require starting up a server first, and somehow getting it to report its port# to us. > But the server configs must know port#s in advance because they need to talk to each other. Bootstrapping from leaf servers up the dependency tree should be possible. Cyclic dependencies could get in the way. Cyclic dependencies usually are a bad idea. But that's off-topic.
> Currently we use the range of 9000-9020. On a build/test machine we don't worry about conflicts, > a developer has no business running conflicting software at the same time as building/testing > our software. And of course, the baseport 9000 can be moved at will. Well, well. I don't subscribe to the idea that anyone can dictate what a developer would run at the same time and what constitutes such software. IDEs might use ports. The system itself uses ports. Other software might use ports. The developer might want to build/test two version of the software in parallel at the same time to do comparative analysis and comparative debugging. The definition of the port range for ephemeral ports could change. There's just so much that could go wrong. In my experience, assuming the availability of ports leads to flaky tests. Developers can lose a lot of time to get tests running if they first have to resolve port conflicts. I get that you want to use job identifiers for this. But the mechanism is just not made for that. That the job identifier is a number is only by coincidence. Job identifiers are identifiers first, and numbers only coincidentally. If someone decides, for whatever reason, to replace them with UUIDs, so what? I'm not saying that using UUIDs would be a good idea in this case, it's just about making the point about the difference between an identifier and a number. But I totally get that you need something that helps you to get unique numbers. Have you considered using a simple shell script that counts up and writes the count into a named pipe, and then having each job read one line from that named pipe to get "your" number? This should do the job: $ mkfifo counter $ for ((uniqueNumber = 0; uniqueNumber < 1000; uniqueNumber++)) ; do echo $uniqueNumber >counter ; done & You can then get a - for that run - unique number using $ head -n 1 counter It should be easy to do this from a Makefile. If you do this directly from the Makefile, this might require SHELL:=/bin/bash, recursive make, and an exit handler. Here is how you could use an exit handler from within a Makefile: https://stackoverflow.com/questions/28597794/how-can-i-clean-up-after-an-error-in-a-makefile/52159940#52159940 I hope that this helps you and provides you with a pathway solution that allows you even more control without accidental coupling to internal features of make. And while I think that using make is great, this solution would even work if you use something else than make. Q'aplah! On Sat, Aug 14, 2021 at 5:57 PM Howard Chu <h...@highlandsun.com> wrote: > Christian Hujer wrote: > > Hi, > > > > Have you considered using port 0 instead to tap into the ephemeral port > range and communicate that port somehow? > > That would require starting up a server first, and somehow getting it to > report its port# to us. > But the server configs must know port#s in advance because they need to > talk to each other. > > > > From what I understand, you want to use the job id as an offset to a > base port to choose a port from a range, right? That sounds non-portable to > me, spelling > > all sorts of ports conflict trouble. > > Currently we use the range of 9000-9020. On a build/test machine we don't > worry about conflicts, > a developer has no business running conflicting software at the same time > as building/testing > our software. And of course, the baseport 9000 can be moved at will. > > > > Christian > > > > On Fri, Aug 13, 2021, 18:44 Howard Chu via Bug reports and discussion > for GNU make <bug-make@gnu.org <mailto:bug-make@gnu.org>> wrote: > > > > In my original jobserver implementation, I filled the job pipe with > > up to 256 bytes, numbers from 0-255. The idea being that then make > > could distinguish each job with a unique ID number. That idea got > > thrown away when we decided to raise the limit to PIPEBUF (4096 on > > traditional Unix, much larger now on Linux). > > > > I'm looking for a way to expose a job ID number to the individual > > jobs, in a consecutive range from 1-[number of jobs]. Not just unique > > numbers, for that we could just use $$ already. The purpose is to > > e.g. assign non-overlapping network port numbers when firing off a > > number of client/server test scripts in parallel. > > > > Is there anything that could do this now, exposed as a Make variable? > > -- > > -- Howard Chu > > CTO, Symas Corp. http://www.symas.com > > Director, Highland Sun http://highlandsun.com/hyc/ > > Chief Architect, OpenLDAP http://www.openldap.org/project/ > > > > > -- > -- Howard Chu > CTO, Symas Corp. http://www.symas.com > Director, Highland Sun http://highlandsun.com/hyc/ > Chief Architect, OpenLDAP http://www.openldap.org/project/ > -- Christian Hujer CEO/CTO Nelkinda Software Craft Pvt Ltd 📧 Christian Hujer <christian.hu...@nelkinda.com> | 🐦 @nelkinda <https://twitter.com/nelkinda> | 🌐 http://nelkinda.com ☏ 🇮🇳 ✆ +91 77 2003 6661 <+917720036661>
