On Tue, Apr 30, 2024 at 12:26 PM Sascha Brawer via Cloud <[email protected]> wrote: > > Dear all, > > Has anyone figured out how to use Toolforge Build Service for a tool (job + > webservice) written in golang?
I got <https://gitlab.wikimedia.org/toolforge-repos/bridgebot> working last week, and in doing so I struggled with the same things you are reporting today. > I'm trying to switch qrank.toolforge.org to Toolforge Build Service, but I'm > struggling with it. > To debug this, I wrote a minimal job that just prints a hello message to > stdout: > https://github.com/brawer/wikidata-qrank/blob/main/cmd/hello/main.go > > Because Toolforge is emulating Heroku, it appears necessary to tell buildpack > which binaries should get packaged into the container. The "// +heroku > install" comment on line 14 of the following go.mod file seems to do the > trick, together with listing the binaries again in a Heroku Procfile: > https://github.com/brawer/wikidata-qrank/blob/main/go.mod#L14 > https://github.com/brawer/wikidata-qrank/blob/main/Procfile > > When running the following command on login.toolforge.org, the project seems > to be built just fine: > $ toolforge build start https://github.com/brawer/wikidata-qrank > > Some interesting log message from build service: > [step-build] 2024-04-30T18:13:31.170521374Z Building packages: > [step-build] 2024-04-30T18:13:31.170585858Z - ./cmd/hello > [step-build] 2024-04-30T18:13:31.170599159Z - ./cmd/qrank-builder > [step-build] 2024-04-30T18:13:31.170608711Z - ./cmd/webserver > [step-build] 2024-04-30T18:13:44.563334902Z > [step-build] 2024-04-30T18:13:44.563404532Z [Setting launch table] > [step-build] 2024-04-30T18:13:44.563417703Z Detected processes: > [step-build] 2024-04-30T18:13:44.563427021Z - hello: hello > [step-build] 2024-04-30T18:13:44.563437573Z - qrank-builder: qrank-builder > [step-build] 2024-04-30T18:13:44.563446463Z - webserver: webserver > [step-build] 2024-04-30T18:13:44.563488376Z - web: hello > [step-build] 2024-04-30T18:13:44.577068596Z > [step-build] 2024-04-30T18:13:44.577163104Z [Discovering process types] > [step-build] 2024-04-30T18:13:44.579933448Z Procfile declares types -> web, > qrank-builder, hello > > The "web: hello" in the launch table looks very suspicious; it's not what my > Procfile states (see link above). I noticed similar behavior when building my project. My sensemaking is that the golang buildpack is configured to always generate a "web" process because of Heroku's internal needs. In our stack Profile processing is done by a separate buildpack that the golang buildpack is not aware of. > But at least the build seems to have been successful: > > $ toolforge build show > Build ID: qrank-buildpacks-pipelinerun-vhcrt > Start Time: 2024-04-30T18:13:03Z > End Time: 2024-04-30T18:14:00Z > Status: ok > Message: Tasks Completed: 1 (Failed: 0, Cancelled 0), Skipped: 0 > Parameters: > Source URL: https://github.com/brawer/wikidata-qrank > Ref: N/A > Envvars: N/A > Destination Image: tools-harbor.wmcloud.org/tool-qrank/tool-qrank:latest > > Now, I'd be happy to just run the "hello" job and see its output log. > However, when I do this: > $ toolforge jobs run --command hello --image tool-qrank/tool-qrank:latest > --mount=all --filelog > > The job runs for about five minutes (?!?!), and then: > $ cat hello.out > ERROR: failed to launch: bash exec: argument list too long > > So clearly something is off... but what? How to run a golang tool on > Toolforge when using Build Service? I ran into this behavior as well and reported it as <https://phabricator.wikimedia.org/T363417>. The workaround that I was able to find so I could move forward was to create a very slim shell script wrapper to launch my process from the Procfile: <https://gitlab.wikimedia.org/toolforge-repos/bridgebot/-/commit/f05aa13d2309a55d5db286e7a38da45facb53b8a> In your case I think that you might be able to just put `/layers/heroku_go/go_target/bin/hello` as the command you want to run in either the Procfile or as the `--command` argument to `toolforge jobs`. You can use `webservice --backend=kubernetes buildservice shell` to get an interactive shell inside of a container running the image you have built. This can be useful in debugging things. We have another task in the backlog about making this shell functionality a bit more discoverable now that we also have need for it with the jobs framework (<https://phabricator.wikimedia.org/T311917>). I hope that all helps a bit. It is frustrating to be using new tools and run into such troubles. Thank you for taking the time to look for help on the mailing list though! Sharing our troubles and wins with each other helps build our community. Bryan -- Bryan Davis Wikimedia Foundation Principal Software Engineer Boise, ID USA [[m:User:BDavis_(WMF)]] irc: bd808 _______________________________________________ Cloud mailing list -- [email protected] List information: https://lists.wikimedia.org/postorius/lists/cloud.lists.wikimedia.org/
