Great investigation, and thanks for reporting back the results! The bootstrap BSD looks like a nice recipe to have. I'd add a statement class for it to the preconfigured Statements [1] so users can easily use that when composing their scripts. And the ComputeService could also add that automatically to BSD nodes when needed.
Also, being able to add a user to the sudores file would be a nice addition to the SudoStatements [2]. Mind opening a JIRA to track this? And even better, are you up to contributing the changes in a pull request? :) Thanks! I. [1] https://github.com/jclouds/jclouds/tree/master/scriptbuilder/src/main/java/org/jclouds/scriptbuilder/statements [2] https://github.com/jclouds/jclouds/blob/master/scriptbuilder/src/main/java/org/jclouds/scriptbuilder/statements/login/SudoStatements.java El 3/2/2016 12:07 a. m., "Klemen Ferjančič" <imba...@gmail.com> escribió: > The problem is that FreeBSD /bin/sh really does not like DOS style line > breaks so it interprets \r as part of the command name, therefore the > weird "not found" messages. I ended up with replacing \r\n with \n in > java on the fly, then uploading script to the server and executing it > with a one liner: > > su - root -c "chmod +x /home/ec2-user/bootstrap_freebsd.sh && > /home/ec2-user/bootstrap_freebsd.sh" > > bootstrap_freebsd.sh content: > #!/bin/sh > pkg install -y sudo > pkg install -y bash > ln /usr/local/bin/bash /bin/bash > echo "ec2-user ALL=(ALL) NOPASSWD:ALL" >> /usr/local/etc/sudoers > > > You need to install sudo, bash, add ec2-user to sudoers and also symlink > the bash path because jclouds uses /bin/bash hardcoded in the init files. > > After these 4 lines are executed you can proceed executing commands with > init script and sudo as normal. > > Hopefully this helps someone in the future as crazy as I who wants to > run FreeBSD. :) > > Best regards, Klemen > > > On 29. 01. 2016 14:55, Ignasi Barrera wrote: > > That one-liner thing is really weird! Glad to see you found a way to > > bypass that. > > > > Another option would be to put your script in a file, upload it to the > > node using the jclouds ssh client, and then run a script that simply > > executes that script file. Could that work and be cleaner? Have a look > > at the "Advanced usage" section of the compute guide [1] for an > > example of how to upload a file to a node. > > > > I. > > > > [1] http://jclouds.apache.org/start/compute/ > > > > On 28 January 2016 at 23:14, cen <imba...@gmail.com> wrote: > >> Found the solution. If I execute commands in a single line using &&, > written > >> as a java string they work as expected. > >> But reading them from a file where I write them line by line it cause > the > >> problems mentioned previously: > >> > >> Files.toString(file, Charsets.UTF_8) > >> > >> If I write them in a single line in the file it works as expected too. I > >> tried to print the commands when they are line by line > >> and everything seems normal in console. Maybe it's the line breaks in > >> combination with tcsh or something in that regard causing the problem. > >> > >> Plus another find: you can't actually su root && command, the proper > way is > >> su - root -c "commands && here". su root seems to break the flow. > >> > >> > >> Ignasi Barrera je 28. 01. 2016 ob 11:12 napisal: > >>> > >>> I have no experience with BSD instances, but could it be a PATH issue? > >>> > >>> Can you try setting the PATH manually? > >>> > >>> On 28 January 2016 at 09:13, Klemen Ferjančič <imba...@gmail.com> > wrote: > >>>> > >>>> It seems I've hit a wall. > >>>> > >>>> 1. For some reason, even when disabling init script on template > builder, > >>>> it still tries to run it. So I decided to skip it for now. > >>>> > >>>> > >>>> > .wrapInInitScript(false).runScript(getBootInstructions(os)).runAsRoot(true); > >>>> > >>>> 2. Running as root and disabling sudo does not actually remove sudo. > >>>> Taking a closer look at RunScriptOnNodeUsingSsh.java->execAsRoot, it > >>>> seems that sudo is always run in front. > >>>> > >>>> > >>>> > opts.wrapInInitScript(false).runAsRoot(true).overrideAuthenticateSudo(false) > >>>> .overrideLoginCredentials(getLoginForCommandExecution(os)); > >>>> > >>>> Result: > >>>> `sudo sh <<'RUN_SCRIPT_AS_ROOT_SSH' ... > >>>> > >>>> which means I can't run as root. > >>>> > >>>> 3. So I decided to run as regular user (not root and not sudo) and > just > >>>> switch to root manually. Works if I do it by hand: > >>>> su root > >>>> pkg install -y sudo > >>>> pkg install -y bash > >>>> > >>>> Opts: > >>>> .wrapInInitScript(false).runAsRoot(false) > >>>> .overrideLoginCredentials(getLoginForCommandExecution(os)) > >>>> > >>>> But for some reason, it seems that jclouds puts me in some weird > >>>> restricted shell where I can't execute anything. > >>>> su returns "Sorry" and trying to run a simple command like "whoami" > >>>> results in: not found. It's clear to me that this is not the same > >>>> environment as the one > >>>> when I manually ssh into the machine so what is the difference? > >>>> > >>>> > >>>> Best regards, Klemen > >>>> > >>>> On 27. 01. 2016 11:06, Ignasi Barrera wrote: > >>>>> > >>>>> No. You can use the one in the template builder to run the first one > and > >>>>> the run the rest by calling computeService.runScriptOnNode > afterhaving > >>>>> created the node. > >>>>> > >>>>> El 27/1/2016 11:03 a. m., "Klemen Ferjančič" <imba...@gmail.com > >>>>> <mailto:imba...@gmail.com>> escribió: > >>>>> > >>>>> If I add additional scripts on top of template builder are they > >>>>> going to > >>>>> be executed in order or is the solution more complex than this? > >>>>> > >>>>> Something like: > >>>>> > >>>>> > >>>>> > templateBuilder.wrapInInitScript(false).runScript(installBashAndSudo()) > >>>>> .wrapInitScript(true).runScript(everythingElse()) > >>>>> > >>>>> On 27. 01. 2016 10:56, Ignasi Barrera wrote: > >>>>> > Yes, the RunScriptOptions provide options to run the scripts > as > >>>>> root, > >>>>> > and also to explicitly disable sudo. > >>>>> > > >>>>> > Currently the wrapper script requires bash, so you'll have to > >>>>> install > >>>>> > it first if you need to use it. You could split the runscript > in > >>>>> two > >>>>> > operations: a first one with the wrapInitScript(false) to just > >>>>> install > >>>>> > bash, and a second one to run the desired script. > >>>>> > > >>>>> > That wrapper script, as said, contains several helpers to let > >>>>> jclouds > >>>>> > query the status of the script execution: see if it is still > >>>>> running, > >>>>> > allow to abort it, etc. If you are running scripts that take > >>>>> time, I'd > >>>>> > recommend you run them with the wrapper script. > >>>>> > > >>>>> > I. > >>>>> > > >>>>> > On 27 January 2016 at 10:48, Klemen Ferjančič < > imba...@gmail.com > >>>>> <mailto:imba...@gmail.com>> wrote: > >>>>> >> Good advice, I suspect I know what is going on. > >>>>> >> > >>>>> >> Home directory contains bootstrap and /tmp/init-bootstrap > >>>>> exists. > >>>>> >> However, if I run ./init-bootstrap it says "not found". I > >>>>> checked the > >>>>> >> init script and it seems that bash is expected to exist on > the > >>>>> system > >>>>> >> (#!/bin/bash), however, bash does not come preinstalled on > >>>>> FreeBSD. I > >>>>> >> could install bash with runScript but I don't think personal > >>>>> script is > >>>>> >> executed before the init one? Another interesting note: sudo > is > >>>>> not > >>>>> >> preinstalled either so I probably need to run my script as > root. > >>>>> I'll > >>>>> >> play around with this and see what I can do. > >>>>> >> > >>>>> >> Code snippet: > >>>>> >> > >>>>> >> templateBuilder.osFamily(OsFamily.FREEBSD); > >>>>> >> templateBuilder.imageId(REGION + "/ami-9f5549f3"); > >>>>> >> EC2TemplateOptions o = > >>>>> EC2TemplateOptions.Builder.keyPair("mykeypair") > >>>>> >> > >>>>> > >>>>> > .overrideLoginCredentials(getLoginForCommandExecution(os)).runScript(getBootInstructions(os)); > >>>>> >> templateBuilder.locationId(REGION); > >>>>> >> templateBuilder.hardwareId(INSTANCE); > >>>>> >> templateBuilder.options(o); > >>>>> >> > >>>>> >> > >>>>> >> On 27. 01. 2016 10:23, Ignasi Barrera wrote: > >>>>> >>> Hi Klemen, > >>>>> >>> > >>>>> >>> jclouds generates and uploads that script to the node. It > is a > >>>>> wrapper > >>>>> >>> script that provides some helpers to get the status of the > >>>>> configured > >>>>> >>> script, and to properly capture the output, stderr, and > make it > >>>>> >>> possible to abort its execution. > >>>>> >>> > >>>>> >>> After the failure, can you log in to the instance and see > which > >>>>> files > >>>>> >>> do you have in the user's home directory and in /tmp? > >>>>> >>> > >>>>> >>> You can also disable the wrapper script by configuring the > >>>>> >>> "wrapInitScript(false)" in the RunScriptOptions, and see if > the > >>>>> >>> problem persists. > >>>>> >>> > >>>>> >>> Could you also share a small code snippet of the code you're > >>>>> using to > >>>>> >>> bootstrap the node? > >>>>> >>> > >>>>> >>> I. > >>>>> >>> > >>>>> >>> On 27 January 2016 at 09:39, Klemen Ferjančič > >>>>> <imba...@gmail.com > >>>>> <mailto:imba...@gmail.com>> wrote: > >>>>> >>>> Hi > >>>>> >>>> > >>>>> >>>> When I create a new Ec2 instance with FreeBSD private AMI, > the > >>>>> >>>> init-bootstrap fails to run. The instance is created and > runs > >>>>> normally, > >>>>> >>>> but the error also fails my own runScript so I'd like to > >>>>> resolve it. > >>>>> >>>> > >>>>> >>>> 00:32:27,588 ERROR [jclouds.compute] (user thread 0) << > >>>>> problem > >>>>> >>>> customizing node(eu-central-1/i-22cd4c9e): : > >>>>> >>>> java.lang.IllegalStateException: error running > >>>>> [/tmp/init-bootstrap > >>>>> >>>> init] as ec2-user@52.59.247.117 > >>>>> <mailto:ec2-user@52.59.247.117>; returnVal !=0: > >>>>> >>>> {output=/tmp/init-bootstrap: not found > >>>>> >>>> , error=, exitStatus=127) > >>>>> >>>> > >>>>> >>>> Full stacktrace: http://pastebin.com/xabqprs5 > >>>>> >>>> > >>>>> >>>> 1. Is /tmp/init-bootstrap supposed to already exist on the > >>>>> image or is > >>>>> >>>> this a script that jclouds uploads to the node? If the > latter, > >>>>> I assume > >>>>> >>>> there is a problem with script not being uploaded in the > first > >>>>> place? No > >>>>> >>>> error indicates this though. > >>>>> >>>> 2. What does this script actually do? Is it possible to > >>>>> disable it? > >>>>> >>>> > >>>>> >>>> Best regards, Klemen > >>>>> >>>> > >>>>> >>>> > >>>>> >> > >>>>> > >> > >
