Some of the points you raise were talked about (briefly) in the review of the spot instance support code; as you have noted, they are primarily driven by the nature of spot instances.
Using SSH to contact the spot slaves would be quite difficult without repeatedly polling the spot request to see when it has been fulfilled and then getting the instance to extract its DNS name/IP address. Unfortunately the AWS API has no means of 'pushing' information to API users. I could see SQS or SNS being a useful mechanism, and in fact we could even use that for on-demand slaves to avoid having to repeatedly attempt to open an SSH session. If the plugin put some sort of token into the userdata for the instance, and then the instance used one of those services to emit a notification containing that token, the plugin would have a reliable indication that the instance is ready for use (and the notification could include connection details, so that SSH could be used to connect to the instance). This would help quite a lot with slaves that take a long time to become ready for SSH. Also, the EC2 plugin does not currently support Windows slaves very well; it's possible to use them by installing an SSH daemon on them. Until the spot instance support was added, there was no support for JNLP slaves on EC2, but now that we have that it's possible that on-demand Windows slaves could use that mechanism, although as you say getting the proper security credentials communicated to the instance could be tricky. It would be much better from a security point of view to use WMI to open a connection to the Windows instance and create a login session for running slave.jar, like we use SSH. ----- Original Message ----- From: [email protected] To: [email protected] Cc: [email protected] At: May 22 2013 06:38:06 Hi, As a user of the EC2 plugin, I'm greatly interested in the support of spot instances. First, I want to thank you for that contribution! I've played a bit with new features merged in 1.19-SNAPSHOT and I have a few questions and remarks. I understand the need of a notification from the slave to the master, thus the use of JNLP (from slave to master) instead of SSH (from master to slave) initially used by the ec2 plugin. But the current implementation is not usable in my context for various reasons. *JNLP connection may require credentials depending on the security strategy. So the ubuntu-init.py script has to manage those credentials. *How to set the credentials on the slave side? * *Should they be setup by the plugin via userdata? Easy to implement but maybe not perfect to send them for each spot request. *Set them in the slave image (AMI)? If so, the credentials location has to be defined by the plugin. *JNLP port must be configurable. Indeed, the whole slave configuration needs to be configurable. * *Wouldn't it be possible to define a slave template which will be used by the plugin instead of copying almost all fields from the slave edit page to the ec2 configuration? *How to fallback to SSH? Finally, JNLP is not satisfying at all in my case and I would like the master to initiate the slave connection via SHH. What do you think about the following solutions: * *Allow the master to repeatedly ask for the slave (pull instead of push; far from perfect but may be useful in some cases). *Make the slave notify/ping the master (like GitHub triggering) via HTTP(s). *Make use of SQS (http://aws.amazon.com/fr/documentation/sqs/) or SNS (http://docs.aws.amazon.com/sns/latest/dg/welcome.html) Please, tell me if I'm wrong, if you have better ideas and if some parts of that work are already initialized. I'll be happy to contribute some of those improvements. PS: for now, I'm only using Linux slaves spot instances. I'll be looking at setting up Windows slaves latter (and in that case, I may have to use JNLP as a service since it's the easier way I succeeded in managing that OS...). Regards, Julien -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups "Jenkins Developers" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
