I thought about the reservation/commit model you mentioned. Indeed FlexLM has a very simple support of license reservation by project. My idea is as follows:
1. in the vendor option file, reserve the number of licenses (features, in term of flexlm) configured in SLURM. For example, create a project name with a random string (in case a user can easily guess it and use it to checkout licenses), and reserve the proper number of licenses. This ensures that SLURM has the configured licenses. 2. on job resource allocation, create a project name according to the job id, and reserved the allocated licenses to the project. The application must set the environment LM_PROJECT to the project name to checkout licenses. 3. on job resource deallocation, the licenses reserved to the project of the job is taken back(reservation in vendor option file deleted and lmreread executed). This is not a very good approach because a user may cheat by guessing LM_PROJECT environment variables. It could work if the users are all well behaved. -----原始邮件----- 发件人: "Gary Brown" <[email protected]> 发送时间: 2013-07-03 00:06:15 (星期三) 收件人: slurm-dev <[email protected]> 抄送: 主题: [slurm-dev] Re: slurm integration with FlexLM license manager Three years ago I tried to work with Flexera Software (FlexLM) to resolve race conditions that arose between a scheduler and FlexLM because the FlexLM license manager was also serving licenses to external users; i.e., the scheduler was not the only one trying to obtain licenses. I proposed a "reservation/commit" model similar to that used by the credit card industry to handle charges where a retail establishment will obtain an "authorization" for a specific amount, which the credit card system "reserves" against a customer's credit limit, and then when the retail establishment "settles" the charge, the reserved amount is actually added to the customer's credit card balance and the "authorization" deleted. This would properly handle the situation where a scheduler "reserves" licenses through FlexLM and the a running job actually "checks out" the reserved licenses. Despite the company and product names, Flexera was completely inflexible and would not do anything in this direction since its customers, the Independent Software Vendors (ISVs), would actually sell fewer software licenses under this model, which is what users actually want, and Flexera's customers would take a very dim view of Flexera if it implemented this model. No logic (cloud model also needs this), cajoling, or begging would get Flexera to budge. I do not know if Flexera has done anything to resolve the issue of race conditions between when a scheduler tries to schedule licenses and when a job actually checks the licenses out during which interval an external user checks out licenses unbeknownst to the scheduler, but I suspect they have done nothing. If anyone hears of anything different, I, for one, would be happy to know. Gary D. Brown On Tue, Jul 2, 2013 at 8:38 AM, David Bigagli <[email protected]> wrote: Indeed currently there is no integration between Flexlm and SLURM, but some ideas are being passed around what to do about it. I am one of the original designers and developers of Platform License Scheduler. The item 1) you mentioned is certainly the first step but consider even that may not be easy, just imagine an electronic design application that is running in the cluster and jobs checking in and out hundreds of features per second. It is important to choose which features has to be managed by the scheduler and it has to be 'well behaved' one, meaning the behavior of the application from license perspective has to be well know. One of the difficulties is to understand how the application uses the licenses as you observed in item 2). The only way to get license information out of Flexlm is indeed lmstat, which could be quite slow if the license servers and handling many applications there is no other supported interface, a possible alternative could be parsing the lmgrd log file. /David On Tue, Jul 2, 2013 at 2:57 PM, Hongjia Cao <[email protected]> wrote: I don't think there is integration with FlexLM in SLURM. There is a simple license management in SLURM by counting the licenses used. I am also considering the interaction between SLURM and FlexLM, but I have no good result yet. The difficulty is that FlexLM has no open API (except for a command line tool lmutil). And the function provided by FlexLM is not enough for SLURM to totally controlling the licenses. For now, I think the following issues should be addressed: 1. Keep the license count in SLURM consistent with FlexLM. There may be applications run out of SLURM which may check out licenses. And a job may request wrong number of licenses (intentionally or unintentionally). 2. Force a job to release the licenses on job termination, even if there are job processes not killed. With LS-DYNA I have run into the case that after the application completes, the licenses will not be released until a long time period (even with out job processes left). LS-DYNA is not using FlexLM for license control and I am not sure whether this could happen for FlexLM managed applications. To handle various applications and the licenses managers, a license plug-in should be introduced. But the interface of the plug-in is not clear yet. I'd like to know if anyone has experiences with SLURM integration with FlexLM or other license managers. Any requirements or considerations would also be welcomed. 在 2013-07-01一的 17:17 -0700，Eva Hocks写道： > > > The documentation announced the integration since 2.4. I am running > slurm 2.4.3. > > Could anyone please point me to where I can find how to onfigure the > FlexLM license manager integration with slurm? > > > Thanks > Eva
