Shuichi Ihara wrote:
>
>> The cobbler command line is written in the Python API and can do 100% of
>> everything.
>>
>
> Sounds great!
> But, why is 'cobbler xxx' slow even though cobbler command line is written
> in the Python API? In my testing for 1000 system registration into cobbler, it
> was 2411[sec] ('cobbler add xxxx') VS 4[sec] (python based script with API).
> So, what's most overheads in cobbler commands?
>
One aspect is the startup cost is largely in starting a new Python
instance. As your system count grows, there is also more cost for
loading files off disk and into memory. Each time you start up a new
CLI session it does this. With an open API handle you are doing these
things in batch. So the total number of file reads is Sigma N...1000 of
N (sum the series in your heads, math students) with additions, or 1000
* 1000 for edits, which is not super efficient as you can imagine. The
API however allows you to only load the configuration once, rather than
1000 times.
There have been a number of recent improvements in cobbler to make the
XMLRPC API also not have this penalty. Right now on the development
branch
the Cobbler API loads the configuration only once -- when the service
starts and is notified of changes that occur outside of cobblerd so it
does not have to do reloads. Also it does not suffer from having to
start up a new interpreter.
Long term, the answer to this long term is to make the command line tool
also speak XMLRPC (locally) and never have to reload
the configuration. (A sql rewrite would be so intrusive to not make it
worth it, and risks being equally slow). The cobbler CLI could also,
if we really wanted, be done in C, but I don't think we care that
much. (time python /tmp/blank.py takes 1/3 of a second on my machine).
For most things this is already pretty easy to do, though it eventually
requires that we make the remaining cobbler functions that are not
callable over XMLRPC usable over XMLRPC (like cobbler reposync and
cobbler import)
I think it makes sense to persue this route sooner than later. In fact,
we could make simple edits speedy before doing the "import-over-XMLRPC"
if we really wanted.
We could do this regardless of the cobbler auth system employed since
the cobbler CLI could have access to a common shared secret that
cobblerd could also read, and we could use SSL from Apache to provide an
encryption against local sniffing.
>
>> It looks like you're using the Python API from what you have below
>> (excellent!), in which case you have numerous examples. The best thing
>> you can do is look in the "modules/" directory of a source checkout.
>> The cobbler CLI makes every call for every command line argument there,
>> and you can see what calls are made for each CLI getopt argument.
>>
>
> Thanks! I had a look at thees codes in modules directory and could make a
> python
> scripts with API to add system information (includes mac_address, ip_address
> whatever I want it) into cobbler.
>
>
>
>> Pretty close. The main thing is that I've exposed most of the internals
>> directly in API.py, so other than calling methods on the actual cobbler
>> objects like 'system', you can access everything through the API
>> module. Anything lower level down is subject to change and isn't a
>> stable API. api.py, however, is stable.
>>
>> import cobbler.api as capi # avoid namespace issue as you
>> use 'api' later.
>> api = capi.BootAPI()
>> s = api.find_system(name='abc') # abc is a string, plus find is a top
>> level API call, it's best not to go any deeper and just use the api.py
>> implementation
>> s.set_hostname('xyz') # use the set_ methods to
>> ensure validation is always run on the input (very important!)
>> api.add_system(s) # again, use the top level
>> API function, which sets reasonable defaults for all the arguments
>>
>
> This is very useful to me also. Thanks again.
>
>
No problem, glad to help!
--Michael
_______________________________________________
cobbler mailing list
[email protected]
https://fedorahosted.org/mailman/listinfo/cobbler
