On 11/8/18 9:52 AM, Ladislav Slezak wrote:
Dne 07. 11. 18 v 16:39 Christopher Hofmann napsal(a):
Hi YaSTees!
I had a deeper look into Ladislav's REST API and I found it very
flexible and easy to use - and furthermore very straightforward and easy
to enhance.
The advantage of the REST API is that it's not bound to any specific language or
testing framework, you can easily write a small wrapper for rspec,cucumber,...
or
openQA ;-)
It looks as the most promising approach for using it in openQA for
testing YaST.
Great! Thank you for looking into it!
Now the question raises how we can proceed with it and what we actually
need to do to get it integrated in YaST's master branch.
The benefit we get when integrate it in mainline YaST is that we are
able to test an testing unmodified YaST as it also is shipped to the
user. Testing a modified system is not really useful as it may behave
different to the unmodified system.
It should be part of the system, patching the installer for testing is not a
nice
solution.
It's not perfect solution, but if we keep fingerprint small and validate
system without patch to mitigate the risks of missing something which
works with the framework, it's fully acceptable practice. For example,
selenium web-driver is not exactly same as user's browser, on top
actions simulation also bypasses some layers of the user action. We
cannot support testing which is repeating customer setup 1 to 1, and
even with openQA we don't. If you've seen set of bootloader options we
use, it's enough to argue that we already don't have user like setup.
Possible downsides of the permanent integration would be
-> more dependencies bloat the instsys
Additional dependencies are:
libjsoncpp19 216.7 KiB
libmicrohttpd12 145.1 KiB
so overall ~360 KiB - not too much
Yes, the overhead is pretty small, just two small libs. Optionally we could
place it
in a separate inst-sys image and load it on demand, like we already do for gdb.
But
it's much bigger (~20MiB), these few kilobytes should be OK even in the default
image.
-> Opening ports on a system is always a risk. We potentially open
security hole when this is part of every customer's system.
Making sure it is only enabled when explicitly adding boot params should
make it quite safe. Of course we also need a security review before.
Maybe we should even mandatory require a user authentication.
I think sooner or later we would need to add some kind of authentication anyway.
Without it anybody who can connect to the API from the network can become root
(e.g.
by sniffing the values entered in the root password dialog). That means you
could
safely use it only in an isolated trusted network.
It would be also nice to support HTTPS so you can send or receive e.g. the root
password value in the tests in encrypted form over the network. (There is a
small
complication with the server certificate, but nothing unsolvable...)
The SSL and basic authentication features are already supported by the
libmicrohttpd
library (see [1] and [2]) so adding these security features should not be
difficult.
Alternative option would be to only listen to a UNIX socket or to
localhost. Then it could be used from outside through an ssh connection.
Currently it by default listens only on the localhost (so the REST API is not
reachable from the other hosts). But as written above, for real usage it would
be
nice to add some more security features.
So I think these downsides are not bringing big trouble.
We could consider if it is possible to make it a kind of plug-in. Can it
be e.g. be enabled only when both of the libraries above are available
in the system?
Ideally I'd make it a plugin and distribute it as a completely separate
package. This
package would not be installed by default into the target system, but would be
available in the inst-sys. And for enabling you would still need some extra boot
parameter to ensure it's user controlled (not enabled by default or by mistake).
I also believe that this would be better approach, and as we are really
interested in getting this in one way or another, as QA SLE Functional
we want to contribute, the only thing we need are to confirm those
design decisions without which we cannot move forward.
(Of course, it would be nice to have some openQA test which would check that
without
the boot option the API port is closed and the API is not active.)
That's a good point, as we need to cover things we cannot fully evaluate
with this framework, e.g. layout. But that's not any kind of issue.
If needed later the plugin could be optionally installed into the target
system, e.g.
for the AutoYaST second stage tests or tests in the installed system.
In the past we had a support for the libqdialogsolver plugin [3] in
libyui-qt-pkg
(already dropped, see [4]). I think we could use a similar solution here.
BTW my plan is to continue with the project in the next HackWeek, if there is
some
must have feature for openQA (I guess that plugin separation) just let me know.
:-)
I believe project is too good to remain as hackweek project any further,
as mentioned above, we have urgent need in such framework. And you've
done big part to show that we can have it running next month with a
single contributor. So, I kindly ask not to put it in the box of
postponed things =).
Ladislav
[1]
https://www.gnu.org/software/libmicrohttpd/tutorial.html#Supporting-basic-authentication
[2]
https://www.gnu.org/software/libmicrohttpd/tutorial.html#Adding-a-layer-of-security
[3] https://github.com/openSUSE/zypp-dialogsolver
[4] https://github.com/libyui/libyui-qt-pkg/pull/48).
--
Kind regards,
Rodion Iafarov <[email protected]>
QA Engineer
SUSE Linux GmbH, Maxfeldstr. 5, D-90409 Nürnberg
Tel: +49-911-74053-0; Fax: +49-911-7417755; https://www.suse.com/
SUSE Linux GmbH, GF: Felix Imendörffer, Jane Smithard,
Graham Norton, HRB 21284 (AG Nürnberg)
--
To unsubscribe, e-mail: [email protected]
To contact the owner, e-mail: [email protected]
