Dear All
As many of you are aware, currently there is no license compliance performed for ONAP docker containers that are distributed via nexus3.onap.org . It needs to change due to legal obligations- as all images contain many different packages and files all binding those who distribute it to perform some legal tasks. While it is highly unlikely that LF would be sued for that, for many companies it might be too much of a risk to open up their instances for customers... Tern When I started the task of introducing License Compliance for Docker containers - *Tern* ( https://github.com/tern-tools/tern ), developed since June '17 seemed to be the only viable option for Docker SBoM generation. Since then, we have integrated tern into weekly integration tests- where all deployed containers are scanned. Last scan results can be seen here: https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-06/05_11-51/security/tern/index.html They are accessible through https://logs.onap.org/onap-integration/weekly/onap_weekly_pod4_master/2021-06/05_11-51/ (security tab-> tern) Sadly Tern doesn't come without some issues: * its as accurate as what package managers found in the image are. From what I've found so far - package managers (e.g. alpine) have often different licenses filled than what can be found in repositories pointed by them * it fails often due to some external components like docker, overlayfs * there are also some security concerns to consider like the tool executing binaries from images in chroot environment or docker.sock access which are design decisions and are unlikely to change. Some of the accuracy issues could be partially alleviated when using scancode-toolkit as plugin for tern but the scans were slow and it was hard to extend the solution further ScanCode.io Since September '20, a new solution has been quickly emerging - ScanCode.io ( https://github.com/nexB/scancode.io scancodeio.readthedocs.io ) It is a server app that is a lot easier to extend and could be triggered on Jenkins Docker builds. It currently supports two main types of input: * Docker image (which is later processed into packages and files - both packaged and not) * "Codebase" (for accurate scan all dependencies should be included with the app sources) And outputs SBoM of all found packages and files. During Docker pipeline- image is broken down into packages (with corresponding files) and files that are not part of any package. The tool also supports defining policies to mark specific licenses as {ok,warning,error}. Mateusz Perc has prepared a POC for "deep" Alpine packages scan (with fetching each package build recipes & source code and scanning them alongside the packages) which gathers all the data necessary for compiling compliance report. We initiated contact line with maintainers & will work towards including those extensions in the upstream of this project. ScanCode.io is based on Django as web framework & Celery for distributed task queue (scanning is resource intensive thus deploying it in cloud environment might make sense considering amount of ONAP dockers). While ScanCode.io lacks features for reviewing license findings and resolving potential issues that Fossology (https://www.fossology.org/) has - it uses a lot more precise method of scanning (scancode-toolkit). Fossology is written in PHP while ScanCode.io - in Python 3 which we (and possibly many ONAP contributors) feel more comfortable with. When reviewing the code of ScanCode.io I've found some first steps towards extracting just the compliance information- indicating that it is a feature that would be welcome. Both me and Mateusz Perc will contribute to ScanCode.io to make it a tool that could be used for full license compliance analysis & report generation. Questions 1. Would it be possible to get infra from LF to integrate ScanCode.io into Docker build pipelines? (Somewhere to deploy the tool, Jenkins sandbox & a project that it could be tested on?) This would allow developers to get feedback on their containers contents before merging. 2. How could we integrate compliance documentation with onap docs? More Detail I have booked a 15min slot for me and Krzysztof Opasiak to present more details on the subject and discuss it further. https://wiki.onap.org/display/Meetings/TSC+2021-06-17 Best Regards Alexander Mazuruk -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#7895): https://lists.onap.org/g/onap-tsc/message/7895 Mute This Topic: https://lists.onap.org/mt/83391980/21656 Group Owner: [email protected] Unsubscribe: https://lists.onap.org/g/onap-tsc/leave/2743226/21656/1412191262/xyzzy [[email protected]] -=-=-=-=-=-=-=-=-=-=-=-
