Brute force testing is not a solution unless the testing is very complete and
extensive. Testing is not required for the case of m9dified default
configuration settings. A simple inspection of the Kconf files will detect the
problem in all cases,Sent from my Galaxy
-------- Original message --------From: Alan Carvalho de Assis
<acas...@gmail.com> Date: 3/21/21 11:03 AM (GMT-06:00) To:
dev@nuttx.apache.org Subject: Re: Problem with SmartFS access Hi Greg and
Anthony,On 3/21/21, Gregory Nutt <spudan...@gmail.com> wrote:>>> ... what we
can>> do to prevent these types of "side-effects" when people are touching
the>> config system.>> The use of defconfig files implies that we never change
default values> of configuration. If someone does change the default value of
a> configuration setting, then it effects ALL configurations that depend on>
that default. We need to catch this is the review step. During> review, if
we notice that there is a change to the default value of the> configuration
setting, then we must insist that all defconfig files be> modified so that the
previous configuration is not changed due to the> side-effect.>> CI cannot
catch this. As noted, the error does not occur until run> time. A proper
regression test would catch this but we do not do any> automated regression
testing. That is a major hole in the testing.>Yes, the CI cannot catch it, but
using some QA Unit Tests running inthe Linux simulator or QEMU we could catch
it. Abdelatif tested thesmartfs example in the "sim" board and the issue also
happened.Of course it cannot catch all the issues, but at least most of
theissues could be avoided using some basic tests.BR,Alan
