Re: [PATCH] eng: Add Software Requirements Engineering chapter

2019-09-30 Thread Gedare Bloom 
On Mon, Sep 30, 2019 at 7:13 AM Sebastian Huber
 wrote:
>
> On 24/07/2019 15:44, Sebastian Huber wrote:
> > +Requirement Traceability
> > +
> > +
> > +The standard |E10-06| demands that requirements shall be under 
> > configuration
> > +management, backwards-traceable and forward-traceable.
> > +
> > +History of Requirements
> > +---
> > +
> > +The RTEMS requirements should placed in the RTEMS sources using Git for 
> > version
> > +control.  The history of requirements can be traced with Git.  Special 
> > commit
> > +procedures for changes in requirement files should be established.  For
> > +example, it should be allowed to change only one requirement per commit.  A
> > +dedicated Git commit message format may be used as well, e.g. use of
> > +``Approved-by:`` or ``Reviewed-by:`` lines which indicate an agreed 
> > statement
> > +(similar to the
> > +`Linux kernel patch submission 
> > guidelines`_).
>
> In the first version of this patch, the proposal was to use Git to track
> the history of specification items (requirements are a specialization
> them). I think it would be better to add the revision and the
> description of a change directly to the files, e.g. add a custom
> attribute "changes" which contains a list of "revision" and
> "description" tuples.  Revision 1 items do not need this attribute.
> Example:
>
> changes:
> - description: The description of the change from revision 1 (initial) to 2.
>revision: 2
> - description: The description of the change from revision 2 to 3.
>revision: 3
> - description: The description of the change from revision 3 to 4.
>revision: 4
>
> This information can be used to add the change log of each specification
> item to a human readable document for example.  The documentation
> generator program does not need a Git repository in this case to extract
> the information.
>
> Also this way the revision of a specification item is independent of the
> current version control system. This makes it easier to reference items
> of a specific revision. For example: "With ABC-001 in revision 1 we
> could meet our system requirements, however the change to revision 2 is
> a problem, due to ...".
>
It seems to be a matter whether one needs the revision history readily
available without digging through git. Your argument for referring to
revisions, and also to extract them without needing git, is reasonable
to me.  I suspect it should make the release process simpler..

> --
> Sebastian Huber, embedded brains GmbH
>
> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> Phone   : +49 89 189 47 41-16
> Fax : +49 89 189 47 41-09
> E-Mail  : sebastian.hu...@embedded-brains.de
> PGP : Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
> ___
> devel mailing list
> devel@rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] eng: Add Software Requirements Engineering chapter

2019-09-30 Thread Sebastian Huber 

On 24/07/2019 15:44, Sebastian Huber wrote:

+Requirement Traceability
+
+
+The standard |E10-06| demands that requirements shall be under configuration
+management, backwards-traceable and forward-traceable.
+
+History of Requirements
+---
+
+The RTEMS requirements should placed in the RTEMS sources using Git for version
+control.  The history of requirements can be traced with Git.  Special commit
+procedures for changes in requirement files should be established.  For
+example, it should be allowed to change only one requirement per commit.  A
+dedicated Git commit message format may be used as well, e.g. use of
+``Approved-by:`` or ``Reviewed-by:`` lines which indicate an agreed statement
+(similar to the
+`Linux kernel patch submission 
guidelines`_).


In the first version of this patch, the proposal was to use Git to track 
the history of specification items (requirements are a specialization 
them). I think it would be better to add the revision and the 
description of a change directly to the files, e.g. add a custom 
attribute "changes" which contains a list of "revision" and 
"description" tuples.  Revision 1 items do not need this attribute. 
Example:


changes:
- description: The description of the change from revision 1 (initial) to 2.
  revision: 2
- description: The description of the change from revision 2 to 3.
  revision: 3
- description: The description of the change from revision 3 to 4.
  revision: 4

This information can be used to add the change log of each specification 
item to a human readable document for example.  The documentation 
generator program does not need a Git repository in this case to extract 
the information.


Also this way the revision of a specification item is independent of the 
current version control system. This makes it easier to reference items 
of a specific revision. For example: "With ABC-001 in revision 1 we 
could meet our system requirements, however the change to revision 2 is 
a problem, due to ...".


--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] eng: Add Software Requirements Engineering chapter

2019-07-31 Thread Gedare Bloom 
On Wed, Jul 31, 2019 at 5:33 AM Sebastian Huber
 wrote:
>
> On 24/07/2019 15:44, Sebastian Huber wrote:
> > +Requirement Verification
> > +
>
> I confused the terms verification and validation.  It should be
> "Requirement Validation".  From ECSS-E-ST-40C:
>
> "3.2.44  validation
>  process to confirm that the requirements baseline functions and
> performances are correctly and completely implemented in the final product
> 3.2.45  verification
>  process to confirm that adequate specifications and inputs
> exist for
> any activity, and that the outputs of the activities are correct and
> consistent
> with the specifications and input"
>
> > +
> > +The verification of each requirement shall be accomplished by one or more 
> > of
> > +the following methods and nothing else:
> > +
> > +**By test*: A test specification is provided to demonstrate that the 
> > requirement
> > +  is satisfied when the software is executed on the target platform.
> > +
> > +**By design*: A rationale is provided to demonstrate how the qualification
> > +  requirement is satisfied implicitly by the software design.
> > +
> > +**By analysis*: A statement is provided how the requirement is met, by 
> > analysing
> > +  static properties of the software.
> > +
> > +.. topic:: Doorstop
> > +
> > +For an item in a parent document it is checked that at least one item 
> > in a
> > +child document has a link to it.  For example a child document could
> > +contain verification items.  With this feature you can check that all
> > +requirements are covered by at least one verification item.
>
> I received a comment via private mail:
>
> "reason for test/design/analysis splitting? Maybe biased, but having
> mostly used (ECSS) RAIT scheme (review, analysis, inspection, test), I
> think 'design' should be replaced with 'inspection': By inspection, you
> verify that the design of the software satisfies a certain requirement.
> Otherwise it does not fit the other two words: 'design' is a property of
> the software, 'test' and 'analysis' are actions of verifying
> requirements (like 'inspection')"
>
> In fact, we have in ECSS-E-ST-40C:
>
> "5.6.3.1 Development and documentation of a software
>  validation specification with respect to the technical
>  specification
> [...]
> b.  Validation shall be performed by test.
>  EXPECTED OUTPUT: Software validation specification with respect
> to the
>  technical specification [DJF, SVS; CDR].
> c.  If it can be justified that validation by test cannot be performed,
>  validation shall be performed by either analysis, inspection or
> review of
>  design.
>  EXPECTED OUTPUT: Software validation specification with respect
> to the
>  technical specification [DJF, SVS; CDR]."
>
> I think this makes more sense than my original proposal. So, validation
> of each requirement shall be done by test, by analysis of design, by
> inspection of design, or by review of design.
>
Good.

> --
> Sebastian Huber, embedded brains GmbH
>
> Address : Dornierstr. 4, D-82178 Puchheim, Germany
> Phone   : +49 89 189 47 41-16
> Fax : +49 89 189 47 41-09
> E-Mail  : sebastian.hu...@embedded-brains.de
> PGP : Public key available on request.
>
> Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
> ___
> devel mailing list
> devel@rtems.org
> http://lists.rtems.org/mailman/listinfo/devel
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] eng: Add Software Requirements Engineering chapter

2019-07-31 Thread Sebastian Huber 

On 24/07/2019 15:44, Sebastian Huber wrote:

+Requirement Verification
+


I confused the terms verification and validation.  It should be 
"Requirement Validation".  From ECSS-E-ST-40C:


"3.2.44  validation
 process to confirm that the requirements baseline functions and
performances are correctly and completely implemented in the final product
3.2.45  verification
 process to confirm that adequate specifications and inputs 
exist for
any activity, and that the outputs of the activities are correct and 
consistent

with the specifications and input"


+
+The verification of each requirement shall be accomplished by one or more of
+the following methods and nothing else:
+
+**By test*: A test specification is provided to demonstrate that the 
requirement
+  is satisfied when the software is executed on the target platform.
+
+**By design*: A rationale is provided to demonstrate how the qualification
+  requirement is satisfied implicitly by the software design.
+
+**By analysis*: A statement is provided how the requirement is met, by 
analysing
+  static properties of the software.
+
+.. topic:: Doorstop
+
+For an item in a parent document it is checked that at least one item in a
+child document has a link to it.  For example a child document could
+contain verification items.  With this feature you can check that all
+requirements are covered by at least one verification item.


I received a comment via private mail:

"reason for test/design/analysis splitting? Maybe biased, but having 
mostly used (ECSS) RAIT scheme (review, analysis, inspection, test), I 
think 'design' should be replaced with 'inspection': By inspection, you 
verify that the design of the software satisfies a certain requirement. 
Otherwise it does not fit the other two words: 'design' is a property of 
the software, 'test' and 'analysis' are actions of verifying 
requirements (like 'inspection')"


In fact, we have in ECSS-E-ST-40C:

"5.6.3.1 Development and documentation of a software
validation specification with respect to the technical
specification
[...]
b.  Validation shall be performed by test.
EXPECTED OUTPUT: Software validation specification with respect 
to the

technical specification [DJF, SVS; CDR].
c.  If it can be justified that validation by test cannot be performed,
validation shall be performed by either analysis, inspection or 
review of

design.
EXPECTED OUTPUT: Software validation specification with respect 
to the

technical specification [DJF, SVS; CDR]."

I think this makes more sense than my original proposal. So, validation 
of each requirement shall be done by test, by analysis of design, by 
inspection of design, or by review of design.


--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] eng: Add Software Requirements Engineering chapter

2019-07-30 Thread Sebastian Huber 

On 30/07/2019 00:16, Gedare Bloom wrote:

Seems to make sense to me. I noted just a few very small grammatical errors.


Thanks for the review, I fixed the errors.

I will now work on a description of the "normal patch review process".

--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

Re: [PATCH] eng: Add Software Requirements Engineering chapter

2019-07-29 Thread Gedare Bloom 
Seems to make sense to me. I noted just a few very small grammatical errors.

On Wed, Jul 24, 2019 at 7:45 AM Sebastian Huber
 wrote:
>
> ---
>  eng/index.rst |   1 +
>  eng/req-eng.rst   | 813 
> ++
>  images/eng/req-add.pdf| Bin 0 -> 81320 bytes
>  images/eng/req-add.png| Bin 0 -> 50516 bytes
>  images/eng/req-add.uml|  40 +++
>  images/eng/req-modify.pdf | Bin 0 -> 68500 bytes
>  images/eng/req-modify.png | Bin 0 -> 37776 bytes
>  images/eng/req-modify.uml |  34 ++
>  8 files changed, 888 insertions(+)
>  create mode 100644 eng/req-eng.rst
>  create mode 100644 images/eng/req-add.pdf
>  create mode 100644 images/eng/req-add.png
>  create mode 100644 images/eng/req-add.uml
>  create mode 100644 images/eng/req-modify.pdf
>  create mode 100644 images/eng/req-modify.png
>  create mode 100644 images/eng/req-modify.uml
>
> diff --git a/eng/index.rst b/eng/index.rst
> index cfc831b..802eec9 100644
> --- a/eng/index.rst
> +++ b/eng/index.rst
> @@ -26,6 +26,7 @@ RTEMS Software Engineering (|version|)
>  preface
>  prequalification
>  stakeholders
> +req-eng
>  management
>  test-plan
>  test-framework
> diff --git a/eng/req-eng.rst b/eng/req-eng.rst
> new file mode 100644
> index 000..6b63c7b
> --- /dev/null
> +++ b/eng/req-eng.rst
> @@ -0,0 +1,813 @@
> +.. SPDX-License-Identifier: CC-BY-SA-4.0
> +
> +.. Copyright (C) 2019 embedded brains GmbH
> +
> +.. |E40| replace:: ECSS-E-ST-40C
> +
> +.. |E10-06| replace:: ECSS-E-ST-10-06C
> +
> +Software Requirements Engineering
> +*
> +
> +Software engineering standards for critical software such as |E40| demand 
> that
> +software requirements for a software product are collected in a software
> +requirements specification (technical specification in |E40| terms).  They 
> are
> +usually derived from system requirements (requirements baseline in |E40|
> +terms).  RTEMS is designed as a reusable software product which can be 
> utilized
> +by application designers to ease the development of their applications.  The
> +requirements of the end system (system requirements) using RTEMS are only 
> known
> +to the application designer.  RTEMS itself is developed by the RTEMS
> +maintainers and they do not know the requirements of a particular end system 
> in
> +general.  RTEMS is designed as a real-time operating system to meet typical
> +system requirements for a wide range of applications.  Its suitability for a
> +particular application must be determined by the application designer based 
> on
> +the technical specification provided by RTEMS accompanied with performance 
> data
> +for a particular target platform.
> +
> +Currently, no technical specification of RTEMS exists in the form of a
> +dedicated document.  Since the beginning of the RTEMS evolution in the late
> +1980s it was developed iteratively.  It was never developed in a waterfall
> +model.  During initial development the RTEID :cite:`Motorola:1988:RTEID` and
> +later the ORKID :cite:`VITA:1990:ORKID` draft specifications were used as
> +requirements.  These were evolving during the development and an iterative
> +approach was followed often using simple algorithms and coming back to
> +optimise.  In 1993 and 1994 a subset of pthreads sufficient to support
> +:term:`GNAT` was added as requirements.  At this time the Ada tasking was
> +defined, however, not implemented in GNAT, so this involved guessing during 
> the
> +development. Later some adjustments were made when Ada tasking was actually
> +implemented.  So, it was consciously iterative with the specifications 
> evolving
> +and feedback from performance analysis.  Benchmarks published from other real
> +time operating systems were used for comparison.  Optimizations were carried
> +out until the results were comparable.  Development was done with distinct
> +contractual phases and tasks for development, optimization, and the addition 
> of
> +priority inheritance and rate monotonic scheduling.  The pthreads requirement
> +has grown to be as much POSIX as possible.  Portability to FreeBSD to use the
> +network stack, USB stack, SD/MMC card stack and device drivers resulted in
> +another set of requirements.  The support for symmetric multiprocessing (SMP)
> +was a huge driver for change.  It was developed step by step and sponsored by
> +several independent users with completely different applications and target
> +platforms in mind.  The high performance OpenMP support introduced the Futex 
> as
> +a new synchronization primitive.  A key success element of RTEMS is the 
> ability
> +to accept changes driven by user needs and still keep the operating system
> +stable enough for production systems.  Procedures that place a high burden on
> +changes are doomed to be discarded by the RTEMS project.  We have to keep 
> this
> +in mind when we introduce a requirements management work flow which should be
> +followed by

Re: [PATCH] eng: Add Software Requirements Engineering chapter

2019-07-24 Thread Sebastian Huber 

Hello,

this is my first attempt to add a "Software Requirements Engineering" 
chapter to the RTEMS Software Engineering manual.  You can find a PDF 
with this patch here:


https://ftp.rtems.org/pub/rtems/people/sebh/eng.pdf

I removed the binary files from the patch.

--
Sebastian Huber, embedded brains GmbH

Address : Dornierstr. 4, D-82178 Puchheim, Germany
Phone   : +49 89 189 47 41-16
Fax : +49 89 189 47 41-09
E-Mail  : sebastian.hu...@embedded-brains.de
PGP : Public key available on request.

Diese Nachricht ist keine geschäftliche Mitteilung im Sinne des EHUG.
___
devel mailing list
devel@rtems.org
http://lists.rtems.org/mailman/listinfo/devel

[PATCH] eng: Add Software Requirements Engineering chapter

2019-07-24 Thread Sebastian Huber 
---
 eng/index.rst |   1 +
 eng/req-eng.rst   | 813 ++
 images/eng/req-add.pdf| Bin 0 -> 81320 bytes
 images/eng/req-add.png| Bin 0 -> 50516 bytes
 images/eng/req-add.uml|  40 +++
 images/eng/req-modify.pdf | Bin 0 -> 68500 bytes
 images/eng/req-modify.png | Bin 0 -> 37776 bytes
 images/eng/req-modify.uml |  34 ++
 8 files changed, 888 insertions(+)
 create mode 100644 eng/req-eng.rst
 create mode 100644 images/eng/req-add.pdf
 create mode 100644 images/eng/req-add.png
 create mode 100644 images/eng/req-add.uml
 create mode 100644 images/eng/req-modify.pdf
 create mode 100644 images/eng/req-modify.png
 create mode 100644 images/eng/req-modify.uml

diff --git a/eng/index.rst b/eng/index.rst
index cfc831b..802eec9 100644
--- a/eng/index.rst
+++ b/eng/index.rst
@@ -26,6 +26,7 @@ RTEMS Software Engineering (|version|)
 preface
 prequalification
 stakeholders
+req-eng
 management
 test-plan
 test-framework
diff --git a/eng/req-eng.rst b/eng/req-eng.rst
new file mode 100644
index 000..6b63c7b
--- /dev/null
+++ b/eng/req-eng.rst
@@ -0,0 +1,813 @@
+.. SPDX-License-Identifier: CC-BY-SA-4.0
+
+.. Copyright (C) 2019 embedded brains GmbH
+
+.. |E40| replace:: ECSS-E-ST-40C
+
+.. |E10-06| replace:: ECSS-E-ST-10-06C
+
+Software Requirements Engineering
+*
+
+Software engineering standards for critical software such as |E40| demand that
+software requirements for a software product are collected in a software
+requirements specification (technical specification in |E40| terms).  They are
+usually derived from system requirements (requirements baseline in |E40|
+terms).  RTEMS is designed as a reusable software product which can be utilized
+by application designers to ease the development of their applications.  The
+requirements of the end system (system requirements) using RTEMS are only known
+to the application designer.  RTEMS itself is developed by the RTEMS
+maintainers and they do not know the requirements of a particular end system in
+general.  RTEMS is designed as a real-time operating system to meet typical
+system requirements for a wide range of applications.  Its suitability for a
+particular application must be determined by the application designer based on
+the technical specification provided by RTEMS accompanied with performance data
+for a particular target platform.
+
+Currently, no technical specification of RTEMS exists in the form of a
+dedicated document.  Since the beginning of the RTEMS evolution in the late
+1980s it was developed iteratively.  It was never developed in a waterfall
+model.  During initial development the RTEID :cite:`Motorola:1988:RTEID` and
+later the ORKID :cite:`VITA:1990:ORKID` draft specifications were used as
+requirements.  These were evolving during the development and an iterative
+approach was followed often using simple algorithms and coming back to
+optimise.  In 1993 and 1994 a subset of pthreads sufficient to support
+:term:`GNAT` was added as requirements.  At this time the Ada tasking was
+defined, however, not implemented in GNAT, so this involved guessing during the
+development. Later some adjustments were made when Ada tasking was actually
+implemented.  So, it was consciously iterative with the specifications evolving
+and feedback from performance analysis.  Benchmarks published from other real
+time operating systems were used for comparison.  Optimizations were carried
+out until the results were comparable.  Development was done with distinct
+contractual phases and tasks for development, optimization, and the addition of
+priority inheritance and rate monotonic scheduling.  The pthreads requirement
+has grown to be as much POSIX as possible.  Portability to FreeBSD to use the
+network stack, USB stack, SD/MMC card stack and device drivers resulted in
+another set of requirements.  The support for symmetric multiprocessing (SMP)
+was a huge driver for change.  It was developed step by step and sponsored by
+several independent users with completely different applications and target
+platforms in mind.  The high performance OpenMP support introduced the Futex as
+a new synchronization primitive.  A key success element of RTEMS is the ability
+to accept changes driven by user needs and still keep the operating system
+stable enough for production systems.  Procedures that place a high burden on
+changes are doomed to be discarded by the RTEMS project.  We have to keep this
+in mind when we introduce a requirements management work flow which should be
+followed by RTEMS community members and new contributors.
+
+We have to put in some effort first into the reconstruction of software
+requirements through reverse engineering using the RTEMS documentation, test
+cases, sources, standard references, mailing list archives, etc. as input.
+Writing a technical specification for the complete RTEMS code base is probably
+a