Re: [PATCH] docs/core-api: make mm-api.rst more structured
On Wed, 28 Nov 2018 16:45:44 +0200 Mike Rapoport wrote: > The mm-api.rst covers variety of memory management APIs under "More Memory > Management Functions" section. The descriptions included there are in a > random order there are quite a few of them which makes the section too > long. > > Regrouping the documentation by subject and splitting the long "More Memory > Management Functions" section into several smaller sections makes the > generated html more usable. > > Signed-off-by: Mike Rapoport Applied, thanks. jon
Re: [PATCH v2 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
On 06.12.18 01:26, Wei Yang wrote: > Currently locking for memory hotplug is a little complicated. > > Generally speaking, we leverage the two global lock: > > * device_hotplug_lock > * mem_hotplug_lock > > to serialise the process. > > While for the long term, we are willing to have more fine-grained lock > to provide higher scalability. > > This patch divides Locking Internal section based on these two global > locks to help readers to understand it. Also it adds some new finding to > enrich it. > > [David: words arrangement] > > Signed-off-by: Wei Yang > --- > v2: adjustment based on David and Mike comment > --- > Documentation/core-api/memory-hotplug.rst | 27 --- > 1 file changed, 24 insertions(+), 3 deletions(-) > > diff --git a/Documentation/core-api/memory-hotplug.rst > b/Documentation/core-api/memory-hotplug.rst > index de7467e48067..51d477ad4b80 100644 > --- a/Documentation/core-api/memory-hotplug.rst > +++ b/Documentation/core-api/memory-hotplug.rst > @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification > queue. > Locking Internals > = > > +In addition to fine grained locks like pgdat_resize_lock, there are three > locks > +involved > + > +- device_hotplug_lock > +- mem_hotplug_lock > +- device_lock > + > +Currently, they are twisted together for all kinds of reasons. The following > +part is divided into device_hotplug_lock and mem_hotplug_lock parts > +respectively to describe those tricky situations. > + > +device_hotplug_lock > +- > + > When adding/removing memory that uses memory block devices (i.e. ordinary > RAM), > the device_hotplug_lock should be held to: > > @@ -111,13 +125,20 @@ As the device is visible to user space before taking > the device_lock(), this > can result in a lock inversion. > > onlining/offlining of memory should be done via device_online()/ > -device_offline() - to make sure it is properly synchronized to actions > -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect > online_type) > +device_offline() - to make sure it is properly synchronized to actions via > +sysfs. Even if mem_hotplug_lock is used to protect the process, because of > the > +lock inversion described above, holding device_hotplug_lock is still advised > +(to e.g. protect online_type) > + > +mem_hotplug_lock > +- > > When adding/removing/onlining/offlining memory or adding/removing > heterogeneous/device memory, we should always hold the mem_hotplug_lock in > write mode to serialise memory hotplug (e.g. access to global/zone > -variables). > +variables). Currently, we take advantage of this to serialise sparsemem's > +mem_section handling in sparse_add_one_section() and > +sparse_remove_one_section(). > > In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > mode allows for a quite efficient get_online_mems/put_online_mems > Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb
Re: [PATCH v2 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
On Thu, Dec 06, 2018 at 08:26:22AM +0800, Wei Yang wrote: > Currently locking for memory hotplug is a little complicated. > > Generally speaking, we leverage the two global lock: > > * device_hotplug_lock > * mem_hotplug_lock > > to serialise the process. > > While for the long term, we are willing to have more fine-grained lock > to provide higher scalability. > > This patch divides Locking Internal section based on these two global > locks to help readers to understand it. Also it adds some new finding to > enrich it. > > [David: words arrangement] > > Signed-off-by: Wei Yang Reviewd-by: Mike Rapoport > --- > v2: adjustment based on David and Mike comment > --- > Documentation/core-api/memory-hotplug.rst | 27 --- > 1 file changed, 24 insertions(+), 3 deletions(-) > > diff --git a/Documentation/core-api/memory-hotplug.rst > b/Documentation/core-api/memory-hotplug.rst > index de7467e48067..51d477ad4b80 100644 > --- a/Documentation/core-api/memory-hotplug.rst > +++ b/Documentation/core-api/memory-hotplug.rst > @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification > queue. > Locking Internals > = > > +In addition to fine grained locks like pgdat_resize_lock, there are three > locks > +involved > + > +- device_hotplug_lock > +- mem_hotplug_lock > +- device_lock > + > +Currently, they are twisted together for all kinds of reasons. The following > +part is divided into device_hotplug_lock and mem_hotplug_lock parts > +respectively to describe those tricky situations. > + > +device_hotplug_lock > +- > + > When adding/removing memory that uses memory block devices (i.e. ordinary > RAM), > the device_hotplug_lock should be held to: > > @@ -111,13 +125,20 @@ As the device is visible to user space before taking > the device_lock(), this > can result in a lock inversion. > > onlining/offlining of memory should be done via device_online()/ > -device_offline() - to make sure it is properly synchronized to actions > -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect > online_type) > +device_offline() - to make sure it is properly synchronized to actions via > +sysfs. Even if mem_hotplug_lock is used to protect the process, because of > the > +lock inversion described above, holding device_hotplug_lock is still advised > +(to e.g. protect online_type) > + > +mem_hotplug_lock > +- > > When adding/removing/onlining/offlining memory or adding/removing > heterogeneous/device memory, we should always hold the mem_hotplug_lock in > write mode to serialise memory hotplug (e.g. access to global/zone > -variables). > +variables). Currently, we take advantage of this to serialise sparsemem's > +mem_section handling in sparse_add_one_section() and > +sparse_remove_one_section(). > > In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > mode allows for a quite efficient get_online_mems/put_online_mems > -- > 2.15.1 > -- Sincerely yours, Mike.
[PATCH v2 1/2] admin-guide/memory-hotplug.rst: remove locking internal part from admin-guide
Locking Internal section exists in core-api documentation, which is more suitable for this. This patch removes the duplication part here. Signed-off-by: Wei Yang Reviewed-by: David Hildenbrand Acked-by: Mike Rapoport --- Documentation/admin-guide/mm/memory-hotplug.rst | 40 - 1 file changed, 40 deletions(-) diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst index 5c4432c96c4b..241f4ce1e387 100644 --- a/Documentation/admin-guide/mm/memory-hotplug.rst +++ b/Documentation/admin-guide/mm/memory-hotplug.rst @@ -392,46 +392,6 @@ Need more implementation yet - Notification completion of remove works by OS to firmware. - Guard from remove if not yet. - -Locking Internals -= - -When adding/removing memory that uses memory block devices (i.e. ordinary RAM), -the device_hotplug_lock should be held to: - -- synchronize against online/offline requests (e.g. via sysfs). This way, memory - block devices can only be accessed (.online/.state attributes) by user - space once memory has been fully added. And when removing memory, we - know nobody is in critical sections. -- synchronize against CPU hotplug and similar (e.g. relevant for ACPI and PPC) - -Especially, there is a possible lock inversion that is avoided using -device_hotplug_lock when adding memory and user space tries to online that -memory faster than expected: - -- device_online() will first take the device_lock(), followed by - mem_hotplug_lock -- add_memory_resource() will first take the mem_hotplug_lock, followed by - the device_lock() (while creating the devices, during bus_add_device()). - -As the device is visible to user space before taking the device_lock(), this -can result in a lock inversion. - -onlining/offlining of memory should be done via device_online()/ -device_offline() - to make sure it is properly synchronized to actions -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type) - -When adding/removing/onlining/offlining memory or adding/removing -heterogeneous/device memory, we should always hold the mem_hotplug_lock in -write mode to serialise memory hotplug (e.g. access to global/zone -variables). - -In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read -mode allows for a quite efficient get_online_mems/put_online_mems -implementation, so code accessing memory can protect from that memory -vanishing. - - Future Work === -- 2.15.1
[PATCH v2 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
Currently locking for memory hotplug is a little complicated. Generally speaking, we leverage the two global lock: * device_hotplug_lock * mem_hotplug_lock to serialise the process. While for the long term, we are willing to have more fine-grained lock to provide higher scalability. This patch divides Locking Internal section based on these two global locks to help readers to understand it. Also it adds some new finding to enrich it. [David: words arrangement] Signed-off-by: Wei Yang --- v2: adjustment based on David and Mike comment --- Documentation/core-api/memory-hotplug.rst | 27 --- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/Documentation/core-api/memory-hotplug.rst b/Documentation/core-api/memory-hotplug.rst index de7467e48067..51d477ad4b80 100644 --- a/Documentation/core-api/memory-hotplug.rst +++ b/Documentation/core-api/memory-hotplug.rst @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification queue. Locking Internals = +In addition to fine grained locks like pgdat_resize_lock, there are three locks +involved + +- device_hotplug_lock +- mem_hotplug_lock +- device_lock + +Currently, they are twisted together for all kinds of reasons. The following +part is divided into device_hotplug_lock and mem_hotplug_lock parts +respectively to describe those tricky situations. + +device_hotplug_lock +- + When adding/removing memory that uses memory block devices (i.e. ordinary RAM), the device_hotplug_lock should be held to: @@ -111,13 +125,20 @@ As the device is visible to user space before taking the device_lock(), this can result in a lock inversion. onlining/offlining of memory should be done via device_online()/ -device_offline() - to make sure it is properly synchronized to actions -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type) +device_offline() - to make sure it is properly synchronized to actions via +sysfs. Even if mem_hotplug_lock is used to protect the process, because of the +lock inversion described above, holding device_hotplug_lock is still advised +(to e.g. protect online_type) + +mem_hotplug_lock +- When adding/removing/onlining/offlining memory or adding/removing heterogeneous/device memory, we should always hold the mem_hotplug_lock in write mode to serialise memory hotplug (e.g. access to global/zone -variables). +variables). Currently, we take advantage of this to serialise sparsemem's +mem_section handling in sparse_add_one_section() and +sparse_remove_one_section(). In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read mode allows for a quite efficient get_online_mems/put_online_mems -- 2.15.1
Re: [PATCH 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
On Wed, Dec 05, 2018 at 01:13:10PM +0100, Michal Hocko wrote: >On Wed 05-12-18 10:34:26, Wei Yang wrote: >> Currently locking for memory hotplug is a little complicated. >> >> Generally speaking, we leverage the two global lock: >> >> * device_hotplug_lock >> * mem_hotplug_lock >> >> to serialise the process. >> >> While for the long term, we are willing to have more fine-grained lock >> to provide higher scalability. >> >> This patch divides Locking Internal section based on these two global >> locks to help readers to understand it. Also it adds some new finding to >> enrich it. >> >> [David: words arrangement] >> >> Signed-off-by: Wei Yang > >For a love of mine I cannot find the locking description by Oscar. Maybe >it never existed and I just made it up ;) But if it is not imaginary >then my recollection is that it was much more comprehensive. If not then >even this is a good start. Thanks. If Oscar has already has some work on it, this could be a complement to his work :-) > >> --- >> Documentation/core-api/memory-hotplug.rst | 27 --- >> 1 file changed, 24 insertions(+), 3 deletions(-) >> >> diff --git a/Documentation/core-api/memory-hotplug.rst >> b/Documentation/core-api/memory-hotplug.rst >> index de7467e48067..95662b283328 100644 >> --- a/Documentation/core-api/memory-hotplug.rst >> +++ b/Documentation/core-api/memory-hotplug.rst >> @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification >> queue. >> Locking Internals >> = >> >> +There are three locks involved in memory-hotplug, two global lock and one >> local >> +lock: >> + >> +- device_hotplug_lock >> +- mem_hotplug_lock >> +- device_lock >> + >> +Currently, they are twisted together for all kinds of reasons. The following >> +part is divided into device_hotplug_lock and mem_hotplug_lock parts >> +respectively to describe those tricky situations. >> + >> +device_hotplug_lock >> +- >> + >> When adding/removing memory that uses memory block devices (i.e. ordinary >> RAM), >> the device_hotplug_lock should be held to: >> >> @@ -111,13 +125,20 @@ As the device is visible to user space before taking >> the device_lock(), this >> can result in a lock inversion. >> >> onlining/offlining of memory should be done via device_online()/ >> -device_offline() - to make sure it is properly synchronized to actions >> -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect >> online_type) >> +device_offline() - to make sure it is properly synchronized to actions via >> +sysfs. Even mem_hotplug_lock is used to protect the process, because of the >> +lock inversion described above, holding device_hotplug_lock is still advised >> +(to e.g. protect online_type) >> + >> +mem_hotplug_lock >> +- >> >> When adding/removing/onlining/offlining memory or adding/removing >> heterogeneous/device memory, we should always hold the mem_hotplug_lock in >> write mode to serialise memory hotplug (e.g. access to global/zone >> -variables). >> +variables). Currently, we take advantage of this to serialise sparsemem's >> +mem_section handling in sparse_add_one_section() and >> +sparse_remove_one_section(). >> >> In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read >> mode allows for a quite efficient get_online_mems/put_online_mems >> -- >> 2.15.1 >> > >-- >Michal Hocko >SUSE Labs -- Wei Yang Help you, Help me
Re: [PATCH 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
On Wed 05-12-18 10:34:26, Wei Yang wrote: > Currently locking for memory hotplug is a little complicated. > > Generally speaking, we leverage the two global lock: > > * device_hotplug_lock > * mem_hotplug_lock > > to serialise the process. > > While for the long term, we are willing to have more fine-grained lock > to provide higher scalability. > > This patch divides Locking Internal section based on these two global > locks to help readers to understand it. Also it adds some new finding to > enrich it. > > [David: words arrangement] > > Signed-off-by: Wei Yang For a love of mine I cannot find the locking description by Oscar. Maybe it never existed and I just made it up ;) But if it is not imaginary then my recollection is that it was much more comprehensive. If not then even this is a good start. > --- > Documentation/core-api/memory-hotplug.rst | 27 --- > 1 file changed, 24 insertions(+), 3 deletions(-) > > diff --git a/Documentation/core-api/memory-hotplug.rst > b/Documentation/core-api/memory-hotplug.rst > index de7467e48067..95662b283328 100644 > --- a/Documentation/core-api/memory-hotplug.rst > +++ b/Documentation/core-api/memory-hotplug.rst > @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification > queue. > Locking Internals > = > > +There are three locks involved in memory-hotplug, two global lock and one > local > +lock: > + > +- device_hotplug_lock > +- mem_hotplug_lock > +- device_lock > + > +Currently, they are twisted together for all kinds of reasons. The following > +part is divided into device_hotplug_lock and mem_hotplug_lock parts > +respectively to describe those tricky situations. > + > +device_hotplug_lock > +- > + > When adding/removing memory that uses memory block devices (i.e. ordinary > RAM), > the device_hotplug_lock should be held to: > > @@ -111,13 +125,20 @@ As the device is visible to user space before taking > the device_lock(), this > can result in a lock inversion. > > onlining/offlining of memory should be done via device_online()/ > -device_offline() - to make sure it is properly synchronized to actions > -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect > online_type) > +device_offline() - to make sure it is properly synchronized to actions via > +sysfs. Even mem_hotplug_lock is used to protect the process, because of the > +lock inversion described above, holding device_hotplug_lock is still advised > +(to e.g. protect online_type) > + > +mem_hotplug_lock > +- > > When adding/removing/onlining/offlining memory or adding/removing > heterogeneous/device memory, we should always hold the mem_hotplug_lock in > write mode to serialise memory hotplug (e.g. access to global/zone > -variables). > +variables). Currently, we take advantage of this to serialise sparsemem's > +mem_section handling in sparse_add_one_section() and > +sparse_remove_one_section(). > > In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > mode allows for a quite efficient get_online_mems/put_online_mems > -- > 2.15.1 > -- Michal Hocko SUSE Labs
Re: [PATCH 1/2] admin-guide/memory-hotplug.rst: remove locking internal part from admin-guide
On Wed 05-12-18 10:34:25, Wei Yang wrote: > Locking Internal section exists in core-api documentation, which is more > suitable for this. > > This patch removes the duplication part here. > > Signed-off-by: Wei Yang Yes this doesn't really make any sense in an admin guide. It is a pure implementation detail nobody should be relying on. -- Michal Hocko SUSE Labs
[PATCH v1 8/9] drm/doc: Add initial komeda driver documentation
Signed-off-by: James (Qian) Wang
---
Documentation/gpu/drivers.rst| 1 +
Documentation/gpu/komeda-kms.rst | 483 +++
2 files changed, 484 insertions(+)
create mode 100644 Documentation/gpu/komeda-kms.rst
diff --git a/Documentation/gpu/drivers.rst b/Documentation/gpu/drivers.rst
index 7c1672118a73..978e6da9bbff 100644
--- a/Documentation/gpu/drivers.rst
+++ b/Documentation/gpu/drivers.rst
@@ -17,6 +17,7 @@ GPU Driver Documentation
vkms
bridge/dw-hdmi
xen-front
+ komeda-kms
.. only:: subproject and html
diff --git a/Documentation/gpu/komeda-kms.rst b/Documentation/gpu/komeda-kms.rst
new file mode 100644
index ..8af925ca0869
--- /dev/null
+++ b/Documentation/gpu/komeda-kms.rst
@@ -0,0 +1,483 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==
+ drm/komeda ARM display driver
+==
+
+The drm/komeda driver supports for the ARM display processor D71 and later
+IPs, this document is for giving a brief overview of driver design: how it
+works and why design it like that.
+
+Overview of D71 like display IPs
+
+
+From D71, Arm display IP begins to adopt a flexible and modularized
+architecture. A display pipeline is made up of multiple individual and
+functional pipeline stages called components, and every component has some
+specific capabilities that can give the flowed pipeline pixel data a
+particular processing.
+
+Typical D71 components:
+
+Layer
+-
+Layer is the first pipeline stage, which is for preparing the pixel data
+for the next stage. It fetches the pixel from memory, decodes it if it's
+AFBC, rotates the source image, unpacks or converts YUV pixels to the device
+internal RGB pixels, then adjust the color_space of pixels if need.
+
+Scaler
+--
+As its name, scaler is taking responsability for scaling, and D71 also
+supports image enhancements by scaler.
+The usage of scaler is very flexible and can be connected to layer output
+for layer scaling, or connected to compositor and scale the whole display
+frame and then feed the output data into wb_layer which will then write it
+into memory.
+
+Compositor (compiz)
+---
+Compositor is for blending multiple layers or pixel data flows into one
+single display frame, and its output frame can be fed into post image
+processor and then on the monitor or fed into wb_layer and written to memory
+at the same time. And user also can insert a scaler between compositor and
+wb_layer to down scale the display frame first and then writing to memory.
+
+Writeback Layer (wb_layer)
+--
+Writeback layer do the opposite things of Layer, Which connect to compiz and
try
+to write the composition result to memory.
+
+Post image processor (improc)
+-
+Post image processor is for adjusting frame data like gamma and color space
+to fit the requirements of the monitor.
+
+Timing controller (timing_ctrlr)
+
+Final stage of display pipeline, Timing controller is not for the pixel
+handling, but only for controlling the display timing.
+
+Merger
+--
+D71 scaler mostly only has half the horizontal input/output capabilities
compare
+with Layer, Like if Layer supports 4K input size, the scaler only can supports
+2K input/output in some time. To achieve the fully frame scaling, D71 introduce
+Layer Split, which split the whole image to two half part and feed them to two
+Layers A and B, and do the scaling independently, after scaling the result need
+to be feed to merger to merge two part image, and then output to compiz.
+
+Splitter
+
+Similar to Layer Split, but Splitter is used for writeback, which split the
+compiz result to two part and then feed them to two scaler.
+
+Possible D71 Pipeline usage
+===
+
+Benefit from the modularized architecture, D71 pipelines can be easily adjusted
+to fit different usages, following are some typical pipeline data flow
+configurations:
+
+Single pipeline data flow
+-
+
+.. kernel-render:: DOT
+ :alt: Single pipeline digraph
+ :caption: Single pipeline data flow
+
+ digraph single_ppl {
+ rankdir=LR;
+
+ subgraph {
+ "Memory";
+ "Monitor";
+ }
+
+ subgraph cluster_pipeline {
+ style=dashed
+ node [shape=box]
+ {
+ node [bgcolor=grey style=dashed]
+ "Scaler-0";
+ "Scaler-1";
+ "Scaler-0/1"
+ }
+
+ node [bgcolor=grey style=filled]
+ "Layer-0" -> "Scaler-0"
+ "Layer-1" -> "Scaler-0"
+ "Layer-2" -> "Scaler-1"
+ "Layer-3" -> "Scaler-1"
+
+ "Layer-0" -> "Compiz"
+ "Layer-1" -> "Compiz"
+ "Layer-2" -> "Compiz"
+ "Layer-3" -> "Compiz"
+ "Scaler-0" -> "Compiz"
+ "Scaler-1" -> "Compiz"
+
+ "Compiz" -> "Scaler-0/1" ->
[PATCH v1 8/9] drm/doc: Add initial komeda driver documentation
Signed-off-by: James (Qian) Wang
---
Documentation/gpu/drivers.rst| 1 +
Documentation/gpu/komeda-kms.rst | 483 +++
2 files changed, 484 insertions(+)
create mode 100644 Documentation/gpu/komeda-kms.rst
diff --git a/Documentation/gpu/drivers.rst b/Documentation/gpu/drivers.rst
index 7c1672118a73..978e6da9bbff 100644
--- a/Documentation/gpu/drivers.rst
+++ b/Documentation/gpu/drivers.rst
@@ -17,6 +17,7 @@ GPU Driver Documentation
vkms
bridge/dw-hdmi
xen-front
+ komeda-kms
.. only:: subproject and html
diff --git a/Documentation/gpu/komeda-kms.rst b/Documentation/gpu/komeda-kms.rst
new file mode 100644
index ..8af925ca0869
--- /dev/null
+++ b/Documentation/gpu/komeda-kms.rst
@@ -0,0 +1,483 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+==
+ drm/komeda ARM display driver
+==
+
+The drm/komeda driver supports for the ARM display processor D71 and later
+IPs, this document is for giving a brief overview of driver design: how it
+works and why design it like that.
+
+Overview of D71 like display IPs
+
+
+From D71, Arm display IP begins to adopt a flexible and modularized
+architecture. A display pipeline is made up of multiple individual and
+functional pipeline stages called components, and every component has some
+specific capabilities that can give the flowed pipeline pixel data a
+particular processing.
+
+Typical D71 components:
+
+Layer
+-
+Layer is the first pipeline stage, which is for preparing the pixel data
+for the next stage. It fetches the pixel from memory, decodes it if it's
+AFBC, rotates the source image, unpacks or converts YUV pixels to the device
+internal RGB pixels, then adjust the color_space of pixels if need.
+
+Scaler
+--
+As its name, scaler is taking responsability for scaling, and D71 also
+supports image enhancements by scaler.
+The usage of scaler is very flexible and can be connected to layer output
+for layer scaling, or connected to compositor and scale the whole display
+frame and then feed the output data into wb_layer which will then write it
+into memory.
+
+Compositor (compiz)
+---
+Compositor is for blending multiple layers or pixel data flows into one
+single display frame, and its output frame can be fed into post image
+processor and then on the monitor or fed into wb_layer and written to memory
+at the same time. And user also can insert a scaler between compositor and
+wb_layer to down scale the display frame first and then writing to memory.
+
+Writeback Layer (wb_layer)
+--
+Writeback layer do the opposite things of Layer, Which connect to compiz and
try
+to write the composition result to memory.
+
+Post image processor (improc)
+-
+Post image processor is for adjusting frame data like gamma and color space
+to fit the requirements of the monitor.
+
+Timing controller (timing_ctrlr)
+
+Final stage of display pipeline, Timing controller is not for the pixel
+handling, but only for controlling the display timing.
+
+Merger
+--
+D71 scaler mostly only has half the horizontal input/output capabilities
compare
+with Layer, Like if Layer supports 4K input size, the scaler only can supports
+2K input/output in some time. To achieve the fully frame scaling, D71 introduce
+Layer Split, which split the whole image to two half part and feed them to two
+Layers A and B, and do the scaling independently, after scaling the result need
+to be feed to merger to merge two part image, and then output to compiz.
+
+Splitter
+
+Similar to Layer Split, but Splitter is used for writeback, which split the
+compiz result to two part and then feed them to two scaler.
+
+Possible D71 Pipeline usage
+===
+
+Benefit from the modularized architecture, D71 pipelines can be easily adjusted
+to fit different usages, following are some typical pipeline data flow
+configurations:
+
+Single pipeline data flow
+-
+
+.. kernel-render:: DOT
+ :alt: Single pipeline digraph
+ :caption: Single pipeline data flow
+
+ digraph single_ppl {
+ rankdir=LR;
+
+ subgraph {
+ "Memory";
+ "Monitor";
+ }
+
+ subgraph cluster_pipeline {
+ style=dashed
+ node [shape=box]
+ {
+ node [bgcolor=grey style=dashed]
+ "Scaler-0";
+ "Scaler-1";
+ "Scaler-0/1"
+ }
+
+ node [bgcolor=grey style=filled]
+ "Layer-0" -> "Scaler-0"
+ "Layer-1" -> "Scaler-0"
+ "Layer-2" -> "Scaler-1"
+ "Layer-3" -> "Scaler-1"
+
+ "Layer-0" -> "Compiz"
+ "Layer-1" -> "Compiz"
+ "Layer-2" -> "Compiz"
+ "Layer-3" -> "Compiz"
+ "Scaler-0" -> "Compiz"
+ "Scaler-1" -> "Compiz"
+
+ "Compiz" -> "Scaler-0/1" ->
Re: [PATCH 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
On Wed, Dec 05, 2018 at 10:40:45AM +0200, Mike Rapoport wrote: >On Wed, Dec 05, 2018 at 10:34:26AM +0800, Wei Yang wrote: >> Currently locking for memory hotplug is a little complicated. >> >> Generally speaking, we leverage the two global lock: >> >> * device_hotplug_lock >> * mem_hotplug_lock >> >> to serialise the process. >> >> While for the long term, we are willing to have more fine-grained lock >> to provide higher scalability. >> >> This patch divides Locking Internal section based on these two global >> locks to help readers to understand it. Also it adds some new finding to >> enrich it. >> >> [David: words arrangement] >> >> Signed-off-by: Wei Yang >> --- >> Documentation/core-api/memory-hotplug.rst | 27 --- >> 1 file changed, 24 insertions(+), 3 deletions(-) >> >> diff --git a/Documentation/core-api/memory-hotplug.rst >> b/Documentation/core-api/memory-hotplug.rst >> index de7467e48067..95662b283328 100644 >> --- a/Documentation/core-api/memory-hotplug.rst >> +++ b/Documentation/core-api/memory-hotplug.rst >> @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification >> queue. >> Locking Internals >> = >> >> +There are three locks involved in memory-hotplug, two global lock and one >> local > >typo: ^locks > Thanks :-) >> +lock: >> + >> +- device_hotplug_lock >> +- mem_hotplug_lock >> +- device_lock >> + >> +Currently, they are twisted together for all kinds of reasons. The following >> +part is divided into device_hotplug_lock and mem_hotplug_lock parts >> +respectively to describe those tricky situations. >> + >> +device_hotplug_lock >> +- >> + >> When adding/removing memory that uses memory block devices (i.e. ordinary >> RAM), >> the device_hotplug_lock should be held to: >> >> @@ -111,13 +125,20 @@ As the device is visible to user space before taking >> the device_lock(), this >> can result in a lock inversion. >> >> onlining/offlining of memory should be done via device_online()/ >> -device_offline() - to make sure it is properly synchronized to actions >> -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect >> online_type) >> +device_offline() - to make sure it is properly synchronized to actions via >> +sysfs. Even mem_hotplug_lock is used to protect the process, because of the > >I think it should be "Even if mem_hotplug_lock ..." > Ah, my poor English, will fix it in next version. :-) >> +lock inversion described above, holding device_hotplug_lock is still advised >> +(to e.g. protect online_type) >> + >> +mem_hotplug_lock >> +- >> >> When adding/removing/onlining/offlining memory or adding/removing >> heterogeneous/device memory, we should always hold the mem_hotplug_lock in >> write mode to serialise memory hotplug (e.g. access to global/zone >> -variables). >> +variables). Currently, we take advantage of this to serialise sparsemem's >> +mem_section handling in sparse_add_one_section() and >> +sparse_remove_one_section(). >> >> In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read >> mode allows for a quite efficient get_online_mems/put_online_mems >> -- >> 2.15.1 >> > >-- >Sincerely yours, >Mike. -- Wei Yang Help you, Help me
Re: [PATCH 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
On Wed, Dec 05, 2018 at 09:08:47AM +0100, David Hildenbrand wrote: >On 05.12.18 03:34, Wei Yang wrote: >> Currently locking for memory hotplug is a little complicated. >> >> Generally speaking, we leverage the two global lock: >> >> * device_hotplug_lock >> * mem_hotplug_lock >> >> to serialise the process. >> >> While for the long term, we are willing to have more fine-grained lock >> to provide higher scalability. >> >> This patch divides Locking Internal section based on these two global >> locks to help readers to understand it. Also it adds some new finding to >> enrich it. >> >> [David: words arrangement] >> >> Signed-off-by: Wei Yang >> --- >> Documentation/core-api/memory-hotplug.rst | 27 --- >> 1 file changed, 24 insertions(+), 3 deletions(-) >> >> diff --git a/Documentation/core-api/memory-hotplug.rst >> b/Documentation/core-api/memory-hotplug.rst >> index de7467e48067..95662b283328 100644 >> --- a/Documentation/core-api/memory-hotplug.rst >> +++ b/Documentation/core-api/memory-hotplug.rst >> @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification >> queue. >> Locking Internals >> = >> >> +There are three locks involved in memory-hotplug, two global lock and one >> local >> +lock: >> + >> +- device_hotplug_lock >> +- mem_hotplug_lock >> +- device_lock >> + > >Do we really only ever use these three and not anything else when >adding/removing/onlining/offlining memory? > >(I am thinking e.g. about pgdat_resize_lock) Yes there are more than those three, pgdat_resize_lock is one of them. > >If so, you should phrase that maybe more generally Or add more details :) Yep, while I don't get a whole picture about the pgdat_resize_lock. The usage of this lock scatter in many places. > >"In addition to fine grained locks like pgdat_resize_lock, there are >three locks involved ..." > Sounds better :-) -- Wei Yang Help you, Help me
Re: [PATCH 1/2] admin-guide/memory-hotplug.rst: remove locking internal part from admin-guide
On Wed, Dec 05, 2018 at 10:30:13AM +0200, Mike Rapoport wrote: >On Wed, Dec 05, 2018 at 09:03:24AM +0100, David Hildenbrand wrote: >> On 05.12.18 03:34, Wei Yang wrote: >> > Locking Internal section exists in core-api documentation, which is more >> > suitable for this. >> > >> > This patch removes the duplication part here. >> > >> > Signed-off-by: Wei Yang >> > --- >> > Documentation/admin-guide/mm/memory-hotplug.rst | 40 >> > - >> > 1 file changed, 40 deletions(-) >> > >> > diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst >> > b/Documentation/admin-guide/mm/memory-hotplug.rst >> > index 5c4432c96c4b..241f4ce1e387 100644 >> > --- a/Documentation/admin-guide/mm/memory-hotplug.rst >> > +++ b/Documentation/admin-guide/mm/memory-hotplug.rst >> > @@ -392,46 +392,6 @@ Need more implementation yet >> > - Notification completion of remove works by OS to firmware. >> > - Guard from remove if not yet. > >[ ... ] > >> > Future Work >> > === >> > >> > >> >> I reported this yesterday to Jonathan and Mike >> >> https://lkml.org/lkml/2018/12/3/340 > >Somehow I've missed it... > >> Anyhow >> >> Reviewed-by: David Hildenbrand > >Acked-by: Mike Rapoport > Thanks :-) >> >> -- >> >> Thanks, >> >> David / dhildenb >> > >-- >Sincerely yours, >Mike. -- Wei Yang Help you, Help me
Re: [PATCH 1/2] admin-guide/memory-hotplug.rst: remove locking internal part from admin-guide
On Wed, Dec 05, 2018 at 09:03:24AM +0100, David Hildenbrand wrote: >On 05.12.18 03:34, Wei Yang wrote: >> Locking Internal section exists in core-api documentation, which is more >> suitable for this. >> >> This patch removes the duplication part here. >> >> Signed-off-by: Wei Yang >> --- >> Documentation/admin-guide/mm/memory-hotplug.rst | 40 >> - >> 1 file changed, 40 deletions(-) >> >> diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst >> b/Documentation/admin-guide/mm/memory-hotplug.rst >> index 5c4432c96c4b..241f4ce1e387 100644 >> --- a/Documentation/admin-guide/mm/memory-hotplug.rst >> +++ b/Documentation/admin-guide/mm/memory-hotplug.rst >> @@ -392,46 +392,6 @@ Need more implementation yet >> - Notification completion of remove works by OS to firmware. >> - Guard from remove if not yet. >> >> - >> -Locking Internals >> -= >> - >> -When adding/removing memory that uses memory block devices (i.e. ordinary >> RAM), >> -the device_hotplug_lock should be held to: >> - >> -- synchronize against online/offline requests (e.g. via sysfs). This way, >> memory >> - block devices can only be accessed (.online/.state attributes) by user >> - space once memory has been fully added. And when removing memory, we >> - know nobody is in critical sections. >> -- synchronize against CPU hotplug and similar (e.g. relevant for ACPI and >> PPC) >> - >> -Especially, there is a possible lock inversion that is avoided using >> -device_hotplug_lock when adding memory and user space tries to online that >> -memory faster than expected: >> - >> -- device_online() will first take the device_lock(), followed by >> - mem_hotplug_lock >> -- add_memory_resource() will first take the mem_hotplug_lock, followed by >> - the device_lock() (while creating the devices, during bus_add_device()). >> - >> -As the device is visible to user space before taking the device_lock(), this >> -can result in a lock inversion. >> - >> -onlining/offlining of memory should be done via device_online()/ >> -device_offline() - to make sure it is properly synchronized to actions >> -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect >> online_type) >> - >> -When adding/removing/onlining/offlining memory or adding/removing >> -heterogeneous/device memory, we should always hold the mem_hotplug_lock in >> -write mode to serialise memory hotplug (e.g. access to global/zone >> -variables). >> - >> -In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read >> -mode allows for a quite efficient get_online_mems/put_online_mems >> -implementation, so code accessing memory can protect from that memory >> -vanishing. >> - >> - >> Future Work >> === >> >> > >I reported this yesterday to Jonathan and Mike > >https://lkml.org/lkml/2018/12/3/340 > Ah, Thanks :-) > >Anyhow > >Reviewed-by: David Hildenbrand > >-- > >Thanks, > >David / dhildenb -- Wei Yang Help you, Help me
Re: [PATCH 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
On Wed, Dec 05, 2018 at 10:34:26AM +0800, Wei Yang wrote: > Currently locking for memory hotplug is a little complicated. > > Generally speaking, we leverage the two global lock: > > * device_hotplug_lock > * mem_hotplug_lock > > to serialise the process. > > While for the long term, we are willing to have more fine-grained lock > to provide higher scalability. > > This patch divides Locking Internal section based on these two global > locks to help readers to understand it. Also it adds some new finding to > enrich it. > > [David: words arrangement] > > Signed-off-by: Wei Yang > --- > Documentation/core-api/memory-hotplug.rst | 27 --- > 1 file changed, 24 insertions(+), 3 deletions(-) > > diff --git a/Documentation/core-api/memory-hotplug.rst > b/Documentation/core-api/memory-hotplug.rst > index de7467e48067..95662b283328 100644 > --- a/Documentation/core-api/memory-hotplug.rst > +++ b/Documentation/core-api/memory-hotplug.rst > @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification > queue. > Locking Internals > = > > +There are three locks involved in memory-hotplug, two global lock and one > local typo: ^locks > +lock: > + > +- device_hotplug_lock > +- mem_hotplug_lock > +- device_lock > + > +Currently, they are twisted together for all kinds of reasons. The following > +part is divided into device_hotplug_lock and mem_hotplug_lock parts > +respectively to describe those tricky situations. > + > +device_hotplug_lock > +- > + > When adding/removing memory that uses memory block devices (i.e. ordinary > RAM), > the device_hotplug_lock should be held to: > > @@ -111,13 +125,20 @@ As the device is visible to user space before taking > the device_lock(), this > can result in a lock inversion. > > onlining/offlining of memory should be done via device_online()/ > -device_offline() - to make sure it is properly synchronized to actions > -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect > online_type) > +device_offline() - to make sure it is properly synchronized to actions via > +sysfs. Even mem_hotplug_lock is used to protect the process, because of the I think it should be "Even if mem_hotplug_lock ..." > +lock inversion described above, holding device_hotplug_lock is still advised > +(to e.g. protect online_type) > + > +mem_hotplug_lock > +- > > When adding/removing/onlining/offlining memory or adding/removing > heterogeneous/device memory, we should always hold the mem_hotplug_lock in > write mode to serialise memory hotplug (e.g. access to global/zone > -variables). > +variables). Currently, we take advantage of this to serialise sparsemem's > +mem_section handling in sparse_add_one_section() and > +sparse_remove_one_section(). > > In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > mode allows for a quite efficient get_online_mems/put_online_mems > -- > 2.15.1 > -- Sincerely yours, Mike.
Re: [PATCH 1/2] admin-guide/memory-hotplug.rst: remove locking internal part from admin-guide
On Wed, Dec 05, 2018 at 09:03:24AM +0100, David Hildenbrand wrote: > On 05.12.18 03:34, Wei Yang wrote: > > Locking Internal section exists in core-api documentation, which is more > > suitable for this. > > > > This patch removes the duplication part here. > > > > Signed-off-by: Wei Yang > > --- > > Documentation/admin-guide/mm/memory-hotplug.rst | 40 > > - > > 1 file changed, 40 deletions(-) > > > > diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst > > b/Documentation/admin-guide/mm/memory-hotplug.rst > > index 5c4432c96c4b..241f4ce1e387 100644 > > --- a/Documentation/admin-guide/mm/memory-hotplug.rst > > +++ b/Documentation/admin-guide/mm/memory-hotplug.rst > > @@ -392,46 +392,6 @@ Need more implementation yet > > - Notification completion of remove works by OS to firmware. > > - Guard from remove if not yet. [ ... ] > > Future Work > > === > > > > > > I reported this yesterday to Jonathan and Mike > > https://lkml.org/lkml/2018/12/3/340 Somehow I've missed it... > Anyhow > > Reviewed-by: David Hildenbrand Acked-by: Mike Rapoport > > -- > > Thanks, > > David / dhildenb > -- Sincerely yours, Mike.
Re: [PATCH 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
On 05.12.18 03:34, Wei Yang wrote: > Currently locking for memory hotplug is a little complicated. > > Generally speaking, we leverage the two global lock: > > * device_hotplug_lock > * mem_hotplug_lock > > to serialise the process. > > While for the long term, we are willing to have more fine-grained lock > to provide higher scalability. > > This patch divides Locking Internal section based on these two global > locks to help readers to understand it. Also it adds some new finding to > enrich it. > > [David: words arrangement] > > Signed-off-by: Wei Yang > --- > Documentation/core-api/memory-hotplug.rst | 27 --- > 1 file changed, 24 insertions(+), 3 deletions(-) > > diff --git a/Documentation/core-api/memory-hotplug.rst > b/Documentation/core-api/memory-hotplug.rst > index de7467e48067..95662b283328 100644 > --- a/Documentation/core-api/memory-hotplug.rst > +++ b/Documentation/core-api/memory-hotplug.rst > @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification > queue. > Locking Internals > = > > +There are three locks involved in memory-hotplug, two global lock and one > local > +lock: > + > +- device_hotplug_lock > +- mem_hotplug_lock > +- device_lock > + Do we really only ever use these three and not anything else when adding/removing/onlining/offlining memory? (I am thinking e.g. about pgdat_resize_lock) If so, you should phrase that maybe more generally Or add more details :) "In addition to fine grained locks like pgdat_resize_lock, there are three locks involved ..." > +Currently, they are twisted together for all kinds of reasons. The following > +part is divided into device_hotplug_lock and mem_hotplug_lock parts > +respectively to describe those tricky situations. > + > +device_hotplug_lock > +- > + > When adding/removing memory that uses memory block devices (i.e. ordinary > RAM), > the device_hotplug_lock should be held to: > > @@ -111,13 +125,20 @@ As the device is visible to user space before taking > the device_lock(), this > can result in a lock inversion. > > onlining/offlining of memory should be done via device_online()/ > -device_offline() - to make sure it is properly synchronized to actions > -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect > online_type) > +device_offline() - to make sure it is properly synchronized to actions via > +sysfs. Even mem_hotplug_lock is used to protect the process, because of the > +lock inversion described above, holding device_hotplug_lock is still advised > +(to e.g. protect online_type) > + > +mem_hotplug_lock > +- > > When adding/removing/onlining/offlining memory or adding/removing > heterogeneous/device memory, we should always hold the mem_hotplug_lock in > write mode to serialise memory hotplug (e.g. access to global/zone > -variables). > +variables). Currently, we take advantage of this to serialise sparsemem's > +mem_section handling in sparse_add_one_section() and > +sparse_remove_one_section(). > > In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > mode allows for a quite efficient get_online_mems/put_online_mems > -- Thanks, David / dhildenb
Re: [PATCH 1/2] admin-guide/memory-hotplug.rst: remove locking internal part from admin-guide
On 05.12.18 03:34, Wei Yang wrote: > Locking Internal section exists in core-api documentation, which is more > suitable for this. > > This patch removes the duplication part here. > > Signed-off-by: Wei Yang > --- > Documentation/admin-guide/mm/memory-hotplug.rst | 40 > - > 1 file changed, 40 deletions(-) > > diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst > b/Documentation/admin-guide/mm/memory-hotplug.rst > index 5c4432c96c4b..241f4ce1e387 100644 > --- a/Documentation/admin-guide/mm/memory-hotplug.rst > +++ b/Documentation/admin-guide/mm/memory-hotplug.rst > @@ -392,46 +392,6 @@ Need more implementation yet > - Notification completion of remove works by OS to firmware. > - Guard from remove if not yet. > > - > -Locking Internals > -= > - > -When adding/removing memory that uses memory block devices (i.e. ordinary > RAM), > -the device_hotplug_lock should be held to: > - > -- synchronize against online/offline requests (e.g. via sysfs). This way, > memory > - block devices can only be accessed (.online/.state attributes) by user > - space once memory has been fully added. And when removing memory, we > - know nobody is in critical sections. > -- synchronize against CPU hotplug and similar (e.g. relevant for ACPI and > PPC) > - > -Especially, there is a possible lock inversion that is avoided using > -device_hotplug_lock when adding memory and user space tries to online that > -memory faster than expected: > - > -- device_online() will first take the device_lock(), followed by > - mem_hotplug_lock > -- add_memory_resource() will first take the mem_hotplug_lock, followed by > - the device_lock() (while creating the devices, during bus_add_device()). > - > -As the device is visible to user space before taking the device_lock(), this > -can result in a lock inversion. > - > -onlining/offlining of memory should be done via device_online()/ > -device_offline() - to make sure it is properly synchronized to actions > -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect > online_type) > - > -When adding/removing/onlining/offlining memory or adding/removing > -heterogeneous/device memory, we should always hold the mem_hotplug_lock in > -write mode to serialise memory hotplug (e.g. access to global/zone > -variables). > - > -In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read > -mode allows for a quite efficient get_online_mems/put_online_mems > -implementation, so code accessing memory can protect from that memory > -vanishing. > - > - > Future Work > === > > I reported this yesterday to Jonathan and Mike https://lkml.org/lkml/2018/12/3/340 Anyhow Reviewed-by: David Hildenbrand -- Thanks, David / dhildenb
[PATCH 2/2] core-api/memory-hotplug.rst: divide Locking Internal section by different locks
Currently locking for memory hotplug is a little complicated. Generally speaking, we leverage the two global lock: * device_hotplug_lock * mem_hotplug_lock to serialise the process. While for the long term, we are willing to have more fine-grained lock to provide higher scalability. This patch divides Locking Internal section based on these two global locks to help readers to understand it. Also it adds some new finding to enrich it. [David: words arrangement] Signed-off-by: Wei Yang --- Documentation/core-api/memory-hotplug.rst | 27 --- 1 file changed, 24 insertions(+), 3 deletions(-) diff --git a/Documentation/core-api/memory-hotplug.rst b/Documentation/core-api/memory-hotplug.rst index de7467e48067..95662b283328 100644 --- a/Documentation/core-api/memory-hotplug.rst +++ b/Documentation/core-api/memory-hotplug.rst @@ -89,6 +89,20 @@ NOTIFY_STOP stops further processing of the notification queue. Locking Internals = +There are three locks involved in memory-hotplug, two global lock and one local +lock: + +- device_hotplug_lock +- mem_hotplug_lock +- device_lock + +Currently, they are twisted together for all kinds of reasons. The following +part is divided into device_hotplug_lock and mem_hotplug_lock parts +respectively to describe those tricky situations. + +device_hotplug_lock +- + When adding/removing memory that uses memory block devices (i.e. ordinary RAM), the device_hotplug_lock should be held to: @@ -111,13 +125,20 @@ As the device is visible to user space before taking the device_lock(), this can result in a lock inversion. onlining/offlining of memory should be done via device_online()/ -device_offline() - to make sure it is properly synchronized to actions -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type) +device_offline() - to make sure it is properly synchronized to actions via +sysfs. Even mem_hotplug_lock is used to protect the process, because of the +lock inversion described above, holding device_hotplug_lock is still advised +(to e.g. protect online_type) + +mem_hotplug_lock +- When adding/removing/onlining/offlining memory or adding/removing heterogeneous/device memory, we should always hold the mem_hotplug_lock in write mode to serialise memory hotplug (e.g. access to global/zone -variables). +variables). Currently, we take advantage of this to serialise sparsemem's +mem_section handling in sparse_add_one_section() and +sparse_remove_one_section(). In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read mode allows for a quite efficient get_online_mems/put_online_mems -- 2.15.1
[PATCH 1/2] admin-guide/memory-hotplug.rst: remove locking internal part from admin-guide
Locking Internal section exists in core-api documentation, which is more suitable for this. This patch removes the duplication part here. Signed-off-by: Wei Yang --- Documentation/admin-guide/mm/memory-hotplug.rst | 40 - 1 file changed, 40 deletions(-) diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst index 5c4432c96c4b..241f4ce1e387 100644 --- a/Documentation/admin-guide/mm/memory-hotplug.rst +++ b/Documentation/admin-guide/mm/memory-hotplug.rst @@ -392,46 +392,6 @@ Need more implementation yet - Notification completion of remove works by OS to firmware. - Guard from remove if not yet. - -Locking Internals -= - -When adding/removing memory that uses memory block devices (i.e. ordinary RAM), -the device_hotplug_lock should be held to: - -- synchronize against online/offline requests (e.g. via sysfs). This way, memory - block devices can only be accessed (.online/.state attributes) by user - space once memory has been fully added. And when removing memory, we - know nobody is in critical sections. -- synchronize against CPU hotplug and similar (e.g. relevant for ACPI and PPC) - -Especially, there is a possible lock inversion that is avoided using -device_hotplug_lock when adding memory and user space tries to online that -memory faster than expected: - -- device_online() will first take the device_lock(), followed by - mem_hotplug_lock -- add_memory_resource() will first take the mem_hotplug_lock, followed by - the device_lock() (while creating the devices, during bus_add_device()). - -As the device is visible to user space before taking the device_lock(), this -can result in a lock inversion. - -onlining/offlining of memory should be done via device_online()/ -device_offline() - to make sure it is properly synchronized to actions -via sysfs. Holding device_hotplug_lock is advised (to e.g. protect online_type) - -When adding/removing/onlining/offlining memory or adding/removing -heterogeneous/device memory, we should always hold the mem_hotplug_lock in -write mode to serialise memory hotplug (e.g. access to global/zone -variables). - -In addition, mem_hotplug_lock (in contrast to device_hotplug_lock) in read -mode allows for a quite efficient get_online_mems/put_online_mems -implementation, so code accessing memory can protect from that memory -vanishing. - - Future Work === -- 2.15.1
RE,
-- Hello, First of all i will like to apologies for my manner of communication because you do not know me personally, its due to the fact that i have a very important proposal for you.
Brauchen Sie einen Kredit?
-- Brauchen Sie einen Kredit?
[RFC] Documentation: use RFC 5735 subnet for examples
RFC 5735 defines this subnets for documentation and example code: 192.0.2.0/24 as TEST-NET-1 198.51.100.0/24 as TEST-NET-2 203.0.113.0/24 as TEST-NET-3 Replace where possible the IP addresses in the documentation with addresses belonging to the test subnets. Signed-off-by: Matteo Croce --- Documentation/ABI/testing/sysfs-bus-rbd| 2 +- Documentation/admin-guide/LSM/Smack.rst| 4 ++-- Documentation/dev-tools/kgdb.rst | 2 +- Documentation/filesystems/nfs/nfs-rdma.txt | 4 ++-- Documentation/isdn/README.HiSax| 4 ++-- Documentation/networking/batman-adv.rst| 6 +++--- Documentation/networking/cops.txt | 2 +- Documentation/networking/l2tp.txt | 6 +++--- Documentation/networking/multiqueue.txt| 4 ++-- Documentation/networking/net_failover.rst | 2 +- Documentation/networking/pktgen.txt| 6 +++--- Documentation/networking/switchdev.txt | 4 ++-- 12 files changed, 23 insertions(+), 23 deletions(-) diff --git a/Documentation/ABI/testing/sysfs-bus-rbd b/Documentation/ABI/testing/sysfs-bus-rbd index cc30bee8b5f4..82e6a942c274 100644 --- a/Documentation/ABI/testing/sysfs-bus-rbd +++ b/Documentation/ABI/testing/sysfs-bus-rbd @@ -7,7 +7,7 @@ Description: Usage: [] -$ echo "192.168.0.1 name=admin rbd foo" > /sys/bus/rbd/add +$ echo "192.0.2.1 name=admin rbd foo" > /sys/bus/rbd/add The snapshot name can be "-" or omitted to map the image read/write. A will be assigned for any registered block diff --git a/Documentation/admin-guide/LSM/Smack.rst b/Documentation/admin-guide/LSM/Smack.rst index 6a5826a13aea..0178467cfb9e 100644 --- a/Documentation/admin-guide/LSM/Smack.rst +++ b/Documentation/admin-guide/LSM/Smack.rst @@ -723,11 +723,11 @@ If you don't know what CIPSO is and don't plan to use it, you can just do:: echo 127.0.0.1 -CIPSO > /sys/fs/smackfs/netlabel echo 0.0.0.0/0 @ > /sys/fs/smackfs/netlabel -If you use CIPSO on your 192.168.0.0/16 local network and need also unlabeled +If you use CIPSO on your 192.0.2.0/16 local network and need also unlabeled Internet access, you can have:: echo 127.0.0.1 -CIPSO > /sys/fs/smackfs/netlabel - echo 192.168.0.0/16 -CIPSO > /sys/fs/smackfs/netlabel + echo 192.0.2.0/16 -CIPSO > /sys/fs/smackfs/netlabel echo 0.0.0.0/0 @ > /sys/fs/smackfs/netlabel Writing Applications for Smack diff --git a/Documentation/dev-tools/kgdb.rst b/Documentation/dev-tools/kgdb.rst index d38be58f872a..50a7641d13c1 100644 --- a/Documentation/dev-tools/kgdb.rst +++ b/Documentation/dev-tools/kgdb.rst @@ -541,7 +541,7 @@ Connecting with gdb to a serial port Example (kgdb to a terminal server on TCP port 2012):: % gdb ./vmlinux - (gdb) target remote 192.168.2.2:2012 + (gdb) target remote 192.0.2.2:2012 Once connected, you can debug a kernel the way you would debug an diff --git a/Documentation/filesystems/nfs/nfs-rdma.txt b/Documentation/filesystems/nfs/nfs-rdma.txt index 22dc0dd6889c..1adb29475d34 100644 --- a/Documentation/filesystems/nfs/nfs-rdma.txt +++ b/Documentation/filesystems/nfs/nfs-rdma.txt @@ -214,8 +214,8 @@ NFS/RDMA Setup Exports entries with the following formats have been tested: -/vol0 192.168.0.47(fsid=0,rw,async,insecure,no_root_squash) -/vol0 192.168.0.0/255.255.255.0(fsid=0,rw,async,insecure,no_root_squash) +/vol0 192.0.2.47(fsid=0,rw,async,insecure,no_root_squash) +/vol0 192.0.2.0/255.255.255.0(fsid=0,rw,async,insecure,no_root_squash) The IP address(es) is(are) the client's IPoIB address for an InfiniBand HCA or the client's iWARP address(es) for an RNIC. diff --git a/Documentation/isdn/README.HiSax b/Documentation/isdn/README.HiSax index b1a573cf4472..0c3a7e0ef80b 100644 --- a/Documentation/isdn/README.HiSax +++ b/Documentation/isdn/README.HiSax @@ -584,8 +584,8 @@ I4L_REMOTE_IS_CISCO=no I4L_MODULE_PARAMS="type=16 io=0x268 irq=7 " I4L_DEBUG=no I4L_LEASED_128K=yes -LOCAL_IP=192.168.1.1 -REMOTE_IP=192.168.2.1 +LOCAL_IP=192.0.2.1 +REMOTE_IP=198.51.100.1 case "$1" in start) diff --git a/Documentation/networking/batman-adv.rst b/Documentation/networking/batman-adv.rst index 245fb6c0ab6f..7c9bfeed628f 100644 --- a/Documentation/networking/batman-adv.rst +++ b/Documentation/networking/batman-adv.rst @@ -118,11 +118,11 @@ IP address which can be either statically configured or dynamically (by using DHCP or similar services):: NodeA: ip link set up dev bat0 - NodeA: ip addr add 192.168.0.1/24 dev bat0 + NodeA: ip addr add 192.0.2.1/24 dev bat0 NodeB: ip link set up dev bat0 - NodeB: ip addr add 192.168.0.2/24 dev bat0 - NodeB: ping 192.168.0.1 + NodeB: ip addr add 192.0.2.2/24 dev bat0 + NodeB: ping 192.0.2.1 Note: In order to avoid problems remove all IP addresses previously assigned to interfaces now used by batman advanced,
[PATCH] docs/core-api: make mm-api.rst more structured
The mm-api.rst covers variety of memory management APIs under "More Memory Management Functions" section. The descriptions included there are in a random order there are quite a few of them which makes the section too long. Regrouping the documentation by subject and splitting the long "More Memory Management Functions" section into several smaller sections makes the generated html more usable. Signed-off-by: Mike Rapoport --- Documentation/core-api/mm-api.rst | 29 - 1 file changed, 20 insertions(+), 9 deletions(-) diff --git a/Documentation/core-api/mm-api.rst b/Documentation/core-api/mm-api.rst index 5ce1ec1..c81e754 100644 --- a/Documentation/core-api/mm-api.rst +++ b/Documentation/core-api/mm-api.rst @@ -49,8 +49,14 @@ The Slab Cache .. kernel-doc:: mm/util.c :functions: kfree_const kvmalloc_node kvfree -More Memory Management Functions - +Virtually Contiguous Mappings += + +.. kernel-doc:: mm/vmalloc.c + :export: + +File Mapping and Page Cache +=== .. kernel-doc:: mm/readahead.c :export: @@ -58,23 +64,28 @@ More Memory Management Functions .. kernel-doc:: mm/filemap.c :export: -.. kernel-doc:: mm/memory.c +.. kernel-doc:: mm/page-writeback.c :export: -.. kernel-doc:: mm/vmalloc.c +.. kernel-doc:: mm/truncate.c :export: -.. kernel-doc:: mm/page_alloc.c - :internal: +Memory pools + .. kernel-doc:: mm/mempool.c :export: +DMA pools += + .. kernel-doc:: mm/dmapool.c :export: -.. kernel-doc:: mm/page-writeback.c - :export: +More Memory Management Functions + -.. kernel-doc:: mm/truncate.c +.. kernel-doc:: mm/memory.c :export: + +.. kernel-doc:: mm/page_alloc.c -- 2.7.4
[no subject]
-- Guten Tag, Wir sind eine registrierte private Geldverleiher. Wir geben Kredite an Firmen, Einzelpersonen, die ihre finanzielle Status auf der ganzen Welt aktualisieren müssen, mit minimalen jährlichen Zinsen von 2% .reply, wenn nötig. Good Day, We are a registered private money lender. We give out loans to firms, Individual who need to update their financial status all over the world, with Minimal annual Interest Rates of 2%reply if needed.
Re: Did You Receive My Last Mail?
Hello, My name is ms. Reem Al-Hashimi. The UAE minister of state for international cooparation. I got your contact from an email database from your country. I have a financial transaction i would like to discuss with you. Please reply to reem2...@daum.net, for more details if you are interested. Regards, Ms. Reem Al-Hashimi
Re: [PATCH 1/1] scripts/kernel-doc: Fix struct and struct field attribute processing
On Thu, 22 Nov 2018 13:06:04 +0200 Sakari Ailus wrote: > The kernel-doc attempts to clear the struct and struct member attributes > from the API documentation it produces. It falls short of the job in the > following respects: > > - extra whitespaces are left where __attribute__((...)) was removed, > > - only a single attribute is removed per struct, > > - attributes (such as aligned) containing numbers were not removed, > > - attributes are only cleared from struct fields, not structs themselves. > > This patch addresses these issues by removing the attributes. > > Signed-off-by: Sakari Ailus That does indeed seem to improve things. I'm waiting for the pile of regexes to fall over and hurt somebody, but I guess we're not there yet. Applied, thanks. jon
RE,
Hello, First of all i will like to apologies for my manner of communication because you do not know me personally, its due to the fact that i have a very important proposal for you.
RE,
Hello, First of all i will like to apologies for my manner of communication because you do not know me personally, its due to the fact that i have a very important proposal for you.
RE,
Hello, First of all i will like to apologies for my manner of communication because you do not know me personally, its due to the fact that i have a very important proposal for you.
RE,
Hello, First of all i will like to apologies for my manner of communication because you do not know me personally, its due to the fact that i have a very important proposal for you.
[PATCH 1/1] scripts/kernel-doc: Fix struct and struct field attribute processing
The kernel-doc attempts to clear the struct and struct member attributes
from the API documentation it produces. It falls short of the job in the
following respects:
- extra whitespaces are left where __attribute__((...)) was removed,
- only a single attribute is removed per struct,
- attributes (such as aligned) containing numbers were not removed,
- attributes are only cleared from struct fields, not structs themselves.
This patch addresses these issues by removing the attributes.
Signed-off-by: Sakari Ailus
---
scripts/kernel-doc | 7 ---
1 file changed, 4 insertions(+), 3 deletions(-)
diff --git a/scripts/kernel-doc b/scripts/kernel-doc
index ffbe901a37b51..5a3945dc50358 100755
--- a/scripts/kernel-doc
+++ b/scripts/kernel-doc
@@ -1062,7 +1062,7 @@ sub dump_struct($$) {
my $x = shift;
my $file = shift;
-if ($x =~ /(struct|union)\s+(\w+)\s*\{(.*)\}/) {
+if ($x =~
/(struct|union)\s+(\w+)\s*\{(.*)\}(\s*(__packed|__aligned|__attribute__\s*\(\([a-z0-9,_\s\(\)]*\)\)))*/)
{
my $decl_type = $1;
$declaration_name = $2;
my $members = $3;
@@ -1073,8 +1073,9 @@ sub dump_struct($$) {
# strip comments:
$members =~ s/\/\*.*?\*\///gos;
# strip attributes
- $members =~ s/__attribute__\s*\(\([a-z,_\*\s\(\)]*\)\)//i;
- $members =~ s/__aligned\s*\([^;]*\)//gos;
+ $members =~ s/\s*__attribute__\s*\(\([a-z0-9,_\*\s\(\)]*\)\)//gi;
+ $members =~ s/\s*__aligned\s*\([^;]*\)//gos;
+ $members =~ s/\s*__packed\s*//gos;
$members =~ s/\s*CRYPTO_MINALIGN_ATTR//gos;
# replace DECLARE_BITMAP
$members =~ s/DECLARE_BITMAP\s*\(([^,)]+),\s*([^,)]+)\)/unsigned long
$1\[BITS_TO_LONGS($2)\]/gos;
--
2.11.0
Re: [PATCH security-next v5 00/30] LSM: Explict ordering
On 11/14/2018 1:04 PM, Casey Schaufler wrote: > On 10/24/2018 1:12 PM, Kees Cook wrote: >> On Wed, Oct 24, 2018 at 1:56 AM, Casey Schaufler >> wrote: >>> On 10/23/2018 12:05 PM, Casey Schaufler wrote: On 10/23/2018 11:50 AM, Kees Cook wrote: > Did you poke around at my combined series? > https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=lsm/ordering-v6-blob-sharing I hope to do that on the plane later today. >>> I had a chance to poke at the combined series and it >>> all seems to work as advertised. >> /me stares at John, Paul, and Stephen. Hurry up and get off your planes! ;) >> >> -Kees >> > Now that we have next-general updated I'm porting these forward. I have ported Kees' set forward to 4.20-rc2: https://github.com/cschaufler/lsm-stacking.git#after-kees-4.20-rc2 There's one bug fix in ordered_lsm_parse() but otherwise just mechanical. The infrastructure blob management required for TOMOYO, Landlock and SARA is available: https://github.com/cschaufler/lsm-stacking.git#blobs-4.20-rc2 There's nothing new here, although I had to make a few changes to account for Kees' changes in the preceding patches. I will post the patches shortly. The full stacking changes based on this are still in the works. Netfilter has added a whole new use of secmarks and I'm working to make the individual patches smaller for easier review.
Re: [PATCH v2] docs/admin-guide/mm/concepts.rst: grammar and style fixups
On Sun, 11 Nov 2018 11:24:23 +0200 Mike Rapoport wrote: > Signed-off-by: Mike Rapoport > Reviewed-by: Randy Dunlap > --- > > v2: address Matthew's feedback > > Documentation/admin-guide/mm/concepts.rst | 51 > --- > 1 file changed, 26 insertions(+), 25 deletions(-) Applied, thanks. jon
Re: [PATCH] docs/mm: update kmalloc kernel-doc description
On Sun, 11 Nov 2018 18:48:44 +0200 Mike Rapoport wrote: > Add references to GFP documentation and the memory-allocation.rst and remove > GFP_USER, GFP_DMA and GFP_NOIO descriptions. > > While on it slightly change the formatting so that the list of GFP flags > will be rendered as "description" in the generated html. > > Signed-off-by: Mike Rapoport > --- > > Probably this should go via the -mm tree as it touches include/linux/slab.h A week and some later it's not there - Andrew is even slower than me, it seems! :) So I went ahead and applied it, fixing the conflict over the addition of the memory_allocation label while I was at it. Thanks, jon
Re: [PATCH] Link the memory allocation guide from the MM docs
On Mon, 19 Nov 2018 08:00:49 -0800 Matthew Wilcox wrote: > I just went looking for the memory allocation guide in the MM docs instead > of in the core API. For the benefit of the next person who makes that > mistake, link to it from the MM docs. > > Signed-off-by: Matthew Wilcox Applied, thanks. jon
Re: [PATCH] Link the memory allocation guide from the MM docs
On Mon, Nov 19, 2018 at 08:00:49AM -0800, Matthew Wilcox wrote: > I just went looking for the memory allocation guide in the MM docs instead > of in the core API. For the benefit of the next person who makes that > mistake, link to it from the MM docs. > > Signed-off-by: Matthew Wilcox Acked-by: Mike Rapoport > > diff --git a/Documentation/core-api/memory-allocation.rst > b/Documentation/core-api/memory-allocation.rst > index f8bb9aa120c4..8954a88ff5b7 100644 > --- a/Documentation/core-api/memory-allocation.rst > +++ b/Documentation/core-api/memory-allocation.rst > @@ -1,3 +1,5 @@ > +.. _memory-allocation: > + > === > Memory Allocation Guide > === > diff --git a/Documentation/vm/index.rst b/Documentation/vm/index.rst > index c4ded22197ca..2b3ab3a1ccf3 100644 > --- a/Documentation/vm/index.rst > +++ b/Documentation/vm/index.rst > @@ -2,7 +2,9 @@ > Linux Memory Management Documentation > = > > -This is a collection of documents about Linux memory management (mm) > subsystem. > +This is a collection of documents about the Linux memory management (mm) > +subsystem. If you are looking for advice on simply allocating memory, > +see the :ref:`memory-allocation`. > > User guides for MM features > === > -- Sincerely yours, Mike.
Re: [PATCH v2] Document /proc/pid PID reuse behavior
On Mon, Nov 19, 2018 at 2:54 AM, Pavel Machek wrote: > On Mon 2018-11-05 13:22:05, Daniel Colascione wrote: >> State explicitly that holding a /proc/pid file descriptor open does >> not reserve the PID. Also note that in the event of PID reuse, these >> open file descriptors refer to the old, now-dead process, and not the >> new one that happens to be named the same numeric PID. >> >> Signed-off-by: Daniel Colascione >> --- >> Documentation/filesystems/proc.txt | 7 +++ >> 1 file changed, 7 insertions(+) >> >> Moved paragraphed to start of /proc/pid section; added signed-off-by. >> >> diff --git a/Documentation/filesystems/proc.txt >> b/Documentation/filesystems/proc.txt >> index 12a5e6e693b6..0b14460f721d 100644 >> --- a/Documentation/filesystems/proc.txt >> +++ b/Documentation/filesystems/proc.txt >> @@ -125,6 +125,13 @@ process running on the system, which is named after the >> process ID (PID). >> The link self points to the process reading the file system. Each >> process >> subdirectory has the entries listed in Table 1-1. >> >> +Note that an open a file descriptor to /proc/ or to any of its >> +contained files or subdirectories does not prevent being reused >> +for some other process in the event that exits. Operations on > > "does not" -> "may not"? > > We want to leave this unspecified, so that we can change it in future. No. "Does not". I'm sick and tired of procfs behavior being vague and unspecified to the point where even kernel developers have an incorrect mental model of how it all works. With Christian Brauner's good work in place and hopefully my own work to follow, we're moving firmly in the direction of procfs handles being struct pid references. Instead of waffling further, let's just buy into this model and do the best job we can of making it work well.
Re: [PATCH v1 2/8] mm: convert PG_balloon to PG_offline
> > diff --git a/include/linux/page-flags.h b/include/linux/page-flags.h > index 50ce1bddaf56..f91da3d0a67e 100644 > --- a/include/linux/page-flags.h > +++ b/include/linux/page-flags.h > @@ -670,7 +670,7 @@ PAGEFLAG_FALSE(DoubleMap) > #define PAGE_TYPE_BASE 0xf000 > /* Reserve 0x007f to catch underflows of page_mapcount */ > #define PG_buddy 0x0080 > -#define PG_balloon 0x0100 > +#define PG_offline 0x0100 > #define PG_kmemcg0x0200 > #define PG_table 0x0400 > > @@ -700,10 +700,13 @@ static __always_inline void __ClearPage##uname(struct > page *page) \ > PAGE_TYPE_OPS(Buddy, buddy) > > /* > - * PageBalloon() is true for pages that are on the balloon page list > - * (see mm/balloon_compaction.c). > + * PageOffline() indicates that the pages is logically offline although the s/pages/page/ :) -- Thanks, David / dhildenb
[PATCH] Link the memory allocation guide from the MM docs
I just went looking for the memory allocation guide in the MM docs instead of in the core API. For the benefit of the next person who makes that mistake, link to it from the MM docs. Signed-off-by: Matthew Wilcox diff --git a/Documentation/core-api/memory-allocation.rst b/Documentation/core-api/memory-allocation.rst index f8bb9aa120c4..8954a88ff5b7 100644 --- a/Documentation/core-api/memory-allocation.rst +++ b/Documentation/core-api/memory-allocation.rst @@ -1,3 +1,5 @@ +.. _memory-allocation: + === Memory Allocation Guide === diff --git a/Documentation/vm/index.rst b/Documentation/vm/index.rst index c4ded22197ca..2b3ab3a1ccf3 100644 --- a/Documentation/vm/index.rst +++ b/Documentation/vm/index.rst @@ -2,7 +2,9 @@ Linux Memory Management Documentation = -This is a collection of documents about Linux memory management (mm) subsystem. +This is a collection of documents about the Linux memory management (mm) +subsystem. If you are looking for advice on simply allocating memory, +see the :ref:`memory-allocation`. User guides for MM features ===
Re: [PATCH security-next v5 00/30] LSM: Explict ordering
On 10/24/2018 1:12 PM, Kees Cook wrote: > On Wed, Oct 24, 2018 at 1:56 AM, Casey Schaufler > wrote: >> On 10/23/2018 12:05 PM, Casey Schaufler wrote: >>> On 10/23/2018 11:50 AM, Kees Cook wrote: >>> Did you poke around at my combined series? https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=lsm/ordering-v6-blob-sharing >>> I hope to do that on the plane later today. >> I had a chance to poke at the combined series and it >> all seems to work as advertised. > /me stares at John, Paul, and Stephen. Hurry up and get off your planes! ;) > > -Kees > Now that we have next-general updated I'm porting these forward.
Inquiry 12/11/2018
Hi,friend, This is Daniel Murray and i am from Sinara Group Co.Ltd Group Co.,LTD in Russia. We are glad to know about your company from the web and we are interested in your products. Could you kindly send us your Latest catalog and price list for our trial order. Best Regards, Daniel Murray Purchasing Manager
Price Inquiry
Hi,friend, This is Daniel Murray and i am from Sinara Group Co.Ltd Group Co.,LTD in Russia. We are glad to know about your company from the web and we are interested in your products. Could you kindly send us your Latest catalog and price list for our trial order. Best Regards, Daniel Murray Purchasing Manager
[PATCH] docs/mm: update kmalloc kernel-doc description
Add references to GFP documentation and the memory-allocation.rst and remove
GFP_USER, GFP_DMA and GFP_NOIO descriptions.
While on it slightly change the formatting so that the list of GFP flags
will be rendered as "description" in the generated html.
Signed-off-by: Mike Rapoport
---
Probably this should go via the -mm tree as it touches include/linux/slab.h
Documentation/core-api/memory-allocation.rst | 2 +
include/linux/slab.h | 55 ++--
2 files changed, 29 insertions(+), 28 deletions(-)
diff --git a/Documentation/core-api/memory-allocation.rst
b/Documentation/core-api/memory-allocation.rst
index f8bb9aa..39f35eb 100644
--- a/Documentation/core-api/memory-allocation.rst
+++ b/Documentation/core-api/memory-allocation.rst
@@ -1,3 +1,5 @@
+.. _memory_allocation:
+
===
Memory Allocation Guide
===
diff --git a/include/linux/slab.h b/include/linux/slab.h
index 918f374..4a342eb 100644
--- a/include/linux/slab.h
+++ b/include/linux/slab.h
@@ -486,48 +486,47 @@ static __always_inline void *kmalloc_large(size_t size,
gfp_t flags)
* kmalloc is the normal method of allocating memory
* for objects smaller than page size in the kernel.
*
- * The @flags argument may be one of:
+ * The @flags argument may be one of the GFP flags defined at
+ * include/linux/gfp.h and described at
+ * :ref:`Documentation/core-api/mm-api.rst `
*
- * %GFP_USER - Allocate memory on behalf of user. May sleep.
+ * The recommended usage of the @flags is described at
+ * :ref:`Documentation/core-api/memory-allocation.rst `
*
- * %GFP_KERNEL - Allocate normal kernel ram. May sleep.
+ * Below is a brief outline of the most useful GFP flags
*
- * %GFP_ATOMIC - Allocation will not sleep. May use emergency pools.
- * For example, use this inside interrupt handlers.
+ * %GFP_KERNEL
+ * Allocate normal kernel ram. May sleep.
*
- * %GFP_HIGHUSER - Allocate pages from high memory.
+ * %GFP_NOWAIT
+ * Allocation will not sleep.
*
- * %GFP_NOIO - Do not do any I/O at all while trying to get memory.
+ * %GFP_ATOMIC
+ * Allocation will not sleep. May use emergency pools.
*
- * %GFP_NOFS - Do not make any fs calls while trying to get memory.
- *
- * %GFP_NOWAIT - Allocation will not sleep.
- *
- * %__GFP_THISNODE - Allocate node-local memory only.
- *
- * %GFP_DMA - Allocation suitable for DMA.
- * Should only be used for kmalloc() caches. Otherwise, use a
- * slab created with SLAB_DMA.
+ * %GFP_HIGHUSER
+ * Allocate memory from high memory on behalf of user.
*
* Also it is possible to set different flags by OR'ing
* in one or more of the following additional @flags:
*
- * %__GFP_HIGH - This allocation has high priority and may use emergency pools.
- *
- * %__GFP_NOFAIL - Indicate that this allocation is in no way allowed to fail
- * (think twice before using).
+ * %__GFP_HIGH
+ * This allocation has high priority and may use emergency pools.
*
- * %__GFP_NORETRY - If memory is not immediately available,
- * then give up at once.
+ * %__GFP_NOFAIL
+ * Indicate that this allocation is in no way allowed to fail
+ * (think twice before using).
*
- * %__GFP_NOWARN - If allocation fails, don't issue any warnings.
+ * %__GFP_NORETRY
+ * If memory is not immediately available,
+ * then give up at once.
*
- * %__GFP_RETRY_MAYFAIL - Try really hard to succeed the allocation but fail
- * eventually.
+ * %__GFP_NOWARN
+ * If allocation fails, don't issue any warnings.
*
- * There are other flags available as well, but these are not intended
- * for general use, and so are not documented here. For a full list of
- * potential flags, always refer to linux/gfp.h.
+ * %__GFP_RETRY_MAYFAIL
+ * Try really hard to succeed the allocation but fail
+ * eventually.
*/
static __always_inline void *kmalloc(size_t size, gfp_t flags)
{
--
2.7.4
[PATCH v2] docs/admin-guide/mm/concepts.rst: grammar and style fixups
Signed-off-by: Mike Rapoport Reviewed-by: Randy Dunlap --- v2: address Matthew's feedback Documentation/admin-guide/mm/concepts.rst | 51 --- 1 file changed, 26 insertions(+), 25 deletions(-) diff --git a/Documentation/admin-guide/mm/concepts.rst b/Documentation/admin-guide/mm/concepts.rst index 291699c..c2531b1 100644 --- a/Documentation/admin-guide/mm/concepts.rst +++ b/Documentation/admin-guide/mm/concepts.rst @@ -4,13 +4,13 @@ Concepts overview = -The memory management in Linux is complex system that evolved over the -years and included more and more functionality to support variety of +The memory management in Linux is a complex system that evolved over the +years and included more and more functionality to support a variety of systems from MMU-less microcontrollers to supercomputers. The memory -management for systems without MMU is called ``nommu`` and it +management for systems without an MMU is called ``nommu`` and it definitely deserves a dedicated document, which hopefully will be eventually written. Yet, although some of the concepts are the same, -here we assume that MMU is available and CPU can translate a virtual +here we assume that an MMU is available and a CPU can translate a virtual address to a physical address. .. contents:: :local: @@ -21,10 +21,10 @@ Virtual Memory Primer The physical memory in a computer system is a limited resource and even for systems that support memory hotplug there is a hard limit on the amount of memory that can be installed. The physical memory is not -necessary contiguous, it might be accessible as a set of distinct +necessarily contiguous; it might be accessible as a set of distinct address ranges. Besides, different CPU architectures, and even -different implementations of the same architecture have different view -how these address ranges defined. +different implementations of the same architecture have different views +of how these address ranges are defined. All this makes dealing directly with physical memory quite complex and to avoid this complexity a concept of virtual memory was developed. @@ -48,8 +48,8 @@ appropriate kernel configuration option. Each physical memory page can be mapped as one or more virtual pages. These mappings are described by page tables that allow -translation from virtual address used by programs to real address in -the physical memory. The page tables organized hierarchically. +translation from a virtual address used by programs to the physical +memory address. The page tables are organized hierarchically. The tables at the lowest level of the hierarchy contain physical addresses of actual pages used by the software. The tables at higher @@ -121,8 +121,8 @@ Nodes Many multi-processor machines are NUMA - Non-Uniform Memory Access - systems. In such systems the memory is arranged into banks that have different access latency depending on the "distance" from the -processor. Each bank is referred as `node` and for each node Linux -constructs an independent memory management subsystem. A node has it's +processor. Each bank is referred to as a `node` and for each node Linux +constructs an independent memory management subsystem. A node has its own set of zones, lists of free and used pages and various statistics counters. You can find more details about NUMA in :ref:`Documentation/vm/numa.rst ` and in @@ -149,9 +149,9 @@ for program's stack and heap or by explicit calls to mmap(2) system call. Usually, the anonymous mappings only define virtual memory areas that the program is allowed to access. The read accesses will result in creation of a page table entry that references a special physical -page filled with zeroes. When the program performs a write, regular +page filled with zeroes. When the program performs a write, a regular physical page will be allocated to hold the written data. The page -will be marked dirty and if the kernel will decide to repurpose it, +will be marked dirty and if the kernel decides to repurpose it, the dirty page will be swapped out. Reclaim @@ -181,8 +181,8 @@ pressure. The process of freeing the reclaimable physical memory pages and repurposing them is called (surprise!) `reclaim`. Linux can reclaim pages either asynchronously or synchronously, depending on the state -of the system. When system is not loaded, most of the memory is free -and allocation request will be satisfied immediately from the free +of the system. When the system is not loaded, most of the memory is free +and allocation requests will be satisfied immediately from the free pages supply. As the load increases, the amount of the free pages goes down and when it reaches a certain threshold (high watermark), an allocation request will awaken the ``kswapd`` daemon. It will @@ -190,7 +190,7 @@ asynchronously scan memory pages and either just free them if the data they contain is available elsewhere, or evict to the backing storage
Re: [PATCH v2] doc: tracing: Fix a number of typos
On Thu, 1 Nov 2018 09:57:17 -0400 Amir Livneh wrote: > Trivial fixes to spelling mistakes in ftrace.rst > > v2: tripple -> triple > > Signed-off-by: Amir Livneh Applied, thanks. jon
[PATCH] mm/gup: fix follow_page_mask kernel-doc comment
Commit df06b37ffe5a ("mm/gup: cache dev_pagemap while pinning pages")
modified the signature of follow_page_mask() function but left the
parameter description behind.
Update the description to make the code and comments agree again.
While on it, update formatting of the return value description to match
Documentation/doc-guide/kernel-doc.rst guidelines.
Signed-off-by: Mike Rapoport
---
mm/gup.c | 10 --
1 file changed, 8 insertions(+), 2 deletions(-)
diff --git a/mm/gup.c b/mm/gup.c
index f76e77a..aa43620 100644
--- a/mm/gup.c
+++ b/mm/gup.c
@@ -385,11 +385,17 @@ static struct page *follow_p4d_mask(struct vm_area_struct
*vma,
* @vma: vm_area_struct mapping @address
* @address: virtual address to look up
* @flags: flags modifying lookup behaviour
- * @page_mask: on output, *page_mask is set according to the size of the page
+ * @ctx: contains dev_pagemap for %ZONE_DEVICE memory pinning and a
+ * pointer to output page_mask
*
* @flags can have FOLL_ flags set, defined in
*
- * Returns the mapped (struct page *), %NULL if no mapping exists, or
+ * When getting pages from ZONE_DEVICE memory, the @ctx->pgmap caches
+ * the device's dev_pagemap metadata to avoid repeating expensive lookups.
+ *
+ * On output, the @ctx->page_mask is set according to the size of the page.
+ *
+ * Return: the mapped (struct page *), %NULL if no mapping exists, or
* an error pointer if there is a mapping to something not represented
* by a page descriptor (see also vm_normal_page()).
*/
--
2.7.4
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On Mon, Nov 05, 2018 at 11:29:27PM -0800, Randy Dunlap wrote: > On 11/5/18 10:35 PM, Mike Rapoport wrote: > > On Mon, Nov 05, 2018 at 01:12:40PM -0800, Matthew Wilcox wrote: > >> On Mon, Nov 05, 2018 at 09:58:15PM +0200, Mike Rapoport wrote: > >>> @@ -21,10 +21,10 @@ Virtual Memory Primer > >>> The physical memory in a computer system is a limited resource and > >>> even for systems that support memory hotplug there is a hard limit on > >>> the amount of memory that can be installed. The physical memory is not > >>> -necessary contiguous, it might be accessible as a set of distinct > >>> +necessary contiguous; it might be accessible as a set of distinct > >> > >> necessarily > >> > >>> address ranges. Besides, different CPU architectures, and even > >>> -different implementations of the same architecture have different view > >>> -how these address ranges defined. > >>> +different implementations of the same architecture have different views > >>> +of how these address ranges defined. > >> > >> "are defined"? > >> > >>> Each physical memory page can be mapped as one or more virtual > >>> pages. These mappings are described by page tables that allow > >>> -translation from virtual address used by programs to real address in > >>> -the physical memory. The page tables organized hierarchically. > >>> +translation from a virtual address used by programs to the real > >>> +address in the physical memory. The page tables are organized > >>> +hierarchically. > >> > >> I don't like the term "real address". Can we say "physical address in > >> memory" here, or "address of physical memory" or something? > > > > I didn't really like it as well, but I couldn't think of any better > > adjective to emphasize that address in the physical memory is "the real > > thing". > > > > Maybe the best would be to drop "real" and make it > > > > "translation from a virtual address used by programs to the > > address in the physical memory" > > physical memory address ? Works for me, thanks! > > -- > ~Randy > -- Sincerely yours, Mike.
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On 11/5/18 10:35 PM, Mike Rapoport wrote: > On Mon, Nov 05, 2018 at 01:12:40PM -0800, Matthew Wilcox wrote: >> On Mon, Nov 05, 2018 at 09:58:15PM +0200, Mike Rapoport wrote: >>> @@ -21,10 +21,10 @@ Virtual Memory Primer >>> The physical memory in a computer system is a limited resource and >>> even for systems that support memory hotplug there is a hard limit on >>> the amount of memory that can be installed. The physical memory is not >>> -necessary contiguous, it might be accessible as a set of distinct >>> +necessary contiguous; it might be accessible as a set of distinct >> >> necessarily >> >>> address ranges. Besides, different CPU architectures, and even >>> -different implementations of the same architecture have different view >>> -how these address ranges defined. >>> +different implementations of the same architecture have different views >>> +of how these address ranges defined. >> >> "are defined"? >> >>> Each physical memory page can be mapped as one or more virtual >>> pages. These mappings are described by page tables that allow >>> -translation from virtual address used by programs to real address in >>> -the physical memory. The page tables organized hierarchically. >>> +translation from a virtual address used by programs to the real >>> +address in the physical memory. The page tables are organized >>> +hierarchically. >> >> I don't like the term "real address". Can we say "physical address in >> memory" here, or "address of physical memory" or something? > > I didn't really like it as well, but I couldn't think of any better > adjective to emphasize that address in the physical memory is "the real > thing". > > Maybe the best would be to drop "real" and make it > > "translation from a virtual address used by programs to the > address in the physical memory" physical memory address ? -- ~Randy
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On Mon, Nov 05, 2018 at 01:12:40PM -0800, Matthew Wilcox wrote: > On Mon, Nov 05, 2018 at 09:58:15PM +0200, Mike Rapoport wrote: > > @@ -21,10 +21,10 @@ Virtual Memory Primer > > The physical memory in a computer system is a limited resource and > > even for systems that support memory hotplug there is a hard limit on > > the amount of memory that can be installed. The physical memory is not > > -necessary contiguous, it might be accessible as a set of distinct > > +necessary contiguous; it might be accessible as a set of distinct > > necessarily > > > address ranges. Besides, different CPU architectures, and even > > -different implementations of the same architecture have different view > > -how these address ranges defined. > > +different implementations of the same architecture have different views > > +of how these address ranges defined. > > "are defined"? > > > Each physical memory page can be mapped as one or more virtual > > pages. These mappings are described by page tables that allow > > -translation from virtual address used by programs to real address in > > -the physical memory. The page tables organized hierarchically. > > +translation from a virtual address used by programs to the real > > +address in the physical memory. The page tables are organized > > +hierarchically. > > I don't like the term "real address". Can we say "physical address in > memory" here, or "address of physical memory" or something? I didn't really like it as well, but I couldn't think of any better adjective to emphasize that address in the physical memory is "the real thing". Maybe the best would be to drop "real" and make it "translation from a virtual address used by programs to the address in the physical memory" > > -page filled with zeroes. When the program performs a write, regular > > +page filled with zeroes. When the program performs a write, a regular > > physical page will be allocated to hold the written data. The page > > will be marked dirty and if the kernel will decide to repurpose it, > > the dirty page will be swapped out. > > "if the kernel will decide to repurpose it" is awkward. How about > > if the kernel decides to repurpose it"? > > > OOM killer > > == > > > > -It may happen, that on a loaded machine memory will be exhausted. When > > +It may happen that on a loaded machine memory will be exhausted. When > > the kernel detects that the system runs out of memory (OOM) it invokes > > `OOM killer`. > > Its mission is simple: all it has to do is to select a > > task to sacrifice for the sake of the overall system health. The > > It is possible for the kernel to be unable to reclaim enough memory to > continue to operate. In order to save the rest of the system, it invokes > the `OOM killer` to sacrifice one task. How about: It is possible that on a loaded machine memory will be exhausted and the kernel will be unable to reclaim enough memory to continue to operate. In order to save the rest of the system, it invokes the `OOM killer`. The `OOM killer` selects a task to sacrifice for the sake of the overall system health. The selected task is killed in a hope that after it exits enough memory will be freed to continue normal operation. -- Sincerely yours, Mike.
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On Mon, Nov 05, 2018 at 09:58:15PM +0200, Mike Rapoport wrote: > @@ -21,10 +21,10 @@ Virtual Memory Primer > The physical memory in a computer system is a limited resource and > even for systems that support memory hotplug there is a hard limit on > the amount of memory that can be installed. The physical memory is not > -necessary contiguous, it might be accessible as a set of distinct > +necessary contiguous; it might be accessible as a set of distinct necessarily > address ranges. Besides, different CPU architectures, and even > -different implementations of the same architecture have different view > -how these address ranges defined. > +different implementations of the same architecture have different views > +of how these address ranges defined. "are defined"? > Each physical memory page can be mapped as one or more virtual > pages. These mappings are described by page tables that allow > -translation from virtual address used by programs to real address in > -the physical memory. The page tables organized hierarchically. > +translation from a virtual address used by programs to the real > +address in the physical memory. The page tables are organized > +hierarchically. I don't like the term "real address". Can we say "physical address in memory" here, or "address of physical memory" or something? > -page filled with zeroes. When the program performs a write, regular > +page filled with zeroes. When the program performs a write, a regular > physical page will be allocated to hold the written data. The page > will be marked dirty and if the kernel will decide to repurpose it, > the dirty page will be swapped out. "if the kernel will decide to repurpose it" is awkward. How about if the kernel decides to repurpose it"? > OOM killer > == > > -It may happen, that on a loaded machine memory will be exhausted. When > +It may happen that on a loaded machine memory will be exhausted. When > the kernel detects that the system runs out of memory (OOM) it invokes > `OOM killer`. > Its mission is simple: all it has to do is to select a > task to sacrifice for the sake of the overall system health. The It is possible for the kernel to be unable to reclaim enough memory to continue to operate. In order to save the rest of the system, it invokes the `OOM killer` to sacrifice one task.
Re: [PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
On 11/5/18 11:58 AM, Mike Rapoport wrote: > From: Mike Rapoport > > Signed-off-by: Mike Rapoport > Cc: Randy Dunlap > --- > > There was a couple of grammar fixes Randy suggested a while ago, but it > seems I've never sent them out. > > Documentation/admin-guide/mm/concepts.rst | 39 > --- > 1 file changed, 20 insertions(+), 19 deletions(-) Hi Mike, one nit: > @@ -121,8 +122,8 @@ Nodes > Many multi-processor machines are NUMA - Non-Uniform Memory Access - > systems. In such systems the memory is arranged into banks that have > different access latency depending on the "distance" from the > -processor. Each bank is referred as `node` and for each node Linux > -constructs an independent memory management subsystem. A node has it's > +processor. Each bank is referred as a `node` and for each node Linux is referred to as a > +constructs an independent memory management subsystem. A node has its > own set of zones, lists of free and used pages and various statistics > counters. You can find more details about NUMA in > :ref:`Documentation/vm/numa.rst ` and in Reviewed-by: Randy Dunlap thanks. -- ~Randy
[PATCH] docs/admin-guide/mm/concepts.rst: grammar fixups
From: Mike Rapoport Signed-off-by: Mike Rapoport Cc: Randy Dunlap --- There was a couple of grammar fixes Randy suggested a while ago, but it seems I've never sent them out. Documentation/admin-guide/mm/concepts.rst | 39 --- 1 file changed, 20 insertions(+), 19 deletions(-) diff --git a/Documentation/admin-guide/mm/concepts.rst b/Documentation/admin-guide/mm/concepts.rst index 291699c..ab7a0f9 100644 --- a/Documentation/admin-guide/mm/concepts.rst +++ b/Documentation/admin-guide/mm/concepts.rst @@ -4,13 +4,13 @@ Concepts overview = -The memory management in Linux is complex system that evolved over the -years and included more and more functionality to support variety of +The memory management in Linux is a complex system that evolved over the +years and included more and more functionality to support a variety of systems from MMU-less microcontrollers to supercomputers. The memory -management for systems without MMU is called ``nommu`` and it +management for systems without an MMU is called ``nommu`` and it definitely deserves a dedicated document, which hopefully will be eventually written. Yet, although some of the concepts are the same, -here we assume that MMU is available and CPU can translate a virtual +here we assume that an MMU is available and a CPU can translate a virtual address to a physical address. .. contents:: :local: @@ -21,10 +21,10 @@ Virtual Memory Primer The physical memory in a computer system is a limited resource and even for systems that support memory hotplug there is a hard limit on the amount of memory that can be installed. The physical memory is not -necessary contiguous, it might be accessible as a set of distinct +necessary contiguous; it might be accessible as a set of distinct address ranges. Besides, different CPU architectures, and even -different implementations of the same architecture have different view -how these address ranges defined. +different implementations of the same architecture have different views +of how these address ranges defined. All this makes dealing directly with physical memory quite complex and to avoid this complexity a concept of virtual memory was developed. @@ -48,8 +48,9 @@ appropriate kernel configuration option. Each physical memory page can be mapped as one or more virtual pages. These mappings are described by page tables that allow -translation from virtual address used by programs to real address in -the physical memory. The page tables organized hierarchically. +translation from a virtual address used by programs to the real +address in the physical memory. The page tables are organized +hierarchically. The tables at the lowest level of the hierarchy contain physical addresses of actual pages used by the software. The tables at higher @@ -121,8 +122,8 @@ Nodes Many multi-processor machines are NUMA - Non-Uniform Memory Access - systems. In such systems the memory is arranged into banks that have different access latency depending on the "distance" from the -processor. Each bank is referred as `node` and for each node Linux -constructs an independent memory management subsystem. A node has it's +processor. Each bank is referred as a `node` and for each node Linux +constructs an independent memory management subsystem. A node has its own set of zones, lists of free and used pages and various statistics counters. You can find more details about NUMA in :ref:`Documentation/vm/numa.rst ` and in @@ -149,7 +150,7 @@ for program's stack and heap or by explicit calls to mmap(2) system call. Usually, the anonymous mappings only define virtual memory areas that the program is allowed to access. The read accesses will result in creation of a page table entry that references a special physical -page filled with zeroes. When the program performs a write, regular +page filled with zeroes. When the program performs a write, a regular physical page will be allocated to hold the written data. The page will be marked dirty and if the kernel will decide to repurpose it, the dirty page will be swapped out. @@ -181,8 +182,8 @@ pressure. The process of freeing the reclaimable physical memory pages and repurposing them is called (surprise!) `reclaim`. Linux can reclaim pages either asynchronously or synchronously, depending on the state -of the system. When system is not loaded, most of the memory is free -and allocation request will be satisfied immediately from the free +of the system. When the system is not loaded, most of the memory is free +and allocation requests will be satisfied immediately from the free pages supply. As the load increases, the amount of the free pages goes down and when it reaches a certain threshold (high watermark), an allocation request will awaken the ``kswapd`` daemon. It will @@ -190,7 +191,7 @@ asynchronously scan memory pages and either just free them if the data they contain is available elsewhere, or evict to the
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Mon, Nov 5, 2018 at 5:15 AM Martin Schwidefsky wrote: > > This patch would work for me: Thanks, applied, Linus
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Mon, 5 Nov 2018 14:15:35 +0100
Martin Schwidefsky wrote:
>
> Follow-up question: the __no_sanitize_address_or_inline define has the
> 'notrace'
> option that is missing for __no_kasan_or_inline. We need that option for
> __load_psw_mask, if we do the replacement all users of __no_kasan_or_inline
> would get the option as well. This affects __read_once_size_nocheck and
> read_word_at_a_time. Do these function have to be traceable ?
Some history on why I added notrace to inline. It was to make things
more consistent. Since gcc wont add a fentry/mcount call to inlined
functions, having those functions show up in the trace depending on
whether or not gcc honored the "inline" tag made things confusing. And
it even once caused a crash when local_irq_save() stopped being
inlined.
I added notrace to inline so that if you mark something as inline, it
would not show up in the trace regardless of gcc deciding to inline the
function or not.
>
> This patch would work for me:
> --
> >From 4aaa09fe4b54e930edabac86606dee979b12647c Mon Sep 17 00:00:00 2001
> From: Martin Schwidefsky
> Date: Mon, 5 Nov 2018 07:36:28 +0100
> Subject: [PATCH] compiler: remove __no_sanitize_address_or_inline again
>
> The __no_sanitize_address_or_inline and __no_kasan_or_inline defines
> are almost identical. The only difference is that __no_kasan_or_inline
> does not have the 'notrace' attribute.
>
> To be able to replace __no_sanitize_address_or_inline with the older
> definition, add 'notrace' to __no_kasan_or_inline and change to two
> users of __no_sanitize_address_or_inline in the s390 code.
>
> The 'notrace' option is necessary for e.g. the __load_psw_mask function
> in arch/s390/include/asm/processor.h. Without the option it is possible
> to trace __load_psw_mask which leads to kernel stack overflow.
Acked-by: Steven Rostedt (VMware)
-- Steve
>
> Signed-off-by: Martin Schwidefsky
> ---
> arch/s390/include/asm/processor.h | 4 ++--
> include/linux/compiler-gcc.h | 12
> include/linux/compiler.h | 2 +-
> 3 files changed, 3 insertions(+), 15 deletions(-)
>
> diff --git a/arch/s390/include/asm/processor.h
> b/arch/s390/include/asm/processor.h
> index 302795c47c06..81038ab357ce 100644
> --- a/arch/s390/include/asm/processor.h
> +++ b/arch/s390/include/asm/processor.h
> @@ -236,7 +236,7 @@ static inline unsigned long current_stack_pointer(void)
> return sp;
> }
>
> -static __no_sanitize_address_or_inline unsigned short stap(void)
> +static __no_kasan_or_inline unsigned short stap(void)
> {
> unsigned short cpu_address;
>
> @@ -330,7 +330,7 @@ static inline void __load_psw(psw_t psw)
> * Set PSW mask to specified value, while leaving the
> * PSW addr pointing to the next instruction.
> */
> -static __no_sanitize_address_or_inline void __load_psw_mask(unsigned long
> mask)
> +static __no_kasan_or_inline void __load_psw_mask(unsigned long mask)
> {
> unsigned long addr;
> psw_t psw;
> diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
> index c0f5db3a9621..2010493e1040 100644
> --- a/include/linux/compiler-gcc.h
> +++ b/include/linux/compiler-gcc.h
> @@ -143,18 +143,6 @@
> #define KASAN_ABI_VERSION 3
> #endif
>
> -/*
> - * Because __no_sanitize_address conflicts with inlining:
> - * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
> - * we do one or the other.
> - */
> -#ifdef CONFIG_KASAN
> -#define __no_sanitize_address_or_inline
> \
> - __no_sanitize_address __maybe_unused notrace
> -#else
> -#define __no_sanitize_address_or_inline inline
> -#endif
> -
> #if GCC_VERSION >= 50100
> #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
> #endif
> diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> index 18c80cfa4fc4..06396c1cf127 100644
> --- a/include/linux/compiler.h
> +++ b/include/linux/compiler.h
> @@ -189,7 +189,7 @@ void __read_once_size(const volatile void *p, void *res,
> int size)
> * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
> * '__maybe_unused' allows us to avoid defined-but-not-used warnings.
> */
> -# define __no_kasan_or_inline __no_sanitize_address __maybe_unused
> +# define __no_kasan_or_inline __no_sanitize_address notrace __maybe_unused
> #else
> # define __no_kasan_or_inline __always_inline
> #endif
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Mon, 5 Nov 2018 07:02:56 +0100
Martin Schwidefsky wrote:
> On Fri, 2 Nov 2018 09:09:32 -0700
> Linus Torvalds wrote:
>
> > On Fri, Nov 2, 2018 at 2:43 AM Andrey Ryabinin
> > wrote:
> > >
> > > You're right, version checks shouldn't matter here. But
> > > __no_sanitize_address_or_inline
> > > shouldn't have been added in the first place, because we already have
> > > almost the same
> > >__no_kasan_or_inline:
> >
> > Ahh, very good.
> >
> > Vasily, Martin - since __no_sanitize_address_or_inline was added just
> > for s390, would you mind replacing it with __no_kasan_or_inline
> > instead, and testing that in whatever failed before?
> >
> > Then we can just remove that unnecessary symbol #define entirely..
>
> Ok, will do.
Follow-up question: the __no_sanitize_address_or_inline define has the 'notrace'
option that is missing for __no_kasan_or_inline. We need that option for
__load_psw_mask, if we do the replacement all users of __no_kasan_or_inline
would get the option as well. This affects __read_once_size_nocheck and
read_word_at_a_time. Do these function have to be traceable ?
This patch would work for me:
--
>From 4aaa09fe4b54e930edabac86606dee979b12647c Mon Sep 17 00:00:00 2001
From: Martin Schwidefsky
Date: Mon, 5 Nov 2018 07:36:28 +0100
Subject: [PATCH] compiler: remove __no_sanitize_address_or_inline again
The __no_sanitize_address_or_inline and __no_kasan_or_inline defines
are almost identical. The only difference is that __no_kasan_or_inline
does not have the 'notrace' attribute.
To be able to replace __no_sanitize_address_or_inline with the older
definition, add 'notrace' to __no_kasan_or_inline and change to two
users of __no_sanitize_address_or_inline in the s390 code.
The 'notrace' option is necessary for e.g. the __load_psw_mask function
in arch/s390/include/asm/processor.h. Without the option it is possible
to trace __load_psw_mask which leads to kernel stack overflow.
Signed-off-by: Martin Schwidefsky
---
arch/s390/include/asm/processor.h | 4 ++--
include/linux/compiler-gcc.h | 12
include/linux/compiler.h | 2 +-
3 files changed, 3 insertions(+), 15 deletions(-)
diff --git a/arch/s390/include/asm/processor.h
b/arch/s390/include/asm/processor.h
index 302795c47c06..81038ab357ce 100644
--- a/arch/s390/include/asm/processor.h
+++ b/arch/s390/include/asm/processor.h
@@ -236,7 +236,7 @@ static inline unsigned long current_stack_pointer(void)
return sp;
}
-static __no_sanitize_address_or_inline unsigned short stap(void)
+static __no_kasan_or_inline unsigned short stap(void)
{
unsigned short cpu_address;
@@ -330,7 +330,7 @@ static inline void __load_psw(psw_t psw)
* Set PSW mask to specified value, while leaving the
* PSW addr pointing to the next instruction.
*/
-static __no_sanitize_address_or_inline void __load_psw_mask(unsigned long mask)
+static __no_kasan_or_inline void __load_psw_mask(unsigned long mask)
{
unsigned long addr;
psw_t psw;
diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index c0f5db3a9621..2010493e1040 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -143,18 +143,6 @@
#define KASAN_ABI_VERSION 3
#endif
-/*
- * Because __no_sanitize_address conflicts with inlining:
- * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
- * we do one or the other.
- */
-#ifdef CONFIG_KASAN
-#define __no_sanitize_address_or_inline
\
- __no_sanitize_address __maybe_unused notrace
-#else
-#define __no_sanitize_address_or_inline inline
-#endif
-
#if GCC_VERSION >= 50100
#define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
#endif
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 18c80cfa4fc4..06396c1cf127 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -189,7 +189,7 @@ void __read_once_size(const volatile void *p, void *res,
int size)
* https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
* '__maybe_unused' allows us to avoid defined-but-not-used warnings.
*/
-# define __no_kasan_or_inline __no_sanitize_address __maybe_unused
+# define __no_kasan_or_inline __no_sanitize_address notrace __maybe_unused
#else
# define __no_kasan_or_inline __always_inline
#endif
--
2.16.4
--
blue skies,
Martin.
"Reality continues to ruin my life." - Calvin.
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Fri, 2 Nov 2018 09:09:32 -0700 Linus Torvalds wrote: > On Fri, Nov 2, 2018 at 2:43 AM Andrey Ryabinin > wrote: > > > > You're right, version checks shouldn't matter here. But > > __no_sanitize_address_or_inline > > shouldn't have been added in the first place, because we already have > > almost the same > >__no_kasan_or_inline: > > Ahh, very good. > > Vasily, Martin - since __no_sanitize_address_or_inline was added just > for s390, would you mind replacing it with __no_kasan_or_inline > instead, and testing that in whatever failed before? > > Then we can just remove that unnecessary symbol #define entirely.. Ok, will do. -- blue skies, Martin. "Reality continues to ruin my life." - Calvin.
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On 11/02/2018 07:11 PM, Linus Torvalds wrote: > On Fri, Nov 2, 2018 at 6:16 AM Andrey Ryabinin > wrote: >> >> On 11/02/2018 04:46 AM, Linus Torvalds wrote: >>> >>> So I _think_ the KASAN config should have a >>> >>> depends on CC_IS_GCC && GCC_VERSION >= 40902 >>> >>> on it, but maybe there is something I'm missing. >> >> I'd rather use cc-option instead of version check, since we also support >> clang. > > That would be even better, but I thought the requirement for 4.9.2 > came not from the option existing, but because of some bugs getting > fixed? It looks unusual but -fsanitize=kernel-address really showed up only in 4.9.2. It was actually backported from 5.0 branch to 4.9 for whatever reason. > > But if we can do it without version checks, that would be lovely. > > Linus >
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Fri, Nov 2, 2018 at 6:16 AM Andrey Ryabinin wrote: > > On 11/02/2018 04:46 AM, Linus Torvalds wrote: > > > > So I _think_ the KASAN config should have a > > > > depends on CC_IS_GCC && GCC_VERSION >= 40902 > > > > on it, but maybe there is something I'm missing. > > I'd rather use cc-option instead of version check, since we also support > clang. That would be even better, but I thought the requirement for 4.9.2 came not from the option existing, but because of some bugs getting fixed? But if we can do it without version checks, that would be lovely. Linus
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Fri, Nov 2, 2018 at 2:43 AM Andrey Ryabinin wrote: > > You're right, version checks shouldn't matter here. But > __no_sanitize_address_or_inline > shouldn't have been added in the first place, because we already have almost > the same >__no_kasan_or_inline: Ahh, very good. Vasily, Martin - since __no_sanitize_address_or_inline was added just for s390, would you mind replacing it with __no_kasan_or_inline instead, and testing that in whatever failed before? Then we can just remove that unnecessary symbol #define entirely.. Thanks, Linus
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On 11/02/2018 04:46 AM, Linus Torvalds wrote: > On Thu, Nov 1, 2018 at 10:06 AM Linus Torvalds > wrote: >> >> The logic for using __no_sanitize_address *used* to be >> >> #if GCC_VERSION >= 40902 > > Ok, looking around, I think this has less to do with the attribute > being recognized, and simply just being because KASAN itself wants > gcc-4.9.2. > > I'm actually not seeing that KASAN dependency in the Kconfig scripts > (and it probably _should_ be now that we can just add compiler version > dependencies there), but that explains why the gcc version check is > different from "gcc supports the attribute". > > Anyway, I decided to do the merge by just getting rid of the > GCC_VERSION check around __no_sanitize_address_or_inline entirely. If > you enable KASAN, then a function with that marking just won't be > marked inline. > > End result: pulled. I'm as confused as you are as to why > __no_sanitize_address_or_inline is in the gcc header, but I guess it > ends up being the same issue: KASAN depends on gcc even if that > dependency doesn't seem to be spelled out in lib/Kconfig.kasan. > > So I _think_ the KASAN config should have a > > depends on CC_IS_GCC && GCC_VERSION >= 40902 > > on it, but maybe there is something I'm missing. > I'd rather use cc-option instead of version check, since we also support clang. It should be possible to express compiler requirements for subfeatures like stack/inline instrumentation in Kconfig. But that would be not that trivial, see the scripts/Makefile.kasan > But from a pull standpoint, I don't want to mess with those > (unrelated) issues, so I just kept the merge resolution as simple and > straightforward as possible. > > Miguel, please do double-check the merge (it's not pushed out yet, I'm > doing the usual build tests etc first). > > Linus >
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Fri, Nov 2, 2018 at 2:52 AM Linus Torvalds
wrote:
>
> Anyway, I decided to do the merge by just getting rid of the
> GCC_VERSION check around __no_sanitize_address_or_inline entirely. If
> you enable KASAN, then a function with that marking just won't be
> marked inline.
I was a bit confused when reading the gcc bug reports, i.e. why gcc
did *not* complain in 4.9 but it did in 5.1 (when it was supposed to
complain also in 4.9). It turns out that gcc 5.1 takes into account
who is the actual caller due to this change:
+ cgraph_node *caller = e->caller->global.inlined_to
+ ? e->caller->global.inlined_to : e->caller;
...
- else if (!sanitize_attrs_match_for_inline_p (e->caller->decl, callee->decl))
+ else if (!sanitize_attrs_match_for_inline_p (caller->decl, callee->decl))
change; e.g. this:
#define really_inline inline __attribute__((always_inline))
#define no_sanitize __attribute__((no_sanitize_address))
really_inline void f() {}
really_inline void g() { f(); }
no_sanitize void h() { g(); }
Complains in gcc 4.9 -O0, 5.1 -O0 and 5.1 -O2; but *not* in 4.9 -O2.
https://godbolt.org/z/kNApqk
Anyway, this is orthogonal but in case it clarifies that for someone else...
> Miguel, please do double-check the merge (it's not pushed out yet, I'm
> doing the usual build tests etc first).
I was sleeping, didn't manage to see it (in your GitHub, I guess?).
I did the merge myself, and arrived at the same thing as you. I
quickly inspected the rest and seems fine. By the way, I spotted an
extra space at:
+ * we do one or the other.
Cheers,
Miguel
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On 11/01/2018 08:06 PM, Linus Torvalds wrote: > On Mon, Oct 22, 2018 at 3:59 AM Miguel Ojeda > wrote: >> >> Here it is the Compiler Attributes series/tree, which tries to disentangle >> the include/linux/compiler*.h headers and bring them up to date. > > I've finally emptied the "normal" pull queues, and am looking at the > odd ones that I felt I needed to review more in-depth. > > This looked fine to me, and I already pulled, but when looking at the > conflict for __no_sanitize_address_or_inline, I also noticed that you > actually changed the gcc version logic. > > The logic for using __no_sanitize_address *used* to be > > #if GCC_VERSION >= 40902 > > but you have changed it to > > # define __GCC4_has_attribute___no_sanitize_address__ (__GNUC_MINOR__ >= 8) > > so now it's gcc-4.8+ rather than gcc-4.9.2+. As you said in other email - "this has less to do with the attribute being recognized, and simply just being because KASAN itself wants gcc-4.9.2." gcc < 4.9.2 simply doesn't have -fsanitize=kernel-address. But no_sanitize attribute originally comes with user space ASAN (-fsanitize=address) which exist in earlier GCCs like 4.8. Checking against 4.8 gcc should be fine. If we compile the kernel with gcc 4.8 it will be compiled without instrumentation, so the attribute won't have any effect. > > I'm not sure how much that matters (maybe the original check for 4.9.2 > was just a random pick by Andrey? Added to cc), but together with the > movement to that looks like it also > wouldn't want the CONFIG_KASAN tests, I wonder what the right merge > resolution would be. > > Yes, I see the resolution in linux-next, and I think that one is odd > and dubious, and now it *mixes* that old test of gcc-4.9.2 with the > different test in compiler-attributes. > > I'm _inclined_ to just do > > #ifdef CONFIG_KASAN > #define __no_sanitize_address_or_inline \ > __no_sanitize_address __maybe_unused notrace > #else > #define __no_sanitize_address_or_inline inline > #endif > > without any compiler versions at all, because I don't think it matters > (maybe we won't get address sanitation, and we will not get inlining > either, but do we care?). You're right, version checks shouldn't matter here. But __no_sanitize_address_or_inline shouldn't have been added in the first place, because we already have almost the same __no_kasan_or_inline: #ifdef CONFIG_KASAN /* * We can't declare function 'inline' because __no_sanitize_address confilcts * with inlining. Attempt to inline it may cause a build failure. * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368 * '__maybe_unused' allows us to avoid defined-but-not-used warnings. */ # define __no_kasan_or_inline __no_sanitize_address __maybe_unused #else # define __no_kasan_or_inline __always_inline #endif There are small differences like absence of notrace, but notrace could be added to the function together with __no_kasan_or_inline. And inline is *not* redefined to __always_inline only on x86 when CONFIG_OPTIMIZE_INLINING=y So there is absolutely no reason to have both __no_kasan_or_inline and __no_sanitize_address_or_inline.
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Thu, Nov 1, 2018 at 10:06 AM Linus Torvalds wrote: > > The logic for using __no_sanitize_address *used* to be > > #if GCC_VERSION >= 40902 Ok, looking around, I think this has less to do with the attribute being recognized, and simply just being because KASAN itself wants gcc-4.9.2. I'm actually not seeing that KASAN dependency in the Kconfig scripts (and it probably _should_ be now that we can just add compiler version dependencies there), but that explains why the gcc version check is different from "gcc supports the attribute". Anyway, I decided to do the merge by just getting rid of the GCC_VERSION check around __no_sanitize_address_or_inline entirely. If you enable KASAN, then a function with that marking just won't be marked inline. End result: pulled. I'm as confused as you are as to why __no_sanitize_address_or_inline is in the gcc header, but I guess it ends up being the same issue: KASAN depends on gcc even if that dependency doesn't seem to be spelled out in lib/Kconfig.kasan. So I _think_ the KASAN config should have a depends on CC_IS_GCC && GCC_VERSION >= 40902 on it, but maybe there is something I'm missing. But from a pull standpoint, I don't want to mess with those (unrelated) issues, so I just kept the merge resolution as simple and straightforward as possible. Miguel, please do double-check the merge (it's not pushed out yet, I'm doing the usual build tests etc first). Linus
Re: [GIT PULL] Compiler Attributes for v4.20-rc1
On Thu, Nov 1, 2018 at 6:06 PM Linus Torvalds
wrote:
>
> I'm not sure how much that matters (maybe the original check for 4.9.2
> was just a random pick by Andrey? Added to cc), but together with the
> movement to that looks like it also
> wouldn't want the CONFIG_KASAN tests, I wonder what the right merge
> resolution would be.
Good catch. I don't recall any special logic when I did that change,
so most likely I simply did like for the rest of the attributes and
took a look at when it was first supported (and documentation in gcc's
docs) in order to implement __has_attribute by hand.
But indeed, it *may* be that there is an (undocumented) problem
between 4.8 <= gcc < 4.9.2 with it. If so, we should document it down
and fix it. Andrey?
> Yes, I see the resolution in linux-next, and I think that one is odd
> and dubious, and now it *mixes* that old test of gcc-4.9.2 with the
> different test in compiler-attributes.
I missed that conflict completely, my bad (I did not miss all of them,
at least; one required fixing).
Hm at a quick look, why is it only on compiler-gcc.h? It should
either have a corresponding #define elsewhere or just be put directly
in another common header, no? (Adding Vasily & Martin to CC.)
> But I'm also unsure whether you meant for the "__has_attribute()" test
> to be usable outside the linux/compiler_attributes.h file, in which
> case I could just do
>
> #if defined(CONFIG_KASAN) && __has_attribute(__no_sanitize_address__)
>
> instead.
I think that (using __has_attribute() outside) may be a good idea: I
wanted to keep compiler_attributes.h as simple as possible by avoiding
#ifdefs inside that header (except for __has_attribute itself), as an
attempt to avoid going back to the mess of #ifdefs we had previously.
Basically, keeping the attributes in compiler_attributes.h that do not
depend on complex logic. So using __has_attribute *outside* the header
actually goes well with that principle, because it helps keeping stuff
out of it if they depend on other config options; without having to
rely on GCC_VERSION either.
[By the way, in case it clarifies: note that "optional" in that file
actually is a bit of a misnomer. I meant to say "optional" as in "not
supported by all compilers, so conditionally defined" ("optionally"
defined); rather than "optional" in the sense of "code still works
without the attribute". It caught Rasmus in one of his patches sent a
few days ago on top of this tree, so I want to change it or explain it
to avoid confusion.]
Cheers,
Miguel
Re: [PATCH] doc: tracing: Fix a number of typos
On 2018-10-31 14:16, Amir Livneh wrote: > @@ -2978,7 +2978,7 @@ The following commands are supported: >When the function is hit, it will dump the contents of the ftrace >ring buffer to the console. This is useful if you need to debug >something, and want to dump the trace when a certain function > - is hit. Perhaps its a function that is called before a tripple > + is hit. Perhaps it's a function that is called before a tripple >fault happens and does not allow you to get a regular dump. > > - cpudump: "tripple" -> "triple", while you're at it? - Will
[PATCH] doc: tracing: Fix a number of typos
Trivial fixes to spelling mistakes in ftrace.rst Signed-off-by: Amir Livneh --- Documentation/trace/ftrace.rst | 14 +++--- 1 file changed, 7 insertions(+), 7 deletions(-) diff --git a/Documentation/trace/ftrace.rst b/Documentation/trace/ftrace.rst index 7ea16a0ceffc..80fde12e6564 100644 --- a/Documentation/trace/ftrace.rst +++ b/Documentation/trace/ftrace.rst @@ -24,13 +24,13 @@ It can be used for debugging or analyzing latencies and performance issues that take place outside of user-space. Although ftrace is typically considered the function tracer, it -is really a frame work of several assorted tracing utilities. +is really a framework of several assorted tracing utilities. There's latency tracing to examine what occurs between interrupts disabled and enabled, as well as for preemption and from a time a task is woken to the task is actually scheduled in. One of the most common uses of ftrace is the event tracing. -Through out the kernel is hundreds of static event points that +Throughout the kernel is hundreds of static event points that can be enabled via the tracefs file system to see what is going on in certain parts of the kernel. @@ -462,7 +462,7 @@ of ftrace. Here is a list of some of the key files: mono_raw: This is the raw monotonic clock (CLOCK_MONOTONIC_RAW) - which is montonic but is not subject to any rate adjustments + which is monotonic but is not subject to any rate adjustments and ticks at the same rate as the hardware clocksource. boot: @@ -914,8 +914,8 @@ The above is mostly meaningful for kernel developers. current trace and the next trace. - '$' - greater than 1 second - - '@' - greater than 100 milisecond - - '*' - greater than 10 milisecond + - '@' - greater than 100 millisecond + - '*' - greater than 10 millisecond - '#' - greater than 1000 microsecond - '!' - greater than 100 microsecond - '+' - greater than 10 microsecond @@ -2541,7 +2541,7 @@ At compile time every C file object is run through the recordmcount program (located in the scripts directory). This program will parse the ELF headers in the C object to find all the locations in the .text section that call mcount. Starting -with gcc verson 4.6, the -mfentry has been added for x86, which +with gcc version 4.6, the -mfentry has been added for x86, which calls "__fentry__" instead of "mcount". Which is called before the creation of the stack frame. @@ -2978,7 +2978,7 @@ The following commands are supported: When the function is hit, it will dump the contents of the ftrace ring buffer to the console. This is useful if you need to debug something, and want to dump the trace when a certain function - is hit. Perhaps its a function that is called before a tripple + is hit. Perhaps it's a function that is called before a tripple fault happens and does not allow you to get a regular dump. - cpudump: -- 2.13.5
Re: [PATCH v10 4/9] dt-bindings: i3c: Document core bindings
On Fri, 26 Oct 2018 16:43:28 +0200, Boris Brezillon wrote:
> A new I3C subsystem has been added and a generic description has been
> created to represent the I3C bus and the devices connected on it.
>
> Document this generic representation.
>
> Cc: Rob Herring
> Signed-off-by: Boris Brezillon
> ---
> Changes in v10:
> - Fix mistakes in the example (reported by Rob)
>
> Changes in v9:
> - Rework the way we encode the I2C vs I3C device in the reg property
> so that we don't need the funky macros to define and I3C or I2C dev
> - Drop Rob's Reviewed-by
>
> Changes in v8:
> - None
>
> Changes in v7:
> - None
>
> Changes in v6:
> - None
>
> Changes in v5:
> - Add Rob's R-b
>
> Changes in v4:
> - Clarify the fact that static address == I3C address and dynamic
> address == I3C address
> - Use i2c-scl-hz in the example
>
> Changes in v3:
> - Rename {i2c,i3c}-scl-frequency DT prop into {i2c,i3c}-scl-hz
> - Rework the way we expose the provisional ID and LVR information
> - Rename dynamic-address into assigned-address
> - Enforce the I3C master node name
>
> Changes in v2:
> - Define how to describe I3C devices in the DT and when it should be
> used. Note that the parsing of I3C devices is not yet implemented in
> the framework. Will be added when someone really needs it.
> ---
> Documentation/devicetree/bindings/i3c/i3c.txt | 139 ++
> 1 file changed, 139 insertions(+)
> create mode 100644 Documentation/devicetree/bindings/i3c/i3c.txt
>
Reviewed-by: Rob Herring
Re: [PATCH security-next v5 00/30] LSM: Explict ordering
On Wed, Oct 24, 2018 at 1:56 AM, Casey Schaufler wrote: > On 10/23/2018 12:05 PM, Casey Schaufler wrote: >> On 10/23/2018 11:50 AM, Kees Cook wrote: >> >>> Did you poke around at my combined series? >>> https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=lsm/ordering-v6-blob-sharing >> I hope to do that on the plane later today. > > I had a chance to poke at the combined series and it > all seems to work as advertised. /me stares at John, Paul, and Stephen. Hurry up and get off your planes! ;) -Kees -- Kees Cook
Re: [PATCH security-next v5 00/30] LSM: Explict ordering
On 10/23/2018 12:05 PM, Casey Schaufler wrote: > On 10/23/2018 11:50 AM, Kees Cook wrote: > >> Did you poke around at my combined series? >> https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=lsm/ordering-v6-blob-sharing > I hope to do that on the plane later today. I had a chance to poke at the combined series and it all seems to work as advertised.
Re: [PATCH security-next v5 00/30] LSM: Explict ordering
On 10/23/2018 11:50 AM, Kees Cook wrote: > On Tue, Oct 23, 2018 at 9:48 AM, Casey Schaufler > wrote: >> On 10/12/2018 12:01 PM, Kees Cook wrote: >>> On Friday, October 12, 2018 3:19 AM, John Johansen >>> wrote: It isn't perfect but it manages consistency across distros as best as can be achieved atm. >>> Yeah, this is why I'm okay with the current series: it provides as >>> consistent a view as possible, but leaves room for future improvements >>> (like adding "+" or "!" or "all" or whatever). >>> >>> I'm curious to see what SELinux folks think of v5, though. I *think* I >>> addressed all the concerns there, even Paul's "I want my distro >>> default to not have extreme stacking" case too. >>> >>> -Kees >> Looks like I should go on vacation more often. :) >> >> I am generally opposed to fancy specification languages. >> I support the explicit lsm= list specification because you >> don't have to know any context to create a boot line that >> will work, and be as close to what you've specified as possible >> for the kernel configuration. One need look no further than >> the mechanisms for setting POSIX ACLs for an example of >> how to ensure a feature isn't used. >> >> Had we the foresight to make security= take a list of >> modules when Yama was added we might have avoided some of >> this brouhaha, but there was no reason to expect that stacking >> was ever going to happen back then. > This sounds like an "Ack" for you? :) I'll harass everyone in person > in a couple days. Acked-by: Casey Schaufler > Did you poke around at my combined series? > https://git.kernel.org/pub/scm/linux/kernel/git/kees/linux.git/log/?h=lsm/ordering-v6-blob-sharing I hope to do that on the plane later today. > -Kees
Re: [PATCH 1/2] Compiler Attributes: add support for __fallthrough (gcc >= 7.1)
On Mon, Oct 22, 2018 at 11:15 AM Bernd Petrovitsch
wrote:
>
> Hi all!
>
> On 22/10/18 19:54, Nick Desaulniers wrote:
> > On Mon, Oct 22, 2018 at 10:50 AM Bernd Petrovitsch
> > wrote:
> [...]
> >> PS: clang++ errors with "fallthrough annotation in unreachable code" if
> >> [[fallthrough]] is after an assert(). clang-devs there, please, the
> >> fallthrough doesn't really generated code (I hope;-).
> [...]
> > Can you send me a link to a simple reproducer in godbolt (godbolt.org)
> > and we'll take a look?
>
> Does https://godbolt.org/z/2Y4zIo do it - I'm a godbolt-newbie?
Moving the kernel folks to bcc, since we don't need to be discussing
C++ on LKML.
https://godbolt.org/z/B1fo9Z shows that this works as intended, for
cases that cannot be statically proven. I guess I'm looking for a
more realistic code sample to show why putting a `break;` statement
there is untenable?
>
> For
> snip
> #include
>
> int main(void)
> {
> switch (1) {
> default:
> assert(0);
> [[fallthrough]];
> case 1:
> ;
> }
> return 0;
> }
> snip
> Just "clang++ -Wimplicit-fallthrough -Werror" it .
>
> MfG,
> Bernd
> --
> "I dislike type abstraction if it has no real reason. And saving
> on typing is not a good reason - if your typing speed is the main
> issue when you're coding, you're doing something seriously wrong."
> - Linus Torvalds
--
Thanks,
~Nick Desaulniers
[GIT PULL] Compiler Attributes for v4.20-rc1
Hi Linus, Here it is the Compiler Attributes series/tree, which tries to disentangle the include/linux/compiler*.h headers and bring them up to date. The patches have been in linux-next for a while, *except* the last two from Nick which came a bit later. Since AFAIU there will be no linux-next this week, I included them here; but let me know if you prefer to take them out. You may see merge conflicts from a few other trees, from what we have seen in linux-next. I am not sure if you followed this, so please let me know if there is anything you don't agree with. And welcome back! Cheers, Miguel The following changes since commit 17b57b1883c1285f3d0dc2266e8f79286a7bef38: Linux 4.19-rc6 (2018-09-30 07:15:35 -0700) are available in the Git repository at: https://github.com/ojeda/linux.git tags/compiler-attributes-for-linus-4.20-rc1 for you to fetch changes up to 1ff2fea5e30ca15752777441ecb64a169fe22e9e: compiler-gcc: remove comment about gcc 4.5 from unreachable() (2018-10-19 08:47:30 +0200) The Compiler Attributes series This is an effort to disentangle the include/linux/compiler*.h headers and bring them up to date. The main idea behind the series is to use feature checking macros (i.e. __has_attribute) instead of compiler version checks (e.g. GCC_VERSION), which are compiler-agnostic (so they can be shared, reducing the size of compiler-specific headers) and version-agnostic. Other related improvements have been performed in the headers as well, which on top of the use of __has_attribute it has amounted to a significant simplification of these headers (e.g. GCC_VERSION is now only guarding a few non-attribute macros). This series should also help the efforts to support compiling the kernel with clang and icc. A fair amount of documentation and comments have also been added, clarified or removed; and the headers are now more readable, which should help kernel developers in general. The series was triggered due to the move to gcc >= 4.6. In turn, this series has also triggered Sparse to gain the ability to recognize __has_attribute on its own. Finally, the __nonstring variable attribute series has been also applied on top; plus two related patches from Nick Desaulniers for unreachable() that came a bit afterwards. Miguel Ojeda (15): Compiler Attributes: remove unused attributes Compiler Attributes: always use the extra-underscores syntax Compiler Attributes: remove unneeded tests Compiler Attributes: homogenize __must_be_array Compiler Attributes: remove unneeded sparse (__CHECKER__) tests Compiler Attributes: add missing SPDX ID in compiler_types.h Compiler Attributes: use feature checks instead of version checks Compiler Attributes: KENTRY used twice the "used" attribute Compiler Attributes: remove uses of __attribute__ from compiler.h Compiler Attributes: add Doc/process/programming-language.rst Compiler Attributes: add MAINTAINERS entry Compiler Attributes: add support for __nonstring (gcc >= 8) Compiler Attributes: enable -Wstringop-truncation on W=1 (gcc >= 8) Compiler Attributes: auxdisplay: panel: use __nonstring Compiler Attributes: ext4: remove local __nonstring definition ndesaulni...@google.com (2): compiler.h: update definition of unreachable() compiler-gcc: remove comment about gcc 4.5 from unreachable() Documentation/process/index.rst| 1 + Documentation/process/programming-language.rst | 45 + MAINTAINERS| 5 + drivers/auxdisplay/panel.c | 7 +- fs/ext4/ext4.h | 9 - include/linux/compiler-clang.h | 5 - include/linux/compiler-gcc.h | 74 +-- include/linux/compiler-intel.h | 9 - include/linux/compiler.h | 24 +-- include/linux/compiler_attributes.h| 258 + include/linux/compiler_types.h | 101 ++ scripts/Makefile.extrawarn | 1 + 12 files changed, 345 insertions(+), 194 deletions(-) create mode 100644 Documentation/process/programming-language.rst create mode 100644 include/linux/compiler_attributes.h
[PATCH] doc: fix a typo in adding-syscalls.rst
From: Guillaume Dore There was a typo in adding-syscalls.rst that could mislead developers to add a C filename in a makefile instead of an object filename. This error, while not keeping developers from contributing could slow the development process down by introducing build errors. Signed-off-by: Guillaume Dore --- Documentation/process/adding-syscalls.rst | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/Documentation/process/adding-syscalls.rst b/Documentation/process/adding-syscalls.rst index 0d4f29bc798b..88a7d5c8bb2f 100644 --- a/Documentation/process/adding-syscalls.rst +++ b/Documentation/process/adding-syscalls.rst @@ -232,7 +232,7 @@ normally be optional, so add a ``CONFIG`` option (typically to by the option. - Make the option depend on EXPERT if it should be hidden from normal users. - Make any new source files implementing the function dependent on the CONFIG - option in the Makefile (e.g. ``obj-$(CONFIG_XYZZY_SYSCALL) += xyzzy.c``). + option in the Makefile (e.g. ``obj-$(CONFIG_XYZZY_SYSCALL) += xyzzy.o``). - Double check that the kernel still builds with the new CONFIG option turned off. -- 2.19.1
Re: [PATCH] doc: fix a typo in adding-syscalls.rst
On Thu, 18 Oct 2018 17:47:50 +0200 cor...@poussif.eu wrote: > There was a typo in adding-syscalls.rst that could mislead developers > to add a C filename in a makefile instead of an object filename. > This error, while not keeping developers from contributing could slow > the development process down by introducing build errors. > > Signed-off-by: Guillaume Dore Applied, thanks. jon
[no subject]
Hallo Am Mrs Mavis Wancyzk, Sie haben eine Spende von 2,800,000.00EUR Ich gewann die America Lottery im Wert von $ 758.7 Millionen und ich spende einen Teil davon an fnf glckliche Menschen und Wohlttigkeits-Huser in Erinnerung an meinen verstorbenen Ehemann, der an Krebs gestorben ist. Kontaktieren Sie mich fr weitere Details unter: [maviswanczy...@hotmail.com]
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Mon, 15 Oct 2018 02:55:49 -0700 Christoph Hellwig wrote: > > OK, I've had a long conversation with the LF lawyer, and she said clearly > > that we really should not be introducing CC-SA material into the kernel > > source tree. It's a pain; I really do like CC-SA better for > > documentation, but it's not GPL-compatible, and that creates a problem for > > the processed docs. > > That was exactly my concern with this patchset. > > Btw, I think we have the same issue with idr.rst, and unless we can > relicense it we should drop it from the tree, as it actually includes > kernel source files. ...and a significant DOC section at that, not just function prototypes. I'd already asked Willy [CC'd] about this once, haven't gotten an answer yet. Hopefully we can address this once he comes back around. Thanks, jon
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Thu, Oct 11, 2018 at 11:27:35AM -0600, Jonathan Corbet wrote: > On Sat, 6 Oct 2018 10:51:54 +1000 > Dave Chinner wrote: > > > Can you let us know whether the CC-by-SA 4.0 license is acceptible > > or not? That's really the only thing that we need clarified at this > > point - if it's OK I'll to pull this into the XFS tree for the 4.20 > > merge window. If not, we'll go back to the drawing board > > OK, I've had a long conversation with the LF lawyer, and she said clearly > that we really should not be introducing CC-SA material into the kernel > source tree. It's a pain; I really do like CC-SA better for > documentation, but it's not GPL-compatible, and that creates a problem for > the processed docs. That was exactly my concern with this patchset. Btw, I think we have the same issue with idr.rst, and unless we can relicense it we should drop it from the tree, as it actually includes kernel source files.
URGENT RESPONSE NEEDED
Hello my dear. Did you receive my email message to you? Please, get back to me ASAP as the matter is becoming late. Expecting your urgent response. Sean.
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Thu, Oct 11, 2018 at 11:27:35AM -0600, Jonathan Corbet wrote: > On Sat, 6 Oct 2018 10:51:54 +1000 Dave Chinner > wrote: > > > Can you let us know whether the CC-by-SA 4.0 license is > > acceptible or not? That's really the only thing that we need > > clarified at this point - if it's OK I'll to pull this into the > > XFS tree for the 4.20 merge window. If not, we'll go back to the > > drawing board > > OK, I've had a long conversation with the LF lawyer, and she said > clearly that we really should not be introducing CC-SA material > into the kernel source tree. It's a pain; I really do like CC-SA > better for documentation, but it's not GPL-compatible, and that > creates a problem for the processed docs. Ok, thanks for following upon this, Jon. We'll just keep it all in the existing external repo and work out what to do from there. Cheers, Dave. -- Dave Chinner da...@fromorbit.com
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Sat, 6 Oct 2018 10:51:54 +1000 Dave Chinner wrote: > Can you let us know whether the CC-by-SA 4.0 license is acceptible > or not? That's really the only thing that we need clarified at this > point - if it's OK I'll to pull this into the XFS tree for the 4.20 > merge window. If not, we'll go back to the drawing board OK, I've had a long conversation with the LF lawyer, and she said clearly that we really should not be introducing CC-SA material into the kernel source tree. It's a pain; I really do like CC-SA better for documentation, but it's not GPL-compatible, and that creates a problem for the processed docs. Sorry, jon
Re: [PATCH 2/2] docs/vm: split memory hotplug notifier description to Documentation/core-api
On Sun, Oct 07, 2018 at 08:46:40AM -0600, Jonathan Corbet wrote: > On Fri, 5 Oct 2018 01:11:01 +0300 > Mike Rapoport wrote: > > > The memory hotplug notifier description is about kernel internals rather > > than admin/user visible API. Place it appropriately. > > > > Signed-off-by: Mike Rapoport > > One little nit... > > > Documentation/admin-guide/mm/memory-hotplug.rst| 83 > > - > > Documentation/core-api/index.rst | 2 + > > Documentation/core-api/memory-hotplug-notifier.rst | 84 > > ++ > > 3 files changed, 86 insertions(+), 83 deletions(-) > > create mode 100644 Documentation/core-api/memory-hotplug-notifier.rst > > > > diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst > > b/Documentation/admin-guide/mm/memory-hotplug.rst > > index a33090c..0b9c83e 100644 > > --- a/Documentation/admin-guide/mm/memory-hotplug.rst > > +++ b/Documentation/admin-guide/mm/memory-hotplug.rst > > @@ -31,7 +31,6 @@ be changed often. > > 6.1 Memory offline and ZONE_MOVABLE > > 6.2. How to offline memory > >7. Physical memory remove > > - 8. Memory hotplug event notifier > >9. Future Work List > > That leaves a gap in the numbering here. > > In general, the best solution to this sort of issue is to take the TOC out > entirely and let Sphinx worry about generating it. People tend not to > think about updating the TOC when they make changes elsewhere, so it often > goes out of sync with the rest of the document anyway. > > I'll apply these, but please feel free to send a patch to fix this up. Sure, below > Thanks, > > jon > >From 45ef951bd04e9d440a0158f991a188fc8ca6e7d4 Mon Sep 17 00:00:00 2001 From: Mike Rapoport Date: Sun, 7 Oct 2018 19:29:00 +0300 Subject: [PATCH] docs/admin-guide: memory-hotplug: remove table of contents Remove "manual" table of contents and leave only the ReST tag so that Sphinx will take care of TOC generation. Signed-off-by: Mike Rapoport --- Documentation/admin-guide/mm/memory-hotplug.rst | 21 - 1 file changed, 21 deletions(-) diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst b/Documentation/admin-guide/mm/memory-hotplug.rst index 0b9c83e..25157ae 100644 --- a/Documentation/admin-guide/mm/memory-hotplug.rst +++ b/Documentation/admin-guide/mm/memory-hotplug.rst @@ -13,27 +13,6 @@ be changed often. .. contents:: :local: -.. CONTENTS - - 1. Introduction -1.1 Purpose of memory hotplug -1.2. Phases of memory hotplug -1.3. Unit of Memory online/offline operation - 2. Kernel Configuration - 3. sysfs files for memory hotplug - 4. Physical memory hot-add phase -4.1 Hardware(Firmware) Support -4.2 Notify memory hot-add event by hand - 5. Logical Memory hot-add phase -5.1. State of memory -5.2. How to online memory - 6. Logical memory remove -6.1 Memory offline and ZONE_MOVABLE -6.2. How to offline memory - 7. Physical memory remove - 9. Future Work List - - .. note:: (1) x86_64's has special implementation for memory hotplug. -- 2.7.4
Re: [PATCH] docs: improve readability for people with poorer eyesight
On Thu, 4 Oct 2018 18:06:03 -0700 "Darrick J. Wong" wrote: > o my eyesight still hasn't fully recovered, so in the meantime it's > been difficult to read the online documentation. Here's some stylesheet > overrides I've been using to make it easier for me to read them: > https://djwong.org/docs/kdoc/index.html > > --- > From: Darrick J. Wong > > My eyesight is not in good shape, which means that I have difficulty > reading the online Linux documentation. Specifically, body text is > oddly small compared to list items and the contrast of various text > elements is too low for me to be able to see easily. > > Therefore, alter the HTML theme overrides to make the text larger and > increase the contrast for better visibility, and trust the typeface > choices of the reader's browser. > > For the PDF output, increase the text size, use a sans-serif typeface > for sans-serif text, and use a serif typeface for "roman" serif text. > > Signed-off-by: Darrick J. Wong I've been wanting to find some time to work on our stylesheets for years; what we have now is far from ideal (or attractive) IMO. This is a good step in the right direction - applied, thanks. jon
Re: [PATCH 2/2] docs/vm: split memory hotplug notifier description to Documentation/core-api
On Fri, 5 Oct 2018 01:11:01 +0300 Mike Rapoport wrote: > The memory hotplug notifier description is about kernel internals rather > than admin/user visible API. Place it appropriately. > > Signed-off-by: Mike Rapoport One little nit... > Documentation/admin-guide/mm/memory-hotplug.rst| 83 - > Documentation/core-api/index.rst | 2 + > Documentation/core-api/memory-hotplug-notifier.rst | 84 > ++ > 3 files changed, 86 insertions(+), 83 deletions(-) > create mode 100644 Documentation/core-api/memory-hotplug-notifier.rst > > diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst > b/Documentation/admin-guide/mm/memory-hotplug.rst > index a33090c..0b9c83e 100644 > --- a/Documentation/admin-guide/mm/memory-hotplug.rst > +++ b/Documentation/admin-guide/mm/memory-hotplug.rst > @@ -31,7 +31,6 @@ be changed often. > 6.1 Memory offline and ZONE_MOVABLE > 6.2. How to offline memory >7. Physical memory remove > - 8. Memory hotplug event notifier >9. Future Work List That leaves a gap in the numbering here. In general, the best solution to this sort of issue is to take the TOC out entirely and let Sphinx worry about generating it. People tend not to think about updating the TOC when they make changes elsewhere, so it often goes out of sync with the rest of the document anyway. I'll apply these, but please feel free to send a patch to fix this up. Thanks, jon
Re: [PATCH 2/2] docs: promote the ext4 data structures book to top level
On Fri, 5 Oct 2018 19:49:52 -0400 "Theodore Y. Ts'o" wrote: > On Thu, Oct 04, 2018 at 05:59:44PM -0700, Darrick J. Wong wrote: > > From: Darrick J. Wong > > > > Move the ext4 data structures book to Documentation/filesystems/ext4/ > > since the administrative information moved elsewhere. > > > > Signed-off-by: Darrick J. Wong > > Thanks, applied and pushed out to the ext4.git tree. > > Randy, Jon: the original patch didn't make it past vger.kernel.org > because it was too large (it was moving a lot of files around). It > looked fine to me, but if you want to take a look it should be on the > dev branch of the ext4.git tree. FWIW, from a quick look it seems fine to me. Acked-by: Jonathan Corbet (though I expect it's too late to add that). jon
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Sat, 6 Oct 2018 06:29:51 -0700 Matthew Wilcox wrote: > I had an informal chat with Bradley Kuhn at LinuxCon Japan about using > CC-BY-SA-4.0 and he indicated that I was probably better off using the > GPL-2(+) for documentation. I've changed the XArray documentation from > CC-BY-SA-4.0 to GPL-2+. Would you consider doing the same for Documentation/core-api/idr.rst? That's the only CC-SA SPDX tag in the kernel now, and it's a clear example of just what I'm worried about: it pulls in a nice DOC section from the code when you run Sphinx on it, so we're not just talking function prototypes here. I was likely to come knocking on your door after my upcoming conversation anyway...:) Thanks, jon
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Sat, Oct 06, 2018 at 10:51:54AM +1000, Dave Chinner wrote: > On Wed, Oct 03, 2018 at 09:18:11PM -0700, Darrick J. Wong wrote: > > Hi all, > > > > This series converts the existing in-kernel xfs documentation to rst > > format, links it in with the rest of the kernel's rst documetation, and > > then begins pulling in the contents of the Data Structures & Algorithms > > book from the xfs-documentation git tree. No changes are made to the > > text during the import process except to fix things that the conversion > > process (asciidoctor + pandoc) didn't do correctly. The goal of this > > series is to tie together the XFS code with the on-disk format > > documentation for the features supported by the code. > > > > I've built the docs and put them here, in case you hate reading rst: > > https://djwong.org/docs/kdoc/admin-guide/xfs.html > > https://djwong.org/docs/kdoc/filesystems/xfs-data-structures/index.html > > > > I've posted a branch here because the png import patch is huge: > > https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=docs-4.20-merge > > > > The patchset should apply cleanly against 4.19-rc6. Comments and > > questions are, as always, welcome. > > Jon, > > Can you let us know whether the CC-by-SA 4.0 license is acceptible > or not? That's really the only thing that we need clarified at this > point - if it's OK I'll to pull this into the XFS tree for the 4.20 > merge window. If not, we'll go back to the drawing board This is (obviously) not legal advice. I had an informal chat with Bradley Kuhn at LinuxCon Japan about using CC-BY-SA-4.0 and he indicated that I was probably better off using the GPL-2(+) for documentation. I've changed the XArray documentation from CC-BY-SA-4.0 to GPL-2+. YMMV, defer to the LF lawyer, but I thought it worth providing a data point.
Re: [PATCH v5 00/15] Compiler Attributes
On Sat, Oct 6, 2018 at 9:13 AM Sedat Dilek wrote: > > Hi Miguel, > > In my testoings I am booting in QEMU and on bare metal. > Both tests were successful. Wow! I only wanted to confirm it booted and/or run for a while -- but having more information is always good. Thanks a *lot*! I will add a reference to your email in the commits. Cheers, Miguel
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Fri, Oct 05, 2018 at 07:01:20PM -0600, Jonathan Corbet wrote: > On Sat, 6 Oct 2018 10:51:54 +1000 > Dave Chinner wrote: > > > Can you let us know whether the CC-by-SA 4.0 license is acceptible > > or not? That's really the only thing that we need clarified at this > > point - if it's OK I'll to pull this into the XFS tree for the 4.20 > > merge window. If not, we'll go back to the drawing board > > I remain pretty concerned about it, to tell the truth. Rather than > continue to guess, though, I've called for help, and will be talking with > the LF lawyer about this next Thursday. Before then, I can't say anything > except "I don't think this works..." > > Will let you know what I hear. Thanks for the update, Jon. I'll put this on the backburner until I hear back from you. Cheers, Dave. -- Dave Chinner da...@fromorbit.com
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Sat, 6 Oct 2018 10:51:54 +1000 Dave Chinner wrote: > Can you let us know whether the CC-by-SA 4.0 license is acceptible > or not? That's really the only thing that we need clarified at this > point - if it's OK I'll to pull this into the XFS tree for the 4.20 > merge window. If not, we'll go back to the drawing board I remain pretty concerned about it, to tell the truth. Rather than continue to guess, though, I've called for help, and will be talking with the LF lawyer about this next Thursday. Before then, I can't say anything except "I don't think this works..." Will let you know what I hear. jon
Re: [PATCH v2 00/22] xfs-4.20: major documentation surgery
On Wed, Oct 03, 2018 at 09:18:11PM -0700, Darrick J. Wong wrote: > Hi all, > > This series converts the existing in-kernel xfs documentation to rst > format, links it in with the rest of the kernel's rst documetation, and > then begins pulling in the contents of the Data Structures & Algorithms > book from the xfs-documentation git tree. No changes are made to the > text during the import process except to fix things that the conversion > process (asciidoctor + pandoc) didn't do correctly. The goal of this > series is to tie together the XFS code with the on-disk format > documentation for the features supported by the code. > > I've built the docs and put them here, in case you hate reading rst: > https://djwong.org/docs/kdoc/admin-guide/xfs.html > https://djwong.org/docs/kdoc/filesystems/xfs-data-structures/index.html > > I've posted a branch here because the png import patch is huge: > https://git.kernel.org/pub/scm/linux/kernel/git/djwong/xfs-linux.git/log/?h=docs-4.20-merge > > The patchset should apply cleanly against 4.19-rc6. Comments and > questions are, as always, welcome. Jon, Can you let us know whether the CC-by-SA 4.0 license is acceptible or not? That's really the only thing that we need clarified at this point - if it's OK I'll to pull this into the XFS tree for the 4.20 merge window. If not, we'll go back to the drawing board Cheers, Dave. -- Dave Chinner da...@fromorbit.com
Re: [PATCH] docs: improve readability for people with poorer eyesight
On Thu, Oct 04, 2018 at 06:06:03PM -0700, Darrick J. Wong wrote: > Hi, > > So my eyesight still hasn't fully recovered, so in the meantime it's > been difficult to read the online documentation. Here's some stylesheet > overrides I've been using to make it easier for me to read them: > https://djwong.org/docs/kdoc/index.html > > --- > From: Darrick J. Wong > > My eyesight is not in good shape, which means that I have difficulty > reading the online Linux documentation. Specifically, body text is > oddly small compared to list items and the contrast of various text > elements is too low for me to be able to see easily. > > Therefore, alter the HTML theme overrides to make the text larger and > increase the contrast for better visibility, and trust the typeface > choices of the reader's browser. > > For the PDF output, increase the text size, use a sans-serif typeface > for sans-serif text, and use a serif typeface for "roman" serif text. > > Signed-off-by: Darrick J. Wong This fixes problems I noticed when trying to review the built html documentation - the inconsistent font sizes on my high-dpi monitor made it almost impossible to read even though I have no eyesight problems Acked-by: Dave Chinner -Dave. -- Dave Chinner da...@fromorbit.com
Re: [PATCH 2/2] docs: promote the ext4 data structures book to top level
On Thu, Oct 04, 2018 at 05:59:44PM -0700, Darrick J. Wong wrote: > From: Darrick J. Wong > > Move the ext4 data structures book to Documentation/filesystems/ext4/ > since the administrative information moved elsewhere. > > Signed-off-by: Darrick J. Wong Thanks, applied and pushed out to the ext4.git tree. Randy, Jon: the original patch didn't make it past vger.kernel.org because it was too large (it was moving a lot of files around). It looked fine to me, but if you want to take a look it should be on the dev branch of the ext4.git tree. - Ted
Re: [PATCH 0/2] ext4: even more documentation fixes
On 10/5/18 8:22 AM, Theodore Y. Ts'o wrote: > On Thu, Oct 04, 2018 at 07:48:31PM -0700, Randy Dunlap wrote: >> Hi Darrick, >> >> I don't see patch 2/2 anywhere (my inbox, email archives)... > > Probably because it's moving a lot of files around, so the diffs were 276k. Oh, yeah. Thanks. -- ~Randy
Re: [PATCH 0/2] ext4: even more documentation fixes
On Thu, Oct 04, 2018 at 07:48:31PM -0700, Randy Dunlap wrote: > Hi Darrick, > > I don't see patch 2/2 anywhere (my inbox, email archives)... Probably because it's moving a lot of files around, so the diffs were 276k. > > -- > ~Randy
Re: [PATCH 0/2] ext4: even more documentation fixes
On 10/4/18 5:59 PM, Darrick J. Wong wrote: > Hi all, > > This series fixes some problems that were brought up during review for > the XFS documentation which I hadn't known about when pushing the ext4 > documentation during the 4.19 cycle. > > The first patch moves the ext4 mount option and sysfs knob information > into the Linux administration guide. > > The second patch moves the ext4 Data Structures & Algorithms book up a > level since it's a self-contained documentation book. > > I've built the docs and put them here, in case you hate reading rst: > https://djwong.org/docs/kdoc/filesystems/ext4/index.html > > The patchset should apply cleanly against 4.19-rc6. Comments and > questions are, as always, welcome. Hi Darrick, I don't see patch 2/2 anywhere (my inbox, email archives)... -- ~Randy
[PATCH] docs: improve readability for people with poorer eyesight
Hi,
So my eyesight still hasn't fully recovered, so in the meantime it's
been difficult to read the online documentation. Here's some stylesheet
overrides I've been using to make it easier for me to read them:
https://djwong.org/docs/kdoc/index.html
---
From: Darrick J. Wong
My eyesight is not in good shape, which means that I have difficulty
reading the online Linux documentation. Specifically, body text is
oddly small compared to list items and the contrast of various text
elements is too low for me to be able to see easily.
Therefore, alter the HTML theme overrides to make the text larger and
increase the contrast for better visibility, and trust the typeface
choices of the reader's browser.
For the PDF output, increase the text size, use a sans-serif typeface
for sans-serif text, and use a serif typeface for "roman" serif text.
Signed-off-by: Darrick J. Wong
---
Documentation/conf.py |6 ++--
Documentation/sphinx-static/theme_overrides.css | 38 +++
2 files changed, 41 insertions(+), 3 deletions(-)
diff --git a/Documentation/conf.py b/Documentation/conf.py
index e45d4d2b5adb..03dd6dc135ea 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -259,7 +259,7 @@ latex_elements = {
'papersize': 'a4paper',
# The font size ('10pt', '11pt' or '12pt').
-'pointsize': '8pt',
+'pointsize': '11pt',
# Latex figure (float) alignment
#'figure_align': 'htbp',
@@ -272,8 +272,8 @@ latex_elements = {
'preamble': '''
% Use some font with UTF-8 support with XeLaTeX
\\usepackage{fontspec}
-\\setsansfont{DejaVu Serif}
-\\setromanfont{DejaVu Sans}
+\\setsansfont{DejaVu Sans}
+\\setromanfont{DejaVu Serif}
\\setmonofont{DejaVu Sans Mono}
'''
diff --git a/Documentation/sphinx-static/theme_overrides.css
b/Documentation/sphinx-static/theme_overrides.css
index 522b6d4c49d4..e21e36cd6761 100644
--- a/Documentation/sphinx-static/theme_overrides.css
+++ b/Documentation/sphinx-static/theme_overrides.css
@@ -4,6 +4,44 @@
*
*/
+/* Improve contrast and increase size for easier reading. */
+
+body {
+ font-family: serif;
+ color: black;
+ font-size: 100%;
+}
+
+h1, h2, .rst-content .toctree-wrapper p.caption, h3, h4, h5, h6, legend {
+ font-family: sans-serif;
+}
+
+.wy-menu-vertical li.current a {
+ color: #505050;
+}
+
+.wy-menu-vertical li.on a, .wy-menu-vertical li.current > a {
+ color: #303030;
+}
+
+div[class^="highlight"] pre {
+ font-family: monospace;
+ color: black;
+ font-size: 100%;
+}
+
+.wy-menu-vertical {
+ font-family: sans-serif;
+}
+
+.c {
+ font-style: normal;
+}
+
+p {
+ font-size: 100%;
+}
+
/* Interim: Code-blocks with line nos - lines and line numbers don't line up.
* see: https://github.com/rtfd/sphinx_rtd_theme/issues/419
*/
[PATCH 1/2] docs: move ext4 administrative docs to admin-guide/
From: Darrick J. Wong Move the ext4 mount option and other administrative stuff to the Linux administrator's guide. Signed-off-by: Darrick J. Wong --- Documentation/admin-guide/ext4.rst | 574 ++ Documentation/admin-guide/index.rst |1 Documentation/conf.py|2 Documentation/filesystems/ext4/ext4.rst | 574 -- Documentation/filesystems/ext4/index.rst |1 5 files changed, 577 insertions(+), 575 deletions(-) create mode 100644 Documentation/admin-guide/ext4.rst delete mode 100644 Documentation/filesystems/ext4/ext4.rst diff --git a/Documentation/admin-guide/ext4.rst b/Documentation/admin-guide/ext4.rst new file mode 100644 index ..e506d3dae510 --- /dev/null +++ b/Documentation/admin-guide/ext4.rst @@ -0,0 +1,574 @@ +.. SPDX-License-Identifier: GPL-2.0 + + +ext4 General Information + + +Ext4 is an advanced level of the ext3 filesystem which incorporates +scalability and reliability enhancements for supporting large filesystems +(64 bit) in keeping with increasing disk capacities and state-of-the-art +feature requirements. + +Mailing list: linux-e...@vger.kernel.org +Web site: http://ext4.wiki.kernel.org + + +Quick usage instructions + + +Note: More extensive information for getting started with ext4 can be +found at the ext4 wiki site at the URL: +http://ext4.wiki.kernel.org/index.php/Ext4_Howto + + - The latest version of e2fsprogs can be found at: + +https://www.kernel.org/pub/linux/kernel/people/tytso/e2fsprogs/ + + or + +http://sourceforge.net/project/showfiles.php?group_id=2406 + + or grab the latest git repository from: + + https://git.kernel.org/pub/scm/fs/ext2/e2fsprogs.git + + - Create a new filesystem using the ext4 filesystem type: + +# mke2fs -t ext4 /dev/hda1 + +Or to configure an existing ext3 filesystem to support extents: + + # tune2fs -O extents /dev/hda1 + +If the filesystem was created with 128 byte inodes, it can be +converted to use 256 byte for greater efficiency via: + +# tune2fs -I 256 /dev/hda1 + + - Mounting: + + # mount -t ext4 /dev/hda1 /wherever + + - When comparing performance with other filesystems, it's always +important to try multiple workloads; very often a subtle change in a +workload parameter can completely change the ranking of which +filesystems do well compared to others. When comparing versus ext3, +note that ext4 enables write barriers by default, while ext3 does +not enable write barriers by default. So it is useful to use +explicitly specify whether barriers are enabled or not when via the +'-o barriers=[0|1]' mount option for both ext3 and ext4 filesystems +for a fair comparison. When tuning ext3 for best benchmark numbers, +it is often worthwhile to try changing the data journaling mode; '-o +data=writeback' can be faster for some workloads. (Note however that +running mounted with data=writeback can potentially leave stale data +exposed in recently written files in case of an unclean shutdown, +which could be a security exposure in some situations.) Configuring +the filesystem with a large journal can also be helpful for +metadata-intensive workloads. + +Features + + +Currently Available +--- + +* ability to use filesystems > 16TB (e2fsprogs support not available yet) +* extent format reduces metadata overhead (RAM, IO for access, transactions) +* extent format more robust in face of on-disk corruption due to magics, +* internal redundancy in tree +* improved file allocation (multi-block alloc) +* lift 32000 subdirectory limit imposed by i_links_count[1] +* nsec timestamps for mtime, atime, ctime, create time +* inode version field on disk (NFSv4, Lustre) +* reduced e2fsck time via uninit_bg feature +* journal checksumming for robustness, performance +* persistent file preallocation (e.g for streaming media, databases) +* ability to pack bitmaps and inode tables into larger virtual groups via the + flex_bg feature +* large file support +* inode allocation using large virtual block groups via flex_bg +* delayed allocation +* large block (up to pagesize) support +* efficient new ordered mode in JBD2 and ext4 (avoid using buffer head to force + the ordering) + +[1] Filesystems with a block size of 1k may see a limit imposed by the +directory hash tree having a maximum depth of two. + +Options +=== + +When mounting an ext4 filesystem, the following option are accepted: +(*) == default + + ro +Mount filesystem read only. Note that ext4 will replay the journal (and +thus write to the partition) even when mounted "read only". The mount +options "ro,noload" can be used to prevent writes to the filesystem. + + journal_checksum +Enable checksumming of the journal transactions.
[PATCH 0/2] ext4: even more documentation fixes
Hi all, This series fixes some problems that were brought up during review for the XFS documentation which I hadn't known about when pushing the ext4 documentation during the 4.19 cycle. The first patch moves the ext4 mount option and sysfs knob information into the Linux administration guide. The second patch moves the ext4 Data Structures & Algorithms book up a level since it's a self-contained documentation book. I've built the docs and put them here, in case you hate reading rst: https://djwong.org/docs/kdoc/filesystems/ext4/index.html The patchset should apply cleanly against 4.19-rc6. Comments and questions are, as always, welcome. --D
[PATCH 2/2] docs/vm: split memory hotplug notifier description to Documentation/core-api
The memory hotplug notifier description is about kernel internals rather
than admin/user visible API. Place it appropriately.
Signed-off-by: Mike Rapoport
---
Documentation/admin-guide/mm/memory-hotplug.rst| 83 -
Documentation/core-api/index.rst | 2 +
Documentation/core-api/memory-hotplug-notifier.rst | 84 ++
3 files changed, 86 insertions(+), 83 deletions(-)
create mode 100644 Documentation/core-api/memory-hotplug-notifier.rst
diff --git a/Documentation/admin-guide/mm/memory-hotplug.rst
b/Documentation/admin-guide/mm/memory-hotplug.rst
index a33090c..0b9c83e 100644
--- a/Documentation/admin-guide/mm/memory-hotplug.rst
+++ b/Documentation/admin-guide/mm/memory-hotplug.rst
@@ -31,7 +31,6 @@ be changed often.
6.1 Memory offline and ZONE_MOVABLE
6.2. How to offline memory
7. Physical memory remove
- 8. Memory hotplug event notifier
9. Future Work List
@@ -414,88 +413,6 @@ Need more implementation yet
- Notification completion of remove works by OS to firmware.
- Guard from remove if not yet.
-Memory hotplug event notifier
-=
-
-Hotplugging events are sent to a notification queue.
-
-There are six types of notification defined in ``include/linux/memory.h``:
-
-MEM_GOING_ONLINE
- Generated before new memory becomes available in order to be able to
- prepare subsystems to handle memory. The page allocator is still unable
- to allocate from the new memory.
-
-MEM_CANCEL_ONLINE
- Generated if MEMORY_GOING_ONLINE fails.
-
-MEM_ONLINE
- Generated when memory has successfully brought online. The callback may
- allocate pages from the new memory.
-
-MEM_GOING_OFFLINE
- Generated to begin the process of offlining memory. Allocations are no
- longer possible from the memory but some of the memory to be offlined
- is still in use. The callback can be used to free memory known to a
- subsystem from the indicated memory block.
-
-MEM_CANCEL_OFFLINE
- Generated if MEMORY_GOING_OFFLINE fails. Memory is available again from
- the memory block that we attempted to offline.
-
-MEM_OFFLINE
- Generated after offlining memory is complete.
-
-A callback routine can be registered by calling::
-
- hotplug_memory_notifier(callback_func, priority)
-
-Callback functions with higher values of priority are called before callback
-functions with lower values.
-
-A callback function must have the following prototype::
-
- int callback_func(
-struct notifier_block *self, unsigned long action, void *arg);
-
-The first argument of the callback function (self) is a pointer to the block
-of the notifier chain that points to the callback function itself.
-The second argument (action) is one of the event types described above.
-The third argument (arg) passes a pointer of struct memory_notify::
-
- struct memory_notify {
- unsigned long start_pfn;
- unsigned long nr_pages;
- int status_change_nid_normal;
- int status_change_nid_high;
- int status_change_nid;
- }
-
-- start_pfn is start_pfn of online/offline memory.
-- nr_pages is # of pages of online/offline memory.
-- status_change_nid_normal is set node id when N_NORMAL_MEMORY of nodemask
- is (will be) set/clear, if this is -1, then nodemask status is not changed.
-- status_change_nid_high is set node id when N_HIGH_MEMORY of nodemask
- is (will be) set/clear, if this is -1, then nodemask status is not changed.
-- status_change_nid is set node id when N_MEMORY of nodemask is (will be)
- set/clear. It means a new(memoryless) node gets new memory by online and a
- node loses all memory. If this is -1, then nodemask status is not changed.
-
- If status_changed_nid* >= 0, callback should create/discard structures for
the
- node if necessary.
-
-The callback routine shall return one of the values
-NOTIFY_DONE, NOTIFY_OK, NOTIFY_BAD, NOTIFY_STOP
-defined in ``include/linux/notifier.h``
-
-NOTIFY_DONE and NOTIFY_OK have no effect on the further processing.
-
-NOTIFY_BAD is used as response to the MEM_GOING_ONLINE, MEM_GOING_OFFLINE,
-MEM_ONLINE, or MEM_OFFLINE action to cancel hotplugging. It stops
-further processing of the notification queue.
-
-NOTIFY_STOP stops further processing of the notification queue.
-
Future Work
===
diff --git a/Documentation/core-api/index.rst b/Documentation/core-api/index.rst
index 165d7688..4f8a426 100644
--- a/Documentation/core-api/index.rst
+++ b/Documentation/core-api/index.rst
@@ -32,6 +32,8 @@ Core utilities
gfp_mask-from-fs-io
timekeeping
boot-time-mm
+ memory-hotplug-notifier
+
Interfaces for kernel debugging
===
diff --git a/Documentation/core-api/memory-hotplug-notifier.rst
b/Documentation/core-api/memory-hotplug-notifier.rst
new file mode 100644
index 000..35347cc
--- /dev/null
+++ b/Documentation/core-api/memory-hotplug-notifier.rst
@@ -0,0 +1,84 @@
+..
