[PATCH] doc/ko_KR/memory-barriers: Update control-dependencies section

[SeongJae Park]

This commit applies upstream change, commit c8241f8553e8 ("doc: Update
control-dependencies section of memory-barriers.txt"), to Korean
translation.

Signed-off-by: SeongJae Park 
---
 .../translations/ko_KR/memory-barriers.txt | 68 --
 1 file changed, 37 insertions(+), 31 deletions(-)

diff --git a/Documentation/translations/ko_KR/memory-barriers.txt 
b/Documentation/translations/ko_KR/memory-barriers.txt
index a3228a676cc1..ce0b48d69eaa 100644
--- a/Documentation/translations/ko_KR/memory-barriers.txt
+++ b/Documentation/translations/ko_KR/memory-barriers.txt
@@ -662,6 +662,10 @@ include/linux/rcupdate.h 의 rcu_assign_pointer() 와 
rcu_dereference() 를
 컨트롤 의존성
 -
 
+현재의 컴파일러들은 컨트롤 의존성을 이해하고 있지 않기 때문에 컨트롤 의존성은
+약간 다루기 어려울 수 있습니다.  이 섹션의 목적은 여러분이 컴파일러의 무시로
+인해 여러분의 코드가 망가지는 걸 막을 수 있도록 돕는겁니다.
+
 로드-로드 컨트롤 의존성은 데이터 의존성 배리어만으로는 정확히 동작할 수가
 없어서 읽기 메모리 배리어를 필요로 합니다.  아래의 코드를 봅시다:
 
@@ -689,20 +693,21 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
 
q = READ_ONCE(a);
if (q) {
-   WRITE_ONCE(b, p);
+   WRITE_ONCE(b, 1);
}
 
 컨트롤 의존성은 보통 다른 타입의 배리어들과 짝을 맞춰 사용됩니다.  그렇다곤
-하나, READ_ONCE() 는 반드시 사용해야 함을 부디 명심하세요!  READ_ONCE() 가
-없다면, 컴파일러가 'a' 로부터의 로드를 'a' 로부터의 또다른 로드와, 'b' 로의
-스토어를 'b' 로의 또다른 스토어와 조합해 버려 매우 비직관적인 결과를 초래할 수
-있습니다.
+하나, READ_ONCE() 도 WRITE_ONCE() 도 선택사항이 아니라 필수사항임을 부디
+명심하세요!  READ_ONCE() 가 없다면, 컴파일러는 'a' 로부터의 로드를 'a' 로부터의
+또다른 로드와 조합할 수 있습니다.  WRITE_ONCE() 가 없다면, 컴파일러는 'b' 로의
+스토어를 'b' 로의 또라느 스토어들과 조합할 수 있습니다.  두 경우 모두 순서에
+있어 상당히 비직관적인 결과를 초래할 수 있습니다.
 
 이걸로 끝이 아닌게, 컴파일러가 변수 'a' 의 값이 항상 0이 아니라고 증명할 수
 있다면, 앞의 예에서 "if" 문을 없애서 다음과 같이 최적화 할 수도 있습니다:
 
q = a;
-   b = p;  /* BUG: Compiler and CPU can both reorder!!! */
+   b = 1;  /* BUG: Compiler and CPU can both reorder!!! */
 
 그러니 READ_ONCE() 를 반드시 사용하세요.
 
@@ -712,11 +717,11 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
q = READ_ONCE(a);
if (q) {
barrier();
-   WRITE_ONCE(b, p);
+   WRITE_ONCE(b, 1);
do_something();
} else {
barrier();
-   WRITE_ONCE(b, p);
+   WRITE_ONCE(b, 1);
do_something_else();
}
 
@@ -725,12 +730,12 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
 
q = READ_ONCE(a);
barrier();
-   WRITE_ONCE(b, p);  /* BUG: No ordering vs. load from a!!! */
+   WRITE_ONCE(b, 1);  /* BUG: No ordering vs. load from a!!! */
if (q) {
-   /* WRITE_ONCE(b, p); -- moved up, BUG!!! */
+   /* WRITE_ONCE(b, 1); -- moved up, BUG!!! */
do_something();
} else {
-   /* WRITE_ONCE(b, p); -- moved up, BUG!!! */
+   /* WRITE_ONCE(b, 1); -- moved up, BUG!!! */
do_something_else();
}
 
@@ -742,10 +747,10 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
 
q = READ_ONCE(a);
if (q) {
-   smp_store_release(, p);
+   smp_store_release(, 1);
do_something();
} else {
-   smp_store_release(, p);
+   smp_store_release(, 1);
do_something_else();
}
 
@@ -754,10 +759,10 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
 
q = READ_ONCE(a);
if (q) {
-   WRITE_ONCE(b, p);
+   WRITE_ONCE(b, 1);
do_something();
} else {
-   WRITE_ONCE(b, r);
+   WRITE_ONCE(b, 2);
do_something_else();
}
 
@@ -770,10 +775,10 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
 
q = READ_ONCE(a);
if (q % MAX) {
-   WRITE_ONCE(b, p);
+   WRITE_ONCE(b, 1);
do_something();
} else {
-   WRITE_ONCE(b, r);
+   WRITE_ONCE(b, 2);
do_something_else();
}
 
@@ -781,7 +786,7 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
 위의 코드를 아래와 같이 바꿔버릴 수 있습니다:
 
q = READ_ONCE(a);
-   WRITE_ONCE(b, p);
+   WRITE_ONCE(b, 1);
do_something_else();
 
 이렇게 되면, CPU 는 변수 'a' 로부터의 로드와 변수 'b' 로의 스토어 사이의 순서를
@@ -793,10 +798,10 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
q = READ_ONCE(a);
BUILD_BUG_ON(MAX <= 1); /* Order load from a with store to b. */
if (q % MAX) {
-   WRITE_ONCE(b, p);
+   WRITE_ONCE(b, 1);
do_something();
} else {
-   WRITE_ONCE(b, r);
+   WRITE_ONCE(b, 2);
do_something_else();
}
 
@@ -828,35 +833,33 @@ CPU 는 b 로부터의 로드 오퍼레이션이 a 로부터의 로드 오퍼레
 
q = READ_ONCE(a);
if (q) {
-   WRITE_ONCE(b, p);
+   WRITE_ONCE(b, 1);
} else {
-   WRITE_ONCE(b, r);
+   WRITE_ONCE(b, 2);
}
-   WRITE_ONCE(c, 1);  /* BUG: No ordering against the read from "a". */
+   WRITE_ONCE(c, 1);  /* BUG: No ordering against the read from 'a'. */
 
-컴파일러는 volatile 타입에 

[RESEND PATCH v5 04/11 (Missed 04/11 in PATCH v5 series)] Documentation: perf: hisi: Documentation for HiP05/06/07 PMU event counting.

[Anurup M]

Documentation for perf usage and Hisilicon SoC PMU uncore events.
The Hisilicon SOC has event counters for hardware modules like
L3 cache, Miscellaneous node etc. These events are all uncore.

Signed-off-by: Anurup M 
Signed-off-by: Shaokun Zhang 
---
 Documentation/perf/hisi-pmu.txt | 76 +
 1 file changed, 76 insertions(+)
 create mode 100644 Documentation/perf/hisi-pmu.txt

diff --git a/Documentation/perf/hisi-pmu.txt b/Documentation/perf/hisi-pmu.txt
new file mode 100644
index 000..e3ac562
--- /dev/null
+++ b/Documentation/perf/hisi-pmu.txt
@@ -0,0 +1,76 @@
+Hisilicon SoC PMU (Performance Monitoring Unit)
+
+The Hisilicon SoC HiP05/06/07 chips consist of various independent system
+device PMU's such as L3 cache(L3C) and Miscellaneous Nodes(MN).
+These PMU devices are independent and have hardware logic to gather
+statistics and performance information.
+
+HiP0x chips are encapsulated by multiple CPU and IO die's. The CPU die is
+called as Super CPU cluster (SCCL) which includes 16 cpu-cores. Every SCCL
+is further grouped as CPU clusters (CCL) which includes 4 cpu-cores each.
+Each SCCL has 1 L3 cache and 1 MN units.
+
+The L3 cache is shared by all CPU cores in a CPU die. The L3C has four banks
+(or instances). Each bank or instance of L3C has Eight 32-bit counter
+registers and also event control registers. The HiP05/06 chip L3 cache has
+22 statistics events. The HiP07 chip has 66 statistics events. These events
+are very useful for debugging.
+
+The MN module is also shared by all CPU cores in a CPU die. It receives
+barriers and DVM(Distributed Virtual Memory) messages from cpu or smmu, and
+perform the required actions and return response messages. These events are
+very useful for debugging. The MN has total 9 statistics events and support
+four 32-bit counter registers in HiP05/06/07 chips.
+
+There is no memory mapping for L3 cache and MN registers. It can be accessed
+by using the Hisilicon djtag interface. The Djtag in a SCCL is an independent
+module which connects with some modules in the SoC by Debug Bus.
+
+Hisilicon SoC (HiP05/06/07) PMU driver
+--
+The HiP0x PMU driver shall register perf PMU drivers like L3 cache, MN, etc.
+The available events and configuration options shall be described in the sysfs.
+The "perf list" shall list the available events from sysfs.
+
+The L3 cache in a SCCL is divided as 4 banks. Each L3 cache bank have separate
+PMU registers for event counting and control. The L3 cache banks also do not
+have any CPU affinity. So each L3 cache banks are registered with perf as a
+separate PMU.
+The PMU name will appear in event listing as hisi_l3c_.
+where "bank-id" is the bank index (0 to 3) and "scl-id" is the SCCL identifier
+e.g. hisi_l3c0_2/read_hit is READ_HIT event of L3 cache bank #0 SCCL ID #2.
+
+The MN in a SCCL is registered as a separate PMU with perf.
+The PMU name will appear in event listing as hisi_mn_.
+e.g. hisi_mn_2/read_req. READ_REQUEST event of MN of Super CPU cluster #2.
+
+The event code is represented by 12 bits.
+   i) event 0-11
+   The event code will be represented using the LSB 12 bits.
+
+The driver also provides a "cpumask" sysfs attribute, which shows the CPU core
+ID used to count the uncore PMU event.
+
+Example usage of perf:
+$# perf list
+hisi_l3c0_2/read_hit/ [kernel PMU event]
+--
+hisi_l3c1_2/write_hit/ [kernel PMU event]
+--
+hisi_l3c0_1/read_hit/ [kernel PMU event]
+--
+hisi_l3c0_1/write_hit/ [kernel PMU event]
+--
+hisi_mn_2/read_req/ [kernel PMU event]
+hisi_mn_2/write_req/ [kernel PMU event]
+--
+
+$# perf stat -a -e "hisi_l3c0_2/read_allocate/" sleep 5
+
+$# perf stat -A -C 0 -e "hisi_l3c0_2/read_allocate/" sleep 5
+
+The current driver doesnot support sampling. so "perf record" is unsupported.
+Also attach to a task is unsupported as the events are all uncore.
+
+Note: Please contact the maintainer for a complete list of events supported for
+the PMU devices in the SoC and its information if needed.
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH RFC v2] docs-rst: Don't use explicit Makefile rules to build SVG and DOT files

[Mauro Carvalho Chehab]

Now that we have an extension to handle images, use it.

Signed-off-by: Mauro Carvalho Chehab 
---

This patch is based on Daniel Vetter & Markus Heiser's work to support
PNG and SVG via kpicture Sphinx extension:

 [PATCH] docs-rst: automatically convert Graphviz and SVG images

v2: Don't use :caption:, as it doesn't exist on kernel-figure.

Unfortunately, this patch showed severl issues at the original patch.
I'm still sending it, as it could help testing the issues there.

 Documentation/media/Makefile   | 47 +-
 Documentation/media/intro.rst  |  6 +--
 Documentation/media/uapi/dvb/intro.rst |  6 +--
 Documentation/media/uapi/v4l/crop.rst  |  4 +-
 Documentation/media/uapi/v4l/dev-raw-vbi.rst   | 20 +
 Documentation/media/uapi/v4l/dev-subdev.rst| 22 +-
 Documentation/media/uapi/v4l/field-order.rst   | 12 --
 Documentation/media/uapi/v4l/pixfmt-nv12mt.rst |  8 ++--
 Documentation/media/uapi/v4l/selection-api-003.rst |  6 +--
 Documentation/media/uapi/v4l/subdev-formats.rst|  4 +-
 10 files changed, 46 insertions(+), 89 deletions(-)

diff --git a/Documentation/media/Makefile b/Documentation/media/Makefile
index 32663602ff25..5bd52ea1eff0 100644
--- a/Documentation/media/Makefile
+++ b/Documentation/media/Makefile
@@ -1,51 +1,6 @@
-# Rules to convert DOT and SVG to Sphinx images
-
-SRC_DIR=$(srctree)/Documentation/media
-
-DOTS = \
-   uapi/v4l/pipeline.dot \
-
-IMAGES = \
-   typical_media_device.svg \
-   uapi/dvb/dvbstb.svg \
-   uapi/v4l/bayer.svg \
-   uapi/v4l/constraints.svg \
-   uapi/v4l/crop.svg \
-   uapi/v4l/fieldseq_bt.svg \
-   uapi/v4l/fieldseq_tb.svg \
-   uapi/v4l/nv12mt.svg \
-   uapi/v4l/nv12mt_example.svg \
-   uapi/v4l/pipeline.svg \
-   uapi/v4l/selection.svg \
-   uapi/v4l/subdev-image-processing-full.svg \
-   uapi/v4l/subdev-image-processing-scaling-multi-source.svg \
-   uapi/v4l/subdev-image-processing-crop.svg \
-   uapi/v4l/vbi_525.svg \
-   uapi/v4l/vbi_625.svg \
-   uapi/v4l/vbi_hsync.svg \
-
-DOTTGT := $(patsubst %.dot,%.svg,$(DOTS))
-IMGDOT := $(patsubst %,$(SRC_DIR)/%,$(DOTTGT))
-
-IMGTGT := $(patsubst %.svg,%.pdf,$(IMAGES))
-IMGPDF := $(patsubst %,$(SRC_DIR)/%,$(IMGTGT))
-
-cmd = $(echo-cmd) $(cmd_$(1))
-
-quiet_cmd_genpdf = GENPDF  $2
-  cmd_genpdf = convert $2 $3
-
-quiet_cmd_gendot = DOT $2
-  cmd_gendot = dot -Tsvg $2 > $3
-
-%.pdf: %.svg
-   @$(call cmd,genpdf,$<,$@)
-
-%.svg: %.dot
-   @$(call cmd,gendot,$<,$@)
-
 # Rules to convert a .h file to inline RST documentation
 
+SRC_DIR=$(srctree)/Documentation/media
 PARSER = $(srctree)/Documentation/sphinx/parse-headers.pl
 UAPI = $(srctree)/include/uapi/linux
 KAPI = $(srctree)/include/linux
diff --git a/Documentation/media/intro.rst b/Documentation/media/intro.rst
index 8f7490c9a8ef..9ce2e23a0236 100644
--- a/Documentation/media/intro.rst
+++ b/Documentation/media/intro.rst
@@ -13,9 +13,9 @@ A typical media device hardware is shown at 
:ref:`typical_media_device`.
 
 .. _typical_media_device:
 
-.. figure::  typical_media_device.*
-:alt:typical_media_device.pdf / typical_media_device.svg
-:align:  center
+.. kernel-figure:: typical_media_device.svg
+:alt:   typical_media_device.svg
+:align: center
 
 Typical Media Device
 
diff --git a/Documentation/media/uapi/dvb/intro.rst 
b/Documentation/media/uapi/dvb/intro.rst
index 2ed5c23102b4..652c4aacd2c6 100644
--- a/Documentation/media/uapi/dvb/intro.rst
+++ b/Documentation/media/uapi/dvb/intro.rst
@@ -55,9 +55,9 @@ Overview
 
 .. _stb_components:
 
-.. figure::  dvbstb.*
-:alt:dvbstb.pdf / dvbstb.svg
-:align:  center
+.. kernel-figure:: dvbstb.svg
+:alt:   dvbstb.svg
+:align: center
 
 Components of a DVB card/STB
 
diff --git a/Documentation/media/uapi/v4l/crop.rst 
b/Documentation/media/uapi/v4l/crop.rst
index be58894c9c89..182565b9ace4 100644
--- a/Documentation/media/uapi/v4l/crop.rst
+++ b/Documentation/media/uapi/v4l/crop.rst
@@ -53,8 +53,8 @@ Cropping Structures
 
 .. _crop-scale:
 
-.. figure::  crop.*
-:alt:crop.pdf / crop.svg
+.. kernel-figure:: crop.svg
+:alt:crop.svg
 :align:  center
 
 Image Cropping, Insertion and Scaling
diff --git a/Documentation/media/uapi/v4l/dev-raw-vbi.rst 
b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
index baf5f2483927..9fcc18dfe3d1 100644
--- a/Documentation/media/uapi/v4l/dev-raw-vbi.rst
+++ b/Documentation/media/uapi/v4l/dev-raw-vbi.rst
@@ -221,33 +221,31 @@ and always returns default parameters as 
:ref:`VIDIOC_G_FMT ` does
 
 .. _vbi-hsync:
 
-.. figure::  vbi_hsync.*
-:alt:vbi_hsync.pdf / vbi_hsync.svg
-:align:  center
+.. kernel-figure:: vbi_hsync.svg
+:alt:   vbi_hsync.svg
+:align: center
 
 **Figure 4.1. Line synchronization**
 
 
 .. _vbi-525:
 
-.. figure::  vbi_525.*
-:alt:

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

Hi Daniel, hi Mauro,

I have to say, that I lost control over our thread ;)

> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.

I have to come back to a more systematic work, this means;
I have to test this v3 and consider what Mauro and you 
already posted.

For this, give me some time. I will send a v4 patch this weekend.

Thanks for all for comments and hints.

 -- Markus --



> Am 02.03.2017 um 16:16 schrieb Daniel Vetter :
> 
> From: Markus Heiser 
> 
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
> 
> * DOT (http://www.graphviz.org)
> * SVG
> 
> For image handling use the 'image' replacement::
> 
>.. kernel-image::  svg_image.svg
>   :alt:simple SVG image
> 
> For figure handling use the 'figure' replacement::
> 
>.. kernel-figure::  svg_image.svg
>   :alt:simple SVG image
> 
>   SVG image example
> 
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
> 
>  .. kernel-render:: DOT
> :alt: foobar digraph
> :caption: Embedded **DOT** (Graphviz) code.
> 
> digraph foo {
>  "bar" -> "baz";
> }
> 
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
> 
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
> 
> v2: s/DOC/DOT/ in a few places (by Daniel).
> 
> v3: Simplify stuff a bit (by Daniel):
> 
> - Remove path detection and setup/check code for that. In
>  Documentation/media/Makefile we already simply use these tools,
>  better to have one consolidated check if we want/need one. Also
>  remove the convertsvg support, we require ImageMagick's convert
>  already in the doc build, no need for a 2nd fallback.
> 
> - Use sphinx for depency tracking, remove hand-rolled version.
> 
> - Forward stderr from dot and convert, otherwise debugging issues with
>  the diagrams is impossible.
> 
> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
> 
> Cc: Jonathan Corbet 
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula 
> Cc: Mauro Carvalho Chehab 
> Cc: Markus Heiser 
> Cc: Laurent Pinchart 
> Signed-off-by: Markus Heiser  (v1)
> Signed-off-by: Daniel Vetter 
> ---
> Documentation/conf.py |   2 +-
> Documentation/doc-guide/hello.dot |   3 +
> Documentation/doc-guide/sphinx.rst|  90 ++-
> Documentation/doc-guide/svg_image.svg |  10 +
> Documentation/process/changes.rst |   7 +-
> Documentation/sphinx/kfigure.py   | 444 ++
> 6 files changed, 550 insertions(+), 6 deletions(-)
> create mode 100644 Documentation/doc-guide/hello.dot
> create mode 100644 Documentation/doc-guide/svg_image.svg
> create mode 100644 Documentation/sphinx/kfigure.py
> 
> diff --git a/Documentation/conf.py b/Documentation/conf.py
> index f6823cf01275..e3f537ce2935 100644
> --- a/Documentation/conf.py
> +++ b/Documentation/conf.py
> @@ -34,7 +34,7 @@ from load_config import loadConfig
> # Add any Sphinx extension module names here, as strings. They can be
> # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
> # ones.
> -extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
> +extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 
> 'kfigure']
> 
> # The name of the math extension changed on Sphinx 1.4
> if major == 1 and minor > 3:
> diff --git a/Documentation/doc-guide/hello.dot 
> b/Documentation/doc-guide/hello.dot
> new file mode 100644
> index ..504621dfc595
> --- /dev/null
> +++ b/Documentation/doc-guide/hello.dot
> @@ -0,0 +1,3 @@
> +graph G {
> +  Hello -- World
> +}
> diff --git a/Documentation/doc-guide/sphinx.rst 
> b/Documentation/doc-guide/sphinx.rst
> index 532d65b70500..b902744ce7dd 100644
> --- a/Documentation/doc-guide/sphinx.rst
> +++ b/Documentation/doc-guide/sphinx.rst
> @@ -34,8 +34,10 @@ format-specific subdirectories under 
> ``Documentation/output``.
> 
> To generate documentation, Sphinx (``sphinx-build``) must obviously be
> installed. For prettier HTML output, the Read the Docs Sphinx theme
> -(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is 
> also
> -needed. All of these are widely available and packaged in distributions.
> +(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
> +``XeLaTeX`` and CairoSVG 

[PATCH v6 2/4] switchtec: Add user interface documentation

[Logan Gunthorpe]

This adds standard documentation for the sysfs switchtec attributes and
a RST formatted text file which documents the char device interface.
Jonathan Corbet has indicated he will move this to a new user-space
developer documentation book once it's created.

Signed-off-by: Logan Gunthorpe 
Signed-off-by: Stephen Bates 
Tested-by: Krishna Dhulipala 
Reviewed-by: Wei Zhang 
Reviewed-by: Jens Axboe 
---
 Documentation/switchtec.txt | 53 +
 MAINTAINERS |  1 +
 2 files changed, 54 insertions(+)
 create mode 100644 Documentation/switchtec.txt

diff --git a/Documentation/switchtec.txt b/Documentation/switchtec.txt
new file mode 100644
index 000..4bced4c
--- /dev/null
+++ b/Documentation/switchtec.txt
@@ -0,0 +1,53 @@
+
+Linux Switchtec Support
+
+
+Microsemi's "Switchtec" line of PCI switch devices is already
+supported by the kernel with standard PCI switch drivers. However, the
+Switchtec device advertises a special management endpoint which
+enables some additional functionality. This includes:
+
+* Packet and Byte Counters
+* Firmware Upgrades
+* Event and Error logs
+* Querying port link status
+* Custom user firmware commands
+
+The switchtec kernel module implements this functionality.
+
+
+Interface
+=
+
+The primary means of communicating with the Switchtec management firmware is
+through the Memory-mapped Remote Procedure Call (MRPC) interface.
+Commands are submitted to the interface with a 4-byte command
+identifier and up to 1KB of command specific data. The firmware will
+respond with a 4 bytes return code and up to 1KB of command specific
+data. The interface only processes a single command at a time.
+
+
+Userspace Interface
+===
+
+The MRPC interface will be exposed to userspace through a simple char
+device: /dev/switchtec#, one for each management endpoint in the system.
+
+The char device has the following semantics:
+
+* A write must consist of at least 4 bytes and no more than 1028 bytes.
+  The first four bytes will be interpreted as the command to run and
+  the remainder will be used as the input data. A write will send the
+  command to the firmware to begin processing.
+
+* Each write must be followed by exactly one read. Any double write will
+  produce an error and any read that doesn't follow a write will
+  produce an error.
+
+* A read will block until the firmware completes the command and return
+  the four bytes of status plus up to 1024 bytes of output data. (The
+  length will be specified by the size parameter of the read call --
+  reading less than 4 bytes will produce an error.
+
+* The poll call will also be supported for userspace applications that
+  need to do other things while waiting for the command to complete.
diff --git a/MAINTAINERS b/MAINTAINERS
index a57686f..aa702b0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9512,6 +9512,7 @@ M:Stephen Bates 
 M: Logan Gunthorpe 
 L: linux-...@vger.kernel.org
 S: Maintained
+F: Documentation/switchtec.txt
 F: drivers/pci/switch/switchtec*
 
 PCI DRIVER FOR NVIDIA TEGRA
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 22:16:49 +0100
Markus Heiser  escreveu:

> > Am 02.03.2017 um 20:34 schrieb Mauro Carvalho Chehab 
> > :
> > 
> > Em Thu, 2 Mar 2017 20:06:39 +0100
> > Markus Heiser  escreveu:
> >   
> >> Hi Mauro,
> >>   
> >>> Tested here with the enclosed patch.
> >> 
> >> great, big step forward making /media/Makefile smaller ...  thanks a 
> >> lot
> >>   
> >>> It crashed:
> >>> Exception occurred:
> >>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
> >>> dot2format
> >>>   sys.stderr.write(err)
> >>> TypeError: write() argument must be str, not bytes
> >>> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you 
> >>> want to report the issue to the developers.
> >>> Please also report this if it was a user error, so that a better error 
> >>> message can be provided next time.
> >>> A bug report can be filed in the tracker at 
> >>> . Thanks!
> >>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> >>> make[1]: *** [htmldocs] Error 1
> >>> Makefile:1450: recipe for target 'htmldocs' failed
> >>> make: *** [htmldocs] Error 2
> >>> 
> >>> Weird enough, it produced a 
> >>> Documentation/output/media/uapi/v4l/pipeline.svg file.
> >> 
> >> I guess that the dot command writes something to stderr. This is captured 
> >> by the extension and printed to stderr ...
> >> 
> >> +def dot2format(dot_fname, out_fname):
> >> ...
> >> +exit_code = 42
> >> +with open(out_fname, "w") as out:
> >> +p = subprocess.Popen(
> >> +cmd, stdout = out, stderr = subprocess.PIPE )
> >> +nil, err = p.communicate()
> >> +
> >> +sys.stderr.write(err)
> >> +
> >> +exit_code = p.returncode
> >> +out.flush()
> >> +return bool(exit_code == 0)
> >>   
> >>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
> >>> dot2format
> >>>   sys.stderr.write(err)
> >>> TypeError: write() argument must be str, not bytes
> >> 
> >> Do we need this stderr output? For a first test, uncomment the 
> >> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> >> stderr, try:
> >> 
> >> -sys.stderr.write(err)
> >> +sys.stderr.write(str(err))  
> > 
> > Yes, this fixed. I actually did:
> > 
> > -sys.stderr.write(err)
> > +sys.stderr.write(str(err))
> > +sys.stderr.write("\n")
> > 
> > It is now printing:
> > b''
> > 
> > I added the \n print to avoid it to be mixed with the "writing output"
> > prints.
> > No idea how to make sense from it - but clearly, the error report
> > logic require some care ;-)  
> 
> 
> Aargh, I’am a idiot ... I guess 'sys.stderr.write(err)‘ is a artefact
> of my development, simply drop it and the subprocess.PIPE of stderr
> also.
> 
> +with open(out_fname, "w") as out:
> +p = subprocess.Popen(
> -cmd, stdout = out, stderr = subprocess.PIPE )
> +cmd, stdout = out)
> +nil, err = p.communicate()
> -
> -sys.stderr.write(err)
> -
> +exit_code = p.returncode
> +out.flush()
> +return bool(exit_code == 0)
> 
> I can’t test it ATM, but without redirect stderr, the stderr
> of the parent process is inherited.
> 
>   https://docs.python.org/3.6/library/subprocess.html#popen-constructor
> 
> The Popen.communicate() always returns a tuple (stdout_data, stderr_data)
> with above the tuple is always (None, None).
> 
>   
> https://docs.python.org/3.6/library/subprocess.html#subprocess.Popen.communicate
> 
>   """to get anything other than None in the result tuple,
>  you need to give stdout=PIPE and/or stderr=PIPE too."""
> 
> Sorry, that I made all this mistakes, but „here“ I have only mail
> and web, no dev-env and I miss my emacs  ;) 
> 
> If the suggestion above does not work, I have to investigate
> more time next weekend.

Hmm... I would be more verbose on output the error code, printing from
where the error came, e. g. printing a message like:

Error #0 when calling dot for 
'/devel/v4l/patchwork/Documentation/media/uapi/v4l/pipeline.dot': None  


As on this patch:

diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..a366a89f4f98 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,13 @@ def dot2format(dot_fname, out_fname):
 with open(out_fname, "w") as out:
 p = subprocess.Popen(
 cmd, stdout = out, stderr = subprocess.PIPE )
-nil, err = p.communicate()
-
-sys.stderr.write(err)
+err = p.communicate()
 
 exit_code = p.returncode
+
+if exit_code != 0:
+sys.stderr.write("Error #%d when calling dot for '%s': %s\n" % 
(exit_code, dot_fname, repr(err[0])))
+
 

[PATCH v6 0/4] New Microsemi PCI Switch Management Driver

[Logan Gunthorpe]

Hi,

Hopefully this is the last change necessary. The new code to handle
unbind follows Jason's suggestions and is a lot easier to reason about.
It's also been tested interrupting a number of operations.

Thanks,

Logan

--

Changes since v5:

* Reworked cleanup during unbind again. This time we follow the pattern
  roughly suggested by Jason Gunthorpe: we use an alive flag
  that's checked with a rw semaphore held by the cdev ops. Except
  instead of using the rwsem we just use the already existing
  mrpc_mutex. (Seeing the vast majority of locations that would use the
  semaphore overlap with the existing mutex.)
* Added a small performance enhancement so the code reads less io memory
  if the user is quick in submitting their read request.

Changes since v4:

* Turns out pushing the pci release code into the device release
  function didn't work as I would have liked. If you try to unbind the
  device with an instance open, then you hit a kernel bug at
  drivers/pci/msi.c:371. (This didn't occur in v3.)

  To solve this, we've moved the pci release code back into the
  unregister function and reintroduced an alive flag. This time,
  however, the alive flag is protected by mrpc_mutex and we're very
  careful about what happens to devices still in use (they should
  all be released through the timeout path and an ENODEV error
  returned to userspace; while new commands are blocked with the
  same error).

Changes since v3:

* Removed stdev_is_alive and moved pci deinit functions
  into the device release function which only occurs after all
  cdev instances are closed. (per Bjorn)
* Reduced locking in read/write functions (per Bjorn)
* Use pci_alloc_irq_vectors to vastly improve ISR initialization (Bjorn)
* A few other minor nits and cleanup as noticed by Bjorn

Changes since v2:

* Collected reviewed and tested tags
* Very minor fix to the error path in the create function

Changes since v1:

* Rebased onto 4.10-rc6 (cleanly)
* Split the patch into a few more easily digestible patches (as
  suggested by Greg Kroah-Hartman)
* Folded switchtec.c into switchtec.h (per Greg)
* Fixed a bunch of 32bit build warnings caught by the kbuild test robot
* Fixed some issues in the documentation so it has a proper
  reStructredText format (as noted by Jonathan Corbet)
* Fixed padding and sizes in the IOCTL structures as noticed by Emil
  Velikov and used pahole to verify their consistency across 32 and 64
  bit builds
* Reworked one of the IOCTL interfaces to be more future proof (per
  Emil).

Changes since RFC:

* Fixed incorrect use of the drive model as pointed out by Greg
  Kroah-Hartman
* Used devm functions as suggested by Keith Busch
* Added a handful of sysfs attributes to the switchtec class
* Added a handful of IOCTLs to the switchtec device
* A number of miscellaneous bug fixes

--

Hi,

This is a continuation of the RFC we posted lasted month [1] which
proposes a management driver for Microsemi's Switchtec line of PCI
switches. This hardware is still looking to be used in the Open
Compute Platform

To make this entirely clear: the Switchtec products are compliant
with the PCI specifications and are supported today with the standard
in-kernel driver. However, these devices also expose a management endpoint
on a separate PCI function address which can be used to perform some
advanced operations. This is a driver for that function. See the patch
for more information.

Since the RFC, we've made the changes requested by Greg Kroah-Hartman
and Keith Busch, and we've also fleshed out a number of features. We've
added a couple of IOCTLs and sysfs attributes which are documented in
the patch. Significant work has also been done on the userspace tool
which is available under a GPL license at [2]. We've also had testing
done by some of the interested parties.

We hope to see this work included in either 4.11 or 4.12 assuming a
smooth review process.

The patch is based off of the v4.10-rc6 release.

Thanks for your review,

Logan

[1] https://www.spinics.net/lists/linux-pci/msg56897.html
[2] https://github.com/sbates130272/switchtec-user

Logan Gunthorpe (4):
  MicroSemi Switchtec management interface driver
  switchtec: Add user interface documentation
  switchtec: Add sysfs attributes to the Switchtec driver
  switchtec: Add IOCTLs to the Switchtec driver

 Documentation/ABI/testing/sysfs-class-switchtec |   96 ++
 Documentation/ioctl/ioctl-number.txt|1 +
 Documentation/switchtec.txt |   80 ++
 MAINTAINERS |   11 +
 drivers/pci/Kconfig |1 +
 drivers/pci/Makefile|1 +
 drivers/pci/switch/Kconfig  |   13 +
 drivers/pci/switch/Makefile |1 +
 drivers/pci/switch/switchtec.c  | 1599 +++
 include/uapi/linux/switchtec_ioctl.h|  132 ++
 10 files changed, 1935 insertions(+)
 create mode 100644 

[PATCH v7 4/4] switchtec: Add IOCTLs to the Switchtec driver

[Logan Gunthorpe]

This patch introduces a couple of special IOCTLs which are provided to:

* Inform userspace of firmware partition locations
* Pass event counts and allow userspace to wait on events
* Translate between PFF numbers used by the switch to
  port numbers.

Signed-off-by: Logan Gunthorpe 
Signed-off-by: Stephen Bates 
Tested-by: Krishna Dhulipala 
Reviewed-by: Wei Zhang 
Reviewed-by: Jens Axboe 
---
 Documentation/ioctl/ioctl-number.txt |   1 +
 Documentation/switchtec.txt  |  27 ++
 MAINTAINERS  |   1 +
 drivers/pci/switch/switchtec.c   | 481 +++
 include/uapi/linux/switchtec_ioctl.h | 132 ++
 5 files changed, 642 insertions(+)
 create mode 100644 include/uapi/linux/switchtec_ioctl.h

diff --git a/Documentation/ioctl/ioctl-number.txt 
b/Documentation/ioctl/ioctl-number.txt
index 81c7f2b..032b33c 100644
--- a/Documentation/ioctl/ioctl-number.txt
+++ b/Documentation/ioctl/ioctl-number.txt
@@ -191,6 +191,7 @@ Code  Seq#(hex) Include FileComments
 'W'00-1F   linux/watchdog.hconflict!
 'W'00-1F   linux/wanrouter.h   conflict!   (pre 3.9)
 'W'00-3F   sound/asound.h  conflict!
+'W'40-5F   drivers/pci/switch/switchtec.c
 'X'all fs/xfs/xfs_fs.h conflict!
and fs/xfs/linux-2.6/xfs_ioctl32.h
and include/linux/falloc.h
diff --git a/Documentation/switchtec.txt b/Documentation/switchtec.txt
index 4bced4c..a0a9c7b 100644
--- a/Documentation/switchtec.txt
+++ b/Documentation/switchtec.txt
@@ -51,3 +51,30 @@ The char device has the following semantics:
 
 * The poll call will also be supported for userspace applications that
   need to do other things while waiting for the command to complete.
+
+The following IOCTLs are also supported by the device:
+
+* SWITCHTEC_IOCTL_FLASH_INFO - Retrieve firmware length and number
+  of partitions in the device.
+
+* SWITCHTEC_IOCTL_FLASH_PART_INFO - Retrieve address and lengeth for
+  any specified partition in flash.
+
+* SWITCHTEC_IOCTL_EVENT_SUMMARY - Read a structure of bitmaps
+  indicating all uncleared events.
+
+* SWITCHTEC_IOCTL_EVENT_CTL - Get the current count, clear and set flags
+  for any event. This ioctl takes in a switchtec_ioctl_event_ctl struct
+  with the event_id, index and flags set (index being the partition or PFF
+  number for non-global events). It returns whether the event has
+  occurred, the number of times and any event specific data. The flags
+  can be used to clear the count or enable and disable actions to
+  happen when the event occurs.
+  By using the SWITCHTEC_IOCTL_EVENT_FLAG_EN_POLL flag,
+  you can set an event to trigger a poll command to return with
+  POLLPRI. In this way, userspace can wait for events to occur.
+
+* SWITCHTEC_IOCTL_PFF_TO_PORT and SWITCHTEC_IOCTL_PORT_TO_PFF convert
+  between PCI Function Framework number (used by the event system)
+  and Switchtec Logic Port ID and Partition number (which is more
+  user friendly).
diff --git a/MAINTAINERS b/MAINTAINERS
index 6fed938..c1a9a30 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9515,6 +9515,7 @@ S:Maintained
 F: Documentation/switchtec.txt
 F: Documentation/ABI/testing/sysfs-class-switchtec
 F: drivers/pci/switch/switchtec*
+F: include/uapi/linux/switchtec_ioctl.h
 
 PCI DRIVER FOR NVIDIA TEGRA
 M: Thierry Reding 
diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
index 4888e23..1f045c9 100644
--- a/drivers/pci/switch/switchtec.c
+++ b/drivers/pci/switch/switchtec.c
@@ -13,6 +13,8 @@
  *
  */
 
+#include 
+
 #include 
 #include 
 #include 
@@ -778,6 +780,431 @@ static unsigned int switchtec_dev_poll(struct file *filp, 
poll_table *wait)
return ret;
 }
 
+static int ioctl_flash_info(struct switchtec_dev *stdev,
+   struct switchtec_ioctl_flash_info __user *uinfo)
+{
+   struct switchtec_ioctl_flash_info info = {0};
+   struct flash_info_regs __iomem *fi = stdev->mmio_flash_info;
+
+   info.flash_length = ioread32(>flash_length);
+   info.num_partitions = SWITCHTEC_IOCTL_NUM_PARTITIONS;
+
+   if (copy_to_user(uinfo, , sizeof(info)))
+   return -EFAULT;
+
+   return 0;
+}
+
+static void set_fw_info_part(struct switchtec_ioctl_flash_part_info *info,
+struct partition_info __iomem *pi)
+{
+   info->address = ioread32(>address);
+   info->length = ioread32(>length);
+}
+
+static int ioctl_flash_part_info(struct switchtec_dev *stdev,
+   struct switchtec_ioctl_flash_part_info __user *uinfo)
+{
+   struct switchtec_ioctl_flash_part_info info = {0};
+   struct flash_info_regs __iomem *fi = stdev->mmio_flash_info;
+   u32 active_addr = -1;
+
+   if (copy_from_user(, uinfo, sizeof(info)))
+   return 

[PATCH v7 1/4] MicroSemi Switchtec management interface driver

[Logan Gunthorpe]

Microsemi's "Switchtec" line of PCI switch devices is already well
supported by the kernel with standard PCI switch drivers. However, the
Switchtec device advertises a special management endpoint with a separate
PCI function address and class code. This endpoint enables some additional
functionality which includes:

 * Packet and Byte Counters
 * Switch Firmware Upgrades
 * Event and Error logs
 * Querying port link status
 * Custom user firmware commands

This patch introduces the switchtec kernel module which provides
PCI driver that exposes a char device. The char device provides
userspace access to this interface through read, write and (optionally)
poll calls.

A userspace tool and library which utilizes this interface is available
at [1]. This tool takes inspiration (and borrows some code) from
nvme-cli [2]. The tool is largely complete at this time but additional
features may be added in the future.

[1] https://github.com/sbates130272/switchtec-user
[2] https://github.com/linux-nvme/nvme-cli

Signed-off-by: Logan Gunthorpe 
Signed-off-by: Stephen Bates 
Tested-by: Krishna Dhulipala 
Reviewed-by: Wei Zhang 
Reviewed-by: Jens Axboe 
Reviewed-by: Greg Kroah-Hartman 
---
 MAINTAINERS|8 +
 drivers/pci/Kconfig|1 +
 drivers/pci/Makefile   |1 +
 drivers/pci/switch/Kconfig |   13 +
 drivers/pci/switch/Makefile|1 +
 drivers/pci/switch/switchtec.c | 1006 
 6 files changed, 1030 insertions(+)
 create mode 100644 drivers/pci/switch/Kconfig
 create mode 100644 drivers/pci/switch/Makefile
 create mode 100644 drivers/pci/switch/switchtec.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 527d137..a57686f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9506,6 +9506,14 @@ S:   Maintained
 F: Documentation/devicetree/bindings/pci/aardvark-pci.txt
 F: drivers/pci/host/pci-aardvark.c
 
+PCI DRIVER FOR MICROSEMI SWITCHTEC
+M: Kurt Schwemmer 
+M: Stephen Bates 
+M: Logan Gunthorpe 
+L: linux-...@vger.kernel.org
+S: Maintained
+F: drivers/pci/switch/switchtec*
+
 PCI DRIVER FOR NVIDIA TEGRA
 M: Thierry Reding 
 L: linux-te...@vger.kernel.org
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 6555eb7..f72e8c5 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -133,3 +133,4 @@ config PCI_HYPERV
 
 source "drivers/pci/hotplug/Kconfig"
 source "drivers/pci/host/Kconfig"
+source "drivers/pci/switch/Kconfig"
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 8db5079..15b46dd 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -68,3 +68,4 @@ ccflags-$(CONFIG_PCI_DEBUG) := -DDEBUG
 
 # PCI host controller drivers
 obj-y += host/
+obj-y += switch/
diff --git a/drivers/pci/switch/Kconfig b/drivers/pci/switch/Kconfig
new file mode 100644
index 000..4c49648
--- /dev/null
+++ b/drivers/pci/switch/Kconfig
@@ -0,0 +1,13 @@
+menu "PCI switch controller drivers"
+   depends on PCI
+
+config PCI_SW_SWITCHTEC
+   tristate "MicroSemi Switchtec PCIe Switch Management Driver"
+   help
+Enables support for the management interface for the MicroSemi
+Switchtec series of PCIe switches. Supports userspace access
+to submit MRPC commands to the switch via /dev/switchtecX
+devices. See  for more
+information.
+
+endmenu
diff --git a/drivers/pci/switch/Makefile b/drivers/pci/switch/Makefile
new file mode 100644
index 000..37d8cfb
--- /dev/null
+++ b/drivers/pci/switch/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_PCI_SW_SWITCHTEC) += switchtec.o
diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
new file mode 100644
index 000..084e4ed
--- /dev/null
+++ b/drivers/pci/switch/switchtec.c
@@ -0,0 +1,1006 @@
+/*
+ * Microsemi Switchtec(tm) PCIe Management Driver
+ * Copyright (c) 2017, Microsemi Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+MODULE_DESCRIPTION("Microsemi Switchtec(tm) PCIe Management Driver");
+MODULE_VERSION("0.1");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Microsemi Corporation");
+
+static int max_devices = 16;
+module_param(max_devices, int, 0644);
+MODULE_PARM_DESC(max_devices, "max number of switchtec device instances");
+

[PATCH v6 1/4] MicroSemi Switchtec management interface driver

[Logan Gunthorpe]

Microsemi's "Switchtec" line of PCI switch devices is already well
supported by the kernel with standard PCI switch drivers. However, the
Switchtec device advertises a special management endpoint with a separate
PCI function address and class code. This endpoint enables some additional
functionality which includes:

 * Packet and Byte Counters
 * Switch Firmware Upgrades
 * Event and Error logs
 * Querying port link status
 * Custom user firmware commands

This patch introduces the switchtec kernel module which provides
PCI driver that exposes a char device. The char device provides
userspace access to this interface through read, write and (optionally)
poll calls.

A userspace tool and library which utilizes this interface is available
at [1]. This tool takes inspiration (and borrows some code) from
nvme-cli [2]. The tool is largely complete at this time but additional
features may be added in the future.

[1] https://github.com/sbates130272/switchtec-user
[2] https://github.com/linux-nvme/nvme-cli

Signed-off-by: Logan Gunthorpe 
Signed-off-by: Stephen Bates 
Tested-by: Krishna Dhulipala 
Reviewed-by: Wei Zhang 
Reviewed-by: Jens Axboe 
Reviewed-by: Greg Kroah-Hartman 
---
 MAINTAINERS|8 +
 drivers/pci/Kconfig|1 +
 drivers/pci/Makefile   |1 +
 drivers/pci/switch/Kconfig |   13 +
 drivers/pci/switch/Makefile|1 +
 drivers/pci/switch/switchtec.c | 1005 
 6 files changed, 1029 insertions(+)
 create mode 100644 drivers/pci/switch/Kconfig
 create mode 100644 drivers/pci/switch/Makefile
 create mode 100644 drivers/pci/switch/switchtec.c

diff --git a/MAINTAINERS b/MAINTAINERS
index 527d137..a57686f 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9506,6 +9506,14 @@ S:   Maintained
 F: Documentation/devicetree/bindings/pci/aardvark-pci.txt
 F: drivers/pci/host/pci-aardvark.c
 
+PCI DRIVER FOR MICROSEMI SWITCHTEC
+M: Kurt Schwemmer 
+M: Stephen Bates 
+M: Logan Gunthorpe 
+L: linux-...@vger.kernel.org
+S: Maintained
+F: drivers/pci/switch/switchtec*
+
 PCI DRIVER FOR NVIDIA TEGRA
 M: Thierry Reding 
 L: linux-te...@vger.kernel.org
diff --git a/drivers/pci/Kconfig b/drivers/pci/Kconfig
index 6555eb7..f72e8c5 100644
--- a/drivers/pci/Kconfig
+++ b/drivers/pci/Kconfig
@@ -133,3 +133,4 @@ config PCI_HYPERV
 
 source "drivers/pci/hotplug/Kconfig"
 source "drivers/pci/host/Kconfig"
+source "drivers/pci/switch/Kconfig"
diff --git a/drivers/pci/Makefile b/drivers/pci/Makefile
index 8db5079..15b46dd 100644
--- a/drivers/pci/Makefile
+++ b/drivers/pci/Makefile
@@ -68,3 +68,4 @@ ccflags-$(CONFIG_PCI_DEBUG) := -DDEBUG
 
 # PCI host controller drivers
 obj-y += host/
+obj-y += switch/
diff --git a/drivers/pci/switch/Kconfig b/drivers/pci/switch/Kconfig
new file mode 100644
index 000..4c49648
--- /dev/null
+++ b/drivers/pci/switch/Kconfig
@@ -0,0 +1,13 @@
+menu "PCI switch controller drivers"
+   depends on PCI
+
+config PCI_SW_SWITCHTEC
+   tristate "MicroSemi Switchtec PCIe Switch Management Driver"
+   help
+Enables support for the management interface for the MicroSemi
+Switchtec series of PCIe switches. Supports userspace access
+to submit MRPC commands to the switch via /dev/switchtecX
+devices. See  for more
+information.
+
+endmenu
diff --git a/drivers/pci/switch/Makefile b/drivers/pci/switch/Makefile
new file mode 100644
index 000..37d8cfb
--- /dev/null
+++ b/drivers/pci/switch/Makefile
@@ -0,0 +1 @@
+obj-$(CONFIG_PCI_SW_SWITCHTEC) += switchtec.o
diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
new file mode 100644
index 000..2993c0c
--- /dev/null
+++ b/drivers/pci/switch/switchtec.c
@@ -0,0 +1,1005 @@
+/*
+ * Microsemi Switchtec(tm) PCIe Management Driver
+ * Copyright (c) 2017, Microsemi Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU General Public License,
+ * version 2, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT
+ * ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
+ * FITNESS FOR A PARTICULAR PURPOSE.  See the GNU General Public License for
+ * more details.
+ *
+ */
+
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+#include 
+
+MODULE_DESCRIPTION("Microsemi Switchtec(tm) PCIe Management Driver");
+MODULE_VERSION("0.1");
+MODULE_LICENSE("GPL");
+MODULE_AUTHOR("Microsemi Corporation");
+
+static int max_devices = 16;
+module_param(max_devices, int, 0644);
+MODULE_PARM_DESC(max_devices, "max number of switchtec device instances");
+

[PATCH v6 3/4] switchtec: Add sysfs attributes to the Switchtec driver

[Logan Gunthorpe]

This patch adds a few read-only sysfs attributes which provide
some device information that is exposed from the devices. Primarily
component and device names and versions. These are documented in
Documentation/ABI/testing/sysfs-class-switchtec.

Signed-off-by: Logan Gunthorpe 
Signed-off-by: Stephen Bates 
Tested-by: Krishna Dhulipala 
Reviewed-by: Wei Zhang 
Reviewed-by: Jens Axboe 
Reviewed-by: Greg Kroah-Hartman 
---
 Documentation/ABI/testing/sysfs-class-switchtec |  96 
 MAINTAINERS |   1 +
 drivers/pci/switch/switchtec.c  | 113 
 3 files changed, 210 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-class-switchtec

diff --git a/Documentation/ABI/testing/sysfs-class-switchtec 
b/Documentation/ABI/testing/sysfs-class-switchtec
new file mode 100644
index 000..48cb4c1
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-class-switchtec
@@ -0,0 +1,96 @@
+switchtec - Microsemi Switchtec PCI Switch Management Endpoint
+
+For details on this subsystem look at Documentation/switchtec.txt.
+
+What:  /sys/class/switchtec
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   The switchtec class subsystem folder.
+   Each registered switchtec driver is represented by a switchtecX
+   subfolder (X being an integer >= 0).
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/component_id
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Component identifier as stored in the hardware (eg. PM8543)
+   (read only)
+Values:arbitrary string.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/component_revision
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Component revision stored in the hardware (read only)
+Values:integer.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/component_vendor
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Component vendor as stored in the hardware (eg. MICROSEM)
+   (read only)
+Values:arbitrary string.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/device_version
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Device version as stored in the hardware (read only)
+Values:integer.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/fw_version
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Currently running firmware version (read only)
+Values:integer (in hexadecimal).
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/partition
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Partition number for this device in the switch (read only)
+Values:integer.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/partition_count
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Total number of partitions in the switch (read only)
+Values:integer.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/product_id
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Product identifier as stored in the hardware (eg. PSX 48XG3)
+   (read only)
+Values:arbitrary string.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/product_revision
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Product revision stored in the hardware (eg. RevB)
+   (read only)
+Values:arbitrary string.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/product_vendor
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Product vendor as stored in the hardware (eg. MICROSEM)
+   (read only)
+Values:arbitrary string.
diff --git a/MAINTAINERS b/MAINTAINERS
index aa702b0..6fed938 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9513,6 +9513,7 @@ M:Logan Gunthorpe 
 L: linux-...@vger.kernel.org
 S: Maintained
 F: Documentation/switchtec.txt
+F: Documentation/ABI/testing/sysfs-class-switchtec
 F: drivers/pci/switch/switchtec*
 
 PCI DRIVER FOR NVIDIA TEGRA
diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
index 2993c0c..ab08aa4 100644
--- 

Re: [PATCH v6 0/4] New Microsemi PCI Switch Management Driver

[Logan Gunthorpe]

Hey,

Ah, sorry, sorry. I noticed another minor mistake. I'll send a v7 in a
second.

Thanks,

Logan

On 02/03/17 04:07 PM, Logan Gunthorpe wrote:
> Hi,
> 
> Hopefully this is the last change necessary. The new code to handle
> unbind follows Jason's suggestions and is a lot easier to reason about.
> It's also been tested interrupting a number of operations.
> 
> Thanks,
> 
> Logan
> 
> --
> 
> Changes since v5:
> 
> * Reworked cleanup during unbind again. This time we follow the pattern
>   roughly suggested by Jason Gunthorpe: we use an alive flag
>   that's checked with a rw semaphore held by the cdev ops. Except
>   instead of using the rwsem we just use the already existing
>   mrpc_mutex. (Seeing the vast majority of locations that would use the
>   semaphore overlap with the existing mutex.)
> * Added a small performance enhancement so the code reads less io memory
>   if the user is quick in submitting their read request.
> 
> Changes since v4:
> 
> * Turns out pushing the pci release code into the device release
>   function didn't work as I would have liked. If you try to unbind the
>   device with an instance open, then you hit a kernel bug at
>   drivers/pci/msi.c:371. (This didn't occur in v3.)
> 
>   To solve this, we've moved the pci release code back into the
>   unregister function and reintroduced an alive flag. This time,
>   however, the alive flag is protected by mrpc_mutex and we're very
>   careful about what happens to devices still in use (they should
>   all be released through the timeout path and an ENODEV error
>   returned to userspace; while new commands are blocked with the
>   same error).
> 
> Changes since v3:
> 
> * Removed stdev_is_alive and moved pci deinit functions
>   into the device release function which only occurs after all
>   cdev instances are closed. (per Bjorn)
> * Reduced locking in read/write functions (per Bjorn)
> * Use pci_alloc_irq_vectors to vastly improve ISR initialization (Bjorn)
> * A few other minor nits and cleanup as noticed by Bjorn
> 
> Changes since v2:
> 
> * Collected reviewed and tested tags
> * Very minor fix to the error path in the create function
> 
> Changes since v1:
> 
> * Rebased onto 4.10-rc6 (cleanly)
> * Split the patch into a few more easily digestible patches (as
>   suggested by Greg Kroah-Hartman)
> * Folded switchtec.c into switchtec.h (per Greg)
> * Fixed a bunch of 32bit build warnings caught by the kbuild test robot
> * Fixed some issues in the documentation so it has a proper
>   reStructredText format (as noted by Jonathan Corbet)
> * Fixed padding and sizes in the IOCTL structures as noticed by Emil
>   Velikov and used pahole to verify their consistency across 32 and 64
>   bit builds
> * Reworked one of the IOCTL interfaces to be more future proof (per
>   Emil).
> 
> Changes since RFC:
> 
> * Fixed incorrect use of the drive model as pointed out by Greg
>   Kroah-Hartman
> * Used devm functions as suggested by Keith Busch
> * Added a handful of sysfs attributes to the switchtec class
> * Added a handful of IOCTLs to the switchtec device
> * A number of miscellaneous bug fixes
> 
> --
> 
> Hi,
> 
> This is a continuation of the RFC we posted lasted month [1] which
> proposes a management driver for Microsemi's Switchtec line of PCI
> switches. This hardware is still looking to be used in the Open
> Compute Platform
> 
> To make this entirely clear: the Switchtec products are compliant
> with the PCI specifications and are supported today with the standard
> in-kernel driver. However, these devices also expose a management endpoint
> on a separate PCI function address which can be used to perform some
> advanced operations. This is a driver for that function. See the patch
> for more information.
> 
> Since the RFC, we've made the changes requested by Greg Kroah-Hartman
> and Keith Busch, and we've also fleshed out a number of features. We've
> added a couple of IOCTLs and sysfs attributes which are documented in
> the patch. Significant work has also been done on the userspace tool
> which is available under a GPL license at [2]. We've also had testing
> done by some of the interested parties.
> 
> We hope to see this work included in either 4.11 or 4.12 assuming a
> smooth review process.
> 
> The patch is based off of the v4.10-rc6 release.
> 
> Thanks for your review,
> 
> Logan
> 
> [1] https://www.spinics.net/lists/linux-pci/msg56897.html
> [2] https://github.com/sbates130272/switchtec-user
> 
> Logan Gunthorpe (4):
>   MicroSemi Switchtec management interface driver
>   switchtec: Add user interface documentation
>   switchtec: Add sysfs attributes to the Switchtec driver
>   switchtec: Add IOCTLs to the Switchtec driver
> 
>  Documentation/ABI/testing/sysfs-class-switchtec |   96 ++
>  Documentation/ioctl/ioctl-number.txt|1 +
>  Documentation/switchtec.txt |   80 ++
>  MAINTAINERS |   11 +
>  drivers/pci/Kconfig 

[PATCH v6 4/4] switchtec: Add IOCTLs to the Switchtec driver

[Logan Gunthorpe]

This patch introduces a couple of special IOCTLs which are provided to:

* Inform userspace of firmware partition locations
* Pass event counts and allow userspace to wait on events
* Translate between PFF numbers used by the switch to
  port numbers.

Signed-off-by: Logan Gunthorpe 
Signed-off-by: Stephen Bates 
Tested-by: Krishna Dhulipala 
Reviewed-by: Wei Zhang 
Reviewed-by: Jens Axboe 
---
 Documentation/ioctl/ioctl-number.txt |   1 +
 Documentation/switchtec.txt  |  27 ++
 MAINTAINERS  |   1 +
 drivers/pci/switch/switchtec.c   | 481 +++
 include/uapi/linux/switchtec_ioctl.h | 132 ++
 5 files changed, 642 insertions(+)
 create mode 100644 include/uapi/linux/switchtec_ioctl.h

diff --git a/Documentation/ioctl/ioctl-number.txt 
b/Documentation/ioctl/ioctl-number.txt
index 81c7f2b..032b33c 100644
--- a/Documentation/ioctl/ioctl-number.txt
+++ b/Documentation/ioctl/ioctl-number.txt
@@ -191,6 +191,7 @@ Code  Seq#(hex) Include FileComments
 'W'00-1F   linux/watchdog.hconflict!
 'W'00-1F   linux/wanrouter.h   conflict!   (pre 3.9)
 'W'00-3F   sound/asound.h  conflict!
+'W'40-5F   drivers/pci/switch/switchtec.c
 'X'all fs/xfs/xfs_fs.h conflict!
and fs/xfs/linux-2.6/xfs_ioctl32.h
and include/linux/falloc.h
diff --git a/Documentation/switchtec.txt b/Documentation/switchtec.txt
index 4bced4c..a0a9c7b 100644
--- a/Documentation/switchtec.txt
+++ b/Documentation/switchtec.txt
@@ -51,3 +51,30 @@ The char device has the following semantics:
 
 * The poll call will also be supported for userspace applications that
   need to do other things while waiting for the command to complete.
+
+The following IOCTLs are also supported by the device:
+
+* SWITCHTEC_IOCTL_FLASH_INFO - Retrieve firmware length and number
+  of partitions in the device.
+
+* SWITCHTEC_IOCTL_FLASH_PART_INFO - Retrieve address and lengeth for
+  any specified partition in flash.
+
+* SWITCHTEC_IOCTL_EVENT_SUMMARY - Read a structure of bitmaps
+  indicating all uncleared events.
+
+* SWITCHTEC_IOCTL_EVENT_CTL - Get the current count, clear and set flags
+  for any event. This ioctl takes in a switchtec_ioctl_event_ctl struct
+  with the event_id, index and flags set (index being the partition or PFF
+  number for non-global events). It returns whether the event has
+  occurred, the number of times and any event specific data. The flags
+  can be used to clear the count or enable and disable actions to
+  happen when the event occurs.
+  By using the SWITCHTEC_IOCTL_EVENT_FLAG_EN_POLL flag,
+  you can set an event to trigger a poll command to return with
+  POLLPRI. In this way, userspace can wait for events to occur.
+
+* SWITCHTEC_IOCTL_PFF_TO_PORT and SWITCHTEC_IOCTL_PORT_TO_PFF convert
+  between PCI Function Framework number (used by the event system)
+  and Switchtec Logic Port ID and Partition number (which is more
+  user friendly).
diff --git a/MAINTAINERS b/MAINTAINERS
index 6fed938..c1a9a30 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9515,6 +9515,7 @@ S:Maintained
 F: Documentation/switchtec.txt
 F: Documentation/ABI/testing/sysfs-class-switchtec
 F: drivers/pci/switch/switchtec*
+F: include/uapi/linux/switchtec_ioctl.h
 
 PCI DRIVER FOR NVIDIA TEGRA
 M: Thierry Reding 
diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
index ab08aa4..27ced6f 100644
--- a/drivers/pci/switch/switchtec.c
+++ b/drivers/pci/switch/switchtec.c
@@ -13,6 +13,8 @@
  *
  */
 
+#include 
+
 #include 
 #include 
 #include 
@@ -778,6 +780,431 @@ static unsigned int switchtec_dev_poll(struct file *filp, 
poll_table *wait)
return ret;
 }
 
+static int ioctl_flash_info(struct switchtec_dev *stdev,
+   struct switchtec_ioctl_flash_info __user *uinfo)
+{
+   struct switchtec_ioctl_flash_info info = {0};
+   struct flash_info_regs __iomem *fi = stdev->mmio_flash_info;
+
+   info.flash_length = ioread32(>flash_length);
+   info.num_partitions = SWITCHTEC_IOCTL_NUM_PARTITIONS;
+
+   if (copy_to_user(uinfo, , sizeof(info)))
+   return -EFAULT;
+
+   return 0;
+}
+
+static void set_fw_info_part(struct switchtec_ioctl_flash_part_info *info,
+struct partition_info __iomem *pi)
+{
+   info->address = ioread32(>address);
+   info->length = ioread32(>length);
+}
+
+static int ioctl_flash_part_info(struct switchtec_dev *stdev,
+   struct switchtec_ioctl_flash_part_info __user *uinfo)
+{
+   struct switchtec_ioctl_flash_part_info info = {0};
+   struct flash_info_regs __iomem *fi = stdev->mmio_flash_info;
+   u32 active_addr = -1;
+
+   if (copy_from_user(, uinfo, sizeof(info)))
+   return 

[PATCH v7 2/4] switchtec: Add user interface documentation

[Logan Gunthorpe]

This adds standard documentation for the sysfs switchtec attributes and
a RST formatted text file which documents the char device interface.
Jonathan Corbet has indicated he will move this to a new user-space
developer documentation book once it's created.

Signed-off-by: Logan Gunthorpe 
Signed-off-by: Stephen Bates 
Tested-by: Krishna Dhulipala 
Reviewed-by: Wei Zhang 
Reviewed-by: Jens Axboe 
---
 Documentation/switchtec.txt | 53 +
 MAINTAINERS |  1 +
 2 files changed, 54 insertions(+)
 create mode 100644 Documentation/switchtec.txt

diff --git a/Documentation/switchtec.txt b/Documentation/switchtec.txt
new file mode 100644
index 000..4bced4c
--- /dev/null
+++ b/Documentation/switchtec.txt
@@ -0,0 +1,53 @@
+
+Linux Switchtec Support
+
+
+Microsemi's "Switchtec" line of PCI switch devices is already
+supported by the kernel with standard PCI switch drivers. However, the
+Switchtec device advertises a special management endpoint which
+enables some additional functionality. This includes:
+
+* Packet and Byte Counters
+* Firmware Upgrades
+* Event and Error logs
+* Querying port link status
+* Custom user firmware commands
+
+The switchtec kernel module implements this functionality.
+
+
+Interface
+=
+
+The primary means of communicating with the Switchtec management firmware is
+through the Memory-mapped Remote Procedure Call (MRPC) interface.
+Commands are submitted to the interface with a 4-byte command
+identifier and up to 1KB of command specific data. The firmware will
+respond with a 4 bytes return code and up to 1KB of command specific
+data. The interface only processes a single command at a time.
+
+
+Userspace Interface
+===
+
+The MRPC interface will be exposed to userspace through a simple char
+device: /dev/switchtec#, one for each management endpoint in the system.
+
+The char device has the following semantics:
+
+* A write must consist of at least 4 bytes and no more than 1028 bytes.
+  The first four bytes will be interpreted as the command to run and
+  the remainder will be used as the input data. A write will send the
+  command to the firmware to begin processing.
+
+* Each write must be followed by exactly one read. Any double write will
+  produce an error and any read that doesn't follow a write will
+  produce an error.
+
+* A read will block until the firmware completes the command and return
+  the four bytes of status plus up to 1024 bytes of output data. (The
+  length will be specified by the size parameter of the read call --
+  reading less than 4 bytes will produce an error.
+
+* The poll call will also be supported for userspace applications that
+  need to do other things while waiting for the command to complete.
diff --git a/MAINTAINERS b/MAINTAINERS
index a57686f..aa702b0 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9512,6 +9512,7 @@ M:Stephen Bates 
 M: Logan Gunthorpe 
 L: linux-...@vger.kernel.org
 S: Maintained
+F: Documentation/switchtec.txt
 F: drivers/pci/switch/switchtec*
 
 PCI DRIVER FOR NVIDIA TEGRA
-- 
2.1.4

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v7 0/4] New Microsemi PCI Switch Management Driver

[Logan Gunthorpe]

Because getting it right correctly the first time is appearantly hard
and I caught this mistake while reading the email I had just sent :(

I'm very sorry about the extra noise.

Logan

--

Changes since v6:

* I screwed up the device_unregister path and left a potential use
  after free bug in. I'm not sure why I didn't see a failure because of
  it but it all tested fine. This version is correct though.

Changes since v5:

* Reworked cleanup during unbind again. This time we follow the pattern
  roughly suggested by Jason Gunthorpe: we use an alive flag
  that's checked with a rw semaphore held by the cdev ops. Except
  instead of using the rwsem we just use the already existing
  mrpc_mutex. (Seeing the vast majority of locations that would use the
  semaphore overlap with the existing mutex.)
* Added a small performance enhancement so the code reads less io memory
  if the user is quick in submitting their read request.

Changes since v4:

* Turns out pushing the pci release code into the device release
  function didn't work as I would have liked. If you try to unbind the
  device with an instance open, then you hit a kernel bug at
  drivers/pci/msi.c:371. (This didn't occur in v3.)

  To solve this, we've moved the pci release code back into the
  unregister function and reintroduced an alive flag. This time,
  however, the alive flag is protected by mrpc_mutex and we're very
  careful about what happens to devices still in use (they should
  all be released through the timeout path and an ENODEV error
  returned to userspace; while new commands are blocked with the
  same error).

Changes since v3:

* Removed stdev_is_alive and moved pci deinit functions
  into the device release function which only occurs after all
  cdev instances are closed. (per Bjorn)
* Reduced locking in read/write functions (per Bjorn)
* Use pci_alloc_irq_vectors to vastly improve ISR initialization (Bjorn)
* A few other minor nits and cleanup as noticed by Bjorn

Changes since v2:

* Collected reviewed and tested tags
* Very minor fix to the error path in the create function

Changes since v1:

* Rebased onto 4.10-rc6 (cleanly)
* Split the patch into a few more easily digestible patches (as
  suggested by Greg Kroah-Hartman)
* Folded switchtec.c into switchtec.h (per Greg)
* Fixed a bunch of 32bit build warnings caught by the kbuild test robot
* Fixed some issues in the documentation so it has a proper
  reStructredText format (as noted by Jonathan Corbet)
* Fixed padding and sizes in the IOCTL structures as noticed by Emil
  Velikov and used pahole to verify their consistency across 32 and 64
  bit builds
* Reworked one of the IOCTL interfaces to be more future proof (per
  Emil).

Changes since RFC:

* Fixed incorrect use of the drive model as pointed out by Greg
  Kroah-Hartman
* Used devm functions as suggested by Keith Busch
* Added a handful of sysfs attributes to the switchtec class
* Added a handful of IOCTLs to the switchtec device
* A number of miscellaneous bug fixes

--

Hi,

This is a continuation of the RFC we posted lasted month [1] which
proposes a management driver for Microsemi's Switchtec line of PCI
switches. This hardware is still looking to be used in the Open
Compute Platform

To make this entirely clear: the Switchtec products are compliant
with the PCI specifications and are supported today with the standard
in-kernel driver. However, these devices also expose a management endpoint
on a separate PCI function address which can be used to perform some
advanced operations. This is a driver for that function. See the patch
for more information.

Since the RFC, we've made the changes requested by Greg Kroah-Hartman
and Keith Busch, and we've also fleshed out a number of features. We've
added a couple of IOCTLs and sysfs attributes which are documented in
the patch. Significant work has also been done on the userspace tool
which is available under a GPL license at [2]. We've also had testing
done by some of the interested parties.

We hope to see this work included in either 4.11 or 4.12 assuming a
smooth review process.

The patch is based off of the v4.10-rc6 release.

Thanks for your review,

Logan

[1] https://www.spinics.net/lists/linux-pci/msg56897.html
[2] https://github.com/sbates130272/switchtec-user

Logan Gunthorpe (4):
  MicroSemi Switchtec management interface driver
  switchtec: Add user interface documentation
  switchtec: Add sysfs attributes to the Switchtec driver
  switchtec: Add IOCTLs to the Switchtec driver

 Documentation/ABI/testing/sysfs-class-switchtec |   96 ++
 Documentation/ioctl/ioctl-number.txt|1 +
 Documentation/switchtec.txt |   80 ++
 MAINTAINERS |   11 +
 drivers/pci/Kconfig |1 +
 drivers/pci/Makefile|1 +
 drivers/pci/switch/Kconfig  |   13 +
 drivers/pci/switch/Makefile |1 +
 

[PATCH v7 3/4] switchtec: Add sysfs attributes to the Switchtec driver

[Logan Gunthorpe]

This patch adds a few read-only sysfs attributes which provide
some device information that is exposed from the devices. Primarily
component and device names and versions. These are documented in
Documentation/ABI/testing/sysfs-class-switchtec.

Signed-off-by: Logan Gunthorpe 
Signed-off-by: Stephen Bates 
Tested-by: Krishna Dhulipala 
Reviewed-by: Wei Zhang 
Reviewed-by: Jens Axboe 
Reviewed-by: Greg Kroah-Hartman 
---
 Documentation/ABI/testing/sysfs-class-switchtec |  96 
 MAINTAINERS |   1 +
 drivers/pci/switch/switchtec.c  | 113 
 3 files changed, 210 insertions(+)
 create mode 100644 Documentation/ABI/testing/sysfs-class-switchtec

diff --git a/Documentation/ABI/testing/sysfs-class-switchtec 
b/Documentation/ABI/testing/sysfs-class-switchtec
new file mode 100644
index 000..48cb4c1
--- /dev/null
+++ b/Documentation/ABI/testing/sysfs-class-switchtec
@@ -0,0 +1,96 @@
+switchtec - Microsemi Switchtec PCI Switch Management Endpoint
+
+For details on this subsystem look at Documentation/switchtec.txt.
+
+What:  /sys/class/switchtec
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   The switchtec class subsystem folder.
+   Each registered switchtec driver is represented by a switchtecX
+   subfolder (X being an integer >= 0).
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/component_id
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Component identifier as stored in the hardware (eg. PM8543)
+   (read only)
+Values:arbitrary string.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/component_revision
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Component revision stored in the hardware (read only)
+Values:integer.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/component_vendor
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Component vendor as stored in the hardware (eg. MICROSEM)
+   (read only)
+Values:arbitrary string.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/device_version
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Device version as stored in the hardware (read only)
+Values:integer.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/fw_version
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Currently running firmware version (read only)
+Values:integer (in hexadecimal).
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/partition
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Partition number for this device in the switch (read only)
+Values:integer.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/partition_count
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Total number of partitions in the switch (read only)
+Values:integer.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/product_id
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Product identifier as stored in the hardware (eg. PSX 48XG3)
+   (read only)
+Values:arbitrary string.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/product_revision
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Product revision stored in the hardware (eg. RevB)
+   (read only)
+Values:arbitrary string.
+
+
+What:  /sys/class/switchtec/switchtec[0-9]+/product_vendor
+Date:  05-Jan-2017
+KernelVersion: v4.11
+Contact:   Logan Gunthorpe 
+Description:   Product vendor as stored in the hardware (eg. MICROSEM)
+   (read only)
+Values:arbitrary string.
diff --git a/MAINTAINERS b/MAINTAINERS
index aa702b0..6fed938 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -9513,6 +9513,7 @@ M:Logan Gunthorpe 
 L: linux-...@vger.kernel.org
 S: Maintained
 F: Documentation/switchtec.txt
+F: Documentation/ABI/testing/sysfs-class-switchtec
 F: drivers/pci/switch/switchtec*
 
 PCI DRIVER FOR NVIDIA TEGRA
diff --git a/drivers/pci/switch/switchtec.c b/drivers/pci/switch/switchtec.c
index 084e4ed..4888e23 100644
--- 

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 23:15:49 +0100
Markus Heiser  escreveu:

> Hi Daniel, hi Mauro,
> 
> I have to say, that I lost control over our thread ;)
> 
> > v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> > Implement Markus suggestion for backwards compatability with earlier
> > releases. Laurent reported this, running sphinx 1.3. Solution entirely
> > untested.  
> 
> I have to come back to a more systematic work, this means;
> I have to test this v3 and consider what Mauro and you 
> already posted.

Yes, you're lost :-) The last patch Daniel Sent was v5. From his last patch:


> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
> 
> v5: Use an explicit version check (suggested by Laurent).

https://marc.info/?l=linux-doc=148847115500505=2


Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 6/6] drm/doc: atomic overview, with graph

[Gabriel Krisman Bertazi]

Daniel Vetter  writes:

> I want to split up a few more things and document some details better
> (like how exactly to subclass drm_atomic_state). And maybe also split
> up the helpers a bit per-topic, but this should be a ok-ish start for
> better atomic overview.

Reviewed-by: Gabriel Krisman Bertazi 


-- 
Gabriel Krisman Bertazi
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 11/32] KVM: MIPS: Add VZ capability

[James Hogan]

On Thu, Mar 02, 2017 at 01:20:05PM +0100, Paolo Bonzini wrote:
> On 02/03/2017 12:39, James Hogan wrote:
> > It can't right now, though with relocation of the kernel now implemented
> > in MIPS Linux for KASLR, and hopes for a more generic EVA implementation
> > (which can require the kernel to be linked in a completely different
> > segment) it isn't completely infeasible.
> 
> What about the other way round, sticking a minimal T stub in kernel
> space and running the kernel in userspace?  Would it be feasible or
> would it be as complex as KVM itself?

You mean have a fallback in the guest kernel to keep kernel running from
userspace addresses in kernel mode so it works in VZ guests and
non-virtualized?

Interesting idea. I think it would involve a lot of complexity. It could
forgo some of the emulation of privileged instructions that KVM T
does since its running in kernel mode, but memory management would be
more complex, and invasive changes would be required to the kernel.

- Memory privilege protection is on the granularity of segments, so with
  the traditional segment layout all of USeg (0x..0x7FFF) is
  accessible to user mode, so you'd still need to utilise ASIDs to
  separate the address spaces of actual user programs running in
  0x..0x3FFF from the kernel code running in
  0x4000..0x7FFF.

- USeg is always TLB mapped. That means any kernel code could trigger
  TLB exceptions, which breaks existing assumptions (e.g. normally from
  unmapped kernel segments you can disable interrupts and then
  manipulate the TLB, but that isn't safe if a TLB refill exception
  could happen at any time and clobber the TLB registers). If in the
  future we manage to workaround these issues and map the kernel (for
  security/protection purposes), then it would be easier, but then we'll
  likely already have the capability to fully relocate into a different
  segment.

> > 1) QEMU, which I've implemented using the kvm_type machine callback.
> > This allows the KVM type to be specified with e.g.
> >   "-machine malta,accel=kvm,kvm-type=TE"
> > Otherwise it defaults to using KVM_VM_MIPS_DEFAULT.
> > 
> > When you try and load a kernel (which happens after kvm_init() has
> > already passed the kvm type into KVM_CREATE_VM) it will check that it
> > supports the current kernel type.
> >
> > 2) My kvm test application, which uses KVM_VM_MIPS_DEFAULT by default
> > and hackily maps itself into the guest physical address space to run C
> > code test cases.
> 
> So this one would work for both TE and VZ because the guest is not a
> Linux kernel.

Yes, the test code is position independent and careful to avoid direct
references to any symbols. The GPA mappings are set up the same, but the
virtual addresses (PC, stack pointer etc) are set up slightly
differently depending on whether the VZ capability is present.

> I don't know...  Instinctively I would think that it's easy to get
> KVM_VM_MIPS_DEFAULT wrong and place the VZ-and-fall-back-to-TE policy in
> userspace, but I can be convinced otherwise if the failure mode is good
> enough.

Yeh, I think I agree. It isn't really necessary to have that decision
making in the kernel, and to use a particular KVM type userspace needs
to be aware about it, so it can always figure out from capabilities
which one to use prior to KVM_CREATE_VM.

I suppose the exception is T It shouldn't assume that just because VZ
is available that T isn't (even if that is the case right now). It
could always just try KVM_CREATE_VM with kvm type 0 and detect the error
I suppose, but capabilities are nicer.

Maybe I'll redefine KVM_CAP_MIPS_VZ a bit, such that the value returned
+ 1 is a bitmask of supported kvm types:
has T = !!( (v + 1) & BIT(KVM_VM_MIPS_TE) )
has VZ  = !!( (v + 1) & BIT(KVM_VM_MIPS_VZ) )

That way old kernels which return 0 are consistent, and other
implementations could be added if really necessary without confusing
userland (but fingers crossed it'll never ever be necessary).

> For example, what happens if you use KVM_SET_USER_MEMORY_REGION
> for a kernel address in TE mode?

That deals with physical addresses and user/kernel memory is
distinguished by the virtual address, so the KVM mode (T vs VZ)
doesn't make a difference here.

Cheers
James


signature.asc
Description: Digital signature

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu,  2 Mar 2017 16:40:02 +0100
Daniel Vetter  escreveu:

> From: Markus Heiser 
> 
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
> 
> * DOT (http://www.graphviz.org)
> * SVG
> 
> For image handling use the 'image' replacement::
> 
> .. kernel-image::  svg_image.svg
>:alt:simple SVG image
> 
> For figure handling use the 'figure' replacement::
> 
> .. kernel-figure::  svg_image.svg
>:alt:simple SVG image
> 
>SVG image example
> 
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
> 
>   .. kernel-render:: DOT
>  :alt: foobar digraph
>  :caption: Embedded **DOT** (Graphviz) code.
> 
>  digraph foo {
>   "bar" -> "baz";
>  }
> 
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
> 
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
> 
> v2: s/DOC/DOT/ in a few places (by Daniel).
> 
> v3: Simplify stuff a bit (by Daniel):
> 
> - Remove path detection and setup/check code for that. In
>   Documentation/media/Makefile we already simply use these tools,
>   better to have one consolidated check if we want/need one. Also
>   remove the convertsvg support, we require ImageMagick's convert
>   already in the doc build, no need for a 2nd fallback.
> 
> - Use sphinx for depency tracking, remove hand-rolled version.
> 
> - Forward stderr from dot and convert, otherwise debugging issues with
>   the diagrams is impossible.
> 
> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
> 
> v5: Use an explicit version check (suggested by Laurent).

Found another issue on the patch. The HTML output is pointing to the
wrong place: instead of using a relative patch, it is keeping 
an absolute one.

This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:


Image Format Negotiation on 
Pipelines

High quality and high speed pipeline configuration


There, the "src=" is pointing to the full patch, with doesn't work, as
my html server uses a different patch to find the file. It should,
instead, use a patch relative to the place where the html file is
stored, e. g. in this case, either:
./pipeline.svg
or just:
pipeline.svg

Regards,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

>> Btw, PDF conversion is also not working:
>> 
>> 
>>  File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
>>cmd = [convert_cmd, svg_fname, pdf_fname]
>> 
>>  NameError: name 'convert_cmd' is not defined
>> 
>> And including SVG files for HTML output also seems to be problematic.
> 
> Forgot to mention, but I'm using here Sphinx 1.4.9, installed via
> pip3 (So, python3).

It seems, that Daniel drops some lines from my first RFC, I miss this lines:

+# setup conversion tools and sphinx extension
+# ---
+
+# Graphviz's dot(1) support
+dot_cmd = which('dot')
+
+# ImageMagick' convert(1) support
+convert_cmd = which('convert')

@Daniel: can you take a look at this / Thanks

-- Markus --

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 18:36:31 -0300
Mauro Carvalho Chehab  escreveu:

> > Found another issue on the patch. The HTML output is pointing to the
> > wrong place: instead of using a relative patch, it is keeping 
> > an absolute one.
> > 
> > This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
> > 
> > 
> >  > src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /> > class="caption">Image Format Negotiation on 
> > Pipelines
> > 
> > High quality and high speed pipeline configuration
> > 
> > 
> > There, the "src=" is pointing to the full patch, with doesn't work, as
> > my html server uses a different patch to find the file. It should,
> > instead, use a patch relative to the place where the html file is
> > stored, e. g. in this case, either:
> > ./pipeline.svg
> > or just:
> > pipeline.svg  
> 
> Btw, PDF conversion is also not working:
> 
> 
>   File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
> cmd = [convert_cmd, svg_fname, pdf_fname]
> 
>   NameError: name 'convert_cmd' is not defined
> 
> And including SVG files for HTML output also seems to be problematic.

Forgot to mention, but I'm using here Sphinx 1.4.9, installed via
pip3 (So, python3).

> 
> I'll post the RFCv2 patch that I'm using to test it.

Patch posted. Hopefully, it will help you to see the problems I'm 
facing on my tests.

Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 18:29:39 -0300
Mauro Carvalho Chehab  escreveu:

> Em Thu,  2 Mar 2017 16:40:02 +0100
> Daniel Vetter  escreveu:
> 
> > From: Markus Heiser 
> > 
> > This patch brings scalable figure, image handling and a concept to
> > embed *render* markups:
> > 
> > * DOT (http://www.graphviz.org)
> > * SVG
> > 
> > For image handling use the 'image' replacement::
> > 
> > .. kernel-image::  svg_image.svg
> >:alt:simple SVG image
> > 
> > For figure handling use the 'figure' replacement::
> > 
> > .. kernel-figure::  svg_image.svg
> >:alt:simple SVG image
> > 
> >SVG image example
> > 
> > Embed *render* markups (or languages) like Graphviz's **DOT** is
> > provided by the *render* directive.::
> > 
> >   .. kernel-render:: DOT
> >  :alt: foobar digraph
> >  :caption: Embedded **DOT** (Graphviz) code.
> > 
> >  digraph foo {
> >   "bar" -> "baz";
> >  }
> > 
> > The *render* directive is a concept to integrate *render* markups and
> > languages, yet supported markups:
> > 
> > * DOT: render embedded Graphviz's **DOT**
> > * SVG: render embedded Scalable Vector Graphics (**SVG**)
> > 
> > v2: s/DOC/DOT/ in a few places (by Daniel).
> > 
> > v3: Simplify stuff a bit (by Daniel):
> > 
> > - Remove path detection and setup/check code for that. In
> >   Documentation/media/Makefile we already simply use these tools,
> >   better to have one consolidated check if we want/need one. Also
> >   remove the convertsvg support, we require ImageMagick's convert
> >   already in the doc build, no need for a 2nd fallback.
> > 
> > - Use sphinx for depency tracking, remove hand-rolled version.
> > 
> > - Forward stderr from dot and convert, otherwise debugging issues with
> >   the diagrams is impossible.
> > 
> > v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> > Implement Markus suggestion for backwards compatability with earlier
> > releases. Laurent reported this, running sphinx 1.3. Solution entirely
> > untested.
> > 
> > v5: Use an explicit version check (suggested by Laurent).  
> 
> Found another issue on the patch. The HTML output is pointing to the
> wrong place: instead of using a relative patch, it is keeping 
> an absolute one.
> 
> This is what it produced from Documentation/media/uapi/v4l/dev-subdev.rst:
> 
> 
>  src="/d00/kernel/Documentation/output/media/uapi/v4l/pipeline.svg" /> class="caption">Image Format Negotiation on 
> Pipelines
> 
> High quality and high speed pipeline configuration
> 
> 
> There, the "src=" is pointing to the full patch, with doesn't work, as
> my html server uses a different patch to find the file. It should,
> instead, use a patch relative to the place where the html file is
> stored, e. g. in this case, either:
>   ./pipeline.svg
> or just:
>   pipeline.svg

Btw, PDF conversion is also not working:


  File "/d00/kernel/Documentation/sphinx/kfigure.py", line 241, in svg2pdf
cmd = [convert_cmd, svg_fname, pdf_fname]

NameError: name 'convert_cmd' is not defined

And including SVG files for HTML output also seems to be problematic.

I'll post the RFCv2 patch that I'm using to test it.

Regards,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

> Am 02.03.2017 um 20:34 schrieb Mauro Carvalho Chehab 
> :
> 
> Em Thu, 2 Mar 2017 20:06:39 +0100
> Markus Heiser  escreveu:
> 
>> Hi Mauro,
>> 
>>> Tested here with the enclosed patch.  
>> 
>> great, big step forward making /media/Makefile smaller ...  thanks a lot
>> 
>>> It crashed:
>>> Exception occurred:
>>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
>>> dot2format
>>>   sys.stderr.write(err)
>>> TypeError: write() argument must be str, not bytes
>>> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you 
>>> want to report the issue to the developers.
>>> Please also report this if it was a user error, so that a better error 
>>> message can be provided next time.
>>> A bug report can be filed in the tracker at 
>>> . Thanks!
>>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
>>> make[1]: *** [htmldocs] Error 1
>>> Makefile:1450: recipe for target 'htmldocs' failed
>>> make: *** [htmldocs] Error 2
>>> 
>>> Weird enough, it produced a 
>>> Documentation/output/media/uapi/v4l/pipeline.svg file.  
>> 
>> I guess that the dot command writes something to stderr. This is captured 
>> by the extension and printed to stderr ...
>> 
>> +def dot2format(dot_fname, out_fname):
>> ...
>> +exit_code = 42
>> +with open(out_fname, "w") as out:
>> +p = subprocess.Popen(
>> +cmd, stdout = out, stderr = subprocess.PIPE )
>> +nil, err = p.communicate()
>> +
>> +sys.stderr.write(err)
>> +
>> +exit_code = p.returncode
>> +out.flush()
>> +return bool(exit_code == 0)
>> 
>>> File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
>>> dot2format
>>>   sys.stderr.write(err)
>>> TypeError: write() argument must be str, not bytes  
>> 
>> Do we need this stderr output? For a first test, uncomment the 
>> "sys.stderr.write(err)“ in line 222. Or, if we really need the
>> stderr, try:
>> 
>> -sys.stderr.write(err)
>> +sys.stderr.write(str(err))
> 
> Yes, this fixed. I actually did:
> 
> -sys.stderr.write(err)
> +sys.stderr.write(str(err))
> +sys.stderr.write("\n")
> 
> It is now printing:
>   b''
> 
> I added the \n print to avoid it to be mixed with the "writing output"
> prints.
> No idea how to make sense from it - but clearly, the error report
> logic require some care ;-)


Aargh, I’am a idiot ... I guess 'sys.stderr.write(err)‘ is a artefact
of my development, simply drop it and the subprocess.PIPE of stderr
also.

+with open(out_fname, "w") as out:
+p = subprocess.Popen(
-cmd, stdout = out, stderr = subprocess.PIPE )
+cmd, stdout = out)
+nil, err = p.communicate()
-
-sys.stderr.write(err)
-
+exit_code = p.returncode
+out.flush()
+return bool(exit_code == 0)

I can’t test it ATM, but without redirect stderr, the stderr
of the parent process is inherited.

  https://docs.python.org/3.6/library/subprocess.html#popen-constructor

The Popen.communicate() always returns a tuple (stdout_data, stderr_data)
with above the tuple is always (None, None).

  
https://docs.python.org/3.6/library/subprocess.html#subprocess.Popen.communicate

  """to get anything other than None in the result tuple,
 you need to give stdout=PIPE and/or stderr=PIPE too."""

Sorry, that I made all this mistakes, but „here“ I have only mail
and web, no dev-env and I miss my emacs  ;) 

If the suggestion above does not work, I have to investigate
more time next weekend.

-- Markus --


> 
> Thanks,
> Mauro

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

> Am 02.03.2017 um 20:09 schrieb Mauro Carvalho Chehab 
> :
...

>> 
>> Yes, its only about "docutils>=0.11" in Sphinx 1.3 dependencies. 
>> In Sphinx 1.5 the error:
>> 
>>  https://github.com/sphinx-doc/sphinx/issues/3212
>> 
>> is fixed:
>> 
>>  
>> https://github.com/tk0miya/sphinx/commit/73663f63672f22304810ce6bb9787490ad250127
>> 
>> But this will never be fixed downwards.
> 
> Crap. This kind of patch should be backported to Sphinx 1.3/1.4,
> or Python's PIP repository should be fixed to require docutils
> version to be either 0.11 or 0.12, if one installs version 1.3
> or 1.4 of Sphinx.
> 
> The way it is, PIP repository is broken!

I leaved a comment at sphinx-doc project:

  https://github.com/sphinx-doc/sphinx/issues/3212#issuecomment-283756374

>> All this is about semantic versioning. If you want to promise your
>> builds, you have to name which versions of dependencies you support.
>> I guess this is nothing new for kernel developers ;)
> 
> No. At the Kernel, we do everything possible to not break APIs.

Sorry I was not precise. I was talking about third tools dependencies
and that this is well handled by the kernel ;)
 
> If this were not the case, you wouldn't be able to run Sphinx 1.5 
> with legacy Kernel versions ;-)

This is what I mean ;)

> 
>> The problem is, that PEP440 defines not only ONE version scheme
>> 
>> """Some hard to read version identifiers are permitted by
>>this scheme in order to better accommodate the wide range
>>of versioning practices across existing public and private
>>Python projects."""
>> 
>> In practice, the python projects use slightly different schemes
>> which not follow one rule like .. 
>> From history packaging in Python is the hell, it becomes better, but
>> the problem with slightly different version schemes still exist.
> 
> IMHO, the way Python and python libraries break compatibility is crazy.
> 
> Good packaging sense says that, if APIs can break on every single new
> release (with seems to be the case of docutils), then a package
> depending on such bad-developed library should require the exact
> version(s) it is known to work.

I can only repeat myself, the main problem is, that PEP440 allows
multiple version schemes. Instead of one scheme 

 ..

Every project use a slightly different scheme and others
do not care about any scheme.

> When we're discussing about the docs toolchain, I mentioned that I
> was afraid that the Python development model would cause this sort
> of issues. Unfortunately, it seems that my concerns were pertinent :-(

Not really ;) .. you are tend to mix at least three parts

1. Python
2. Python packaging
3. Sphinx developers who do not stick there depencies

But you are right, when you say that in all parts some confusion
prevail. E.g.

1. Python 2 to 3 movement has been done reckless
--> in the meantime we have https://pythonhosted.org/six/

2. Python packaging is a mess (setup-tools, distutils, pip, easy_install ..) 
--> in the meantime we have PyPA who brings us more structure 
(https://www.pypa.io/en/latest)

3. Developer using Python
--> we all have a learn curve and making errors all days long. 
But this should not stop us from continue :)


>> I guess this is something we should discuss with Jon, he
>> is also familiar with it virtualenv. 
> 
> Yeah, making kernel build to depend on network can be a problem.
> 
> Maybe one way would be to have a sort of "prepare" script that
> would be network-dependent, and will install whatever needed to
> build the docs. If called, make *docs will use the virtualenv.
> Otherwise, it would print a warning message saying that the doc
> build is not reliable, but would try to use whatever installed
> on the machine.

Could be a workaround. May Jon has continuative ideas, nothing
we have to solve today ... give Jon some time.
 
-- Markus 
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 17:04:01 -0300
Mauro Carvalho Chehab  escreveu:

> Em Thu, 2 Mar 2017 20:52:59 +0100
> Daniel Vetter  escreveu:
> 
> > On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:  
> > > Hi Mauro,
> > > 
> > > > Tested here with the enclosed patch.
> > > 
> > > great, big step forward making /media/Makefile smaller ...  thanks a 
> > > lot
> > > 
> > > > It crashed:
> > > > Exception occurred:
> > > >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, 
> > > > in dot2format
> > > >sys.stderr.write(err)
> > > > TypeError: write() argument must be str, not bytes
> > > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if 
> > > > you want to report the issue to the developers.
> > > > Please also report this if it was a user error, so that a better error 
> > > > message can be provided next time.
> > > > A bug report can be filed in the tracker at 
> > > > . Thanks!
> > > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > > make[1]: *** [htmldocs] Error 1
> > > > Makefile:1450: recipe for target 'htmldocs' failed
> > > > make: *** [htmldocs] Error 2
> > > > 
> > > > Weird enough, it produced a 
> > > > Documentation/output/media/uapi/v4l/pipeline.svg file.
> > > 
> > > I guess that the dot command writes something to stderr. This is captured 
> > > by the extension and printed to stderr ...
> > > 
> > > +def dot2format(dot_fname, out_fname):
> > > ...
> > > +exit_code = 42
> > > +with open(out_fname, "w") as out:
> > > +p = subprocess.Popen(
> > > +cmd, stdout = out, stderr = subprocess.PIPE )
> > > +nil, err = p.communicate()
> > > +
> > > +sys.stderr.write(err)
> > > +
> > > +exit_code = p.returncode
> > > +out.flush()
> > > +return bool(exit_code == 0)
> > > 
> > > >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, 
> > > > in dot2format
> > > >sys.stderr.write(err)
> > > > TypeError: write() argument must be str, not bytes
> > > 
> > > Do we need this stderr output? For a first test, uncomment the 
> > > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > > stderr, try:
> > > 
> > > -sys.stderr.write(err)
> > > +sys.stderr.write(str(err))
> > > 
> > > I this fixes, there is another "sys.stderr.write(err)" in 
> > > func svg2pdf(..) which should also fixed ….
> > >  
> > > +def svg2pdf(svg_fname, pdf_fname):
> > > ...
> > > +cmd = [convert_cmd, svg_fname, pdf_fname]
> > > +p = subprocess.Popen(
> > > +cmd, stdout = out, stderr = subprocess.PIPE )
> > > +nil, err = p.communicate()
> > > +
> > > -sys.stderr.write(err)
> > > +sys.stderr.write(str(err))
> > > +
> > > +exit_code = p.returncode
> > > +return bool(exit_code == 0)
> > 
> > Yes, I very much want stderr to be forward.  
> 
> Yes, error report is required.
> 
> > Without that you don't see
> > error output from dot or convert, and that makes it impossible to debug
> > anything. If I want a direct forwarding of the bytes, how should I do this
> > in python? Capturing stderr and then re-dumping it is kinda silly ...  
> 
> Markus or some other Python programmer may help us with that.
> 
> > Note that I copied this pattern from the kernel-doc extension, seems to
> > have worked there.  
> 
> Maybe it is broken there too then, or this is another python API
> that changed over time. Here, I'm testing with:
>   python3-3.5.2-4.fc25.x86_64
> 
> From here:
>   https://docs.python.org/2/library/subprocess.html
> 
> communicate returns a tuple.
> 
> I used repr(p.communicate()[0], on the code snippet I sent, 
> as I copied from an example that I found at:
>   
> https://stackoverflow.com/questions/33295284/python-subprocess-popen-write-to-stderr
> 
> Thanks,
> Mauro

I suspect that the actual fixup would be something like:


commit ddf93ae81af10bb43caa651b9acd355f1d74cebe
Author: Mauro Carvalho Chehab 
Date:   Thu Mar 2 17:11:47 2017 -0300

kfigure.py fixups

Signed-off-by: Mauro Carvalho Chehab 

diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..19389cb34d6f 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,13 @@ def dot2format(dot_fname, out_fname):
 with open(out_fname, "w") as out:
 p = subprocess.Popen(
 cmd, stdout = out, stderr = subprocess.PIPE )
-nil, err = p.communicate()
-
-sys.stderr.write(err)
+err = p.communicate()
 
 exit_code = p.returncode
+
+if exit_code == 0:
+sys.stderr.write("Error:" + repr(err[0]) + "\n")
+
 out.flush()
 return bool(exit_code == 0)
 
@@ -239,11 +241,13 @@ def svg2pdf(svg_fname, pdf_fname):
  

[PATCH RFC] docs-rst: Don't use explicit Makefile rules to build SVG and DOT files

[Mauro Carvalho Chehab]

Now that we have an extension to handle images, use it.

Signed-off-by: Mauro Carvalho Chehab 
---

This patch is based on Daniel Vetter & Markus Heiser's work to support
PNG and SVG via kpicture Sphinx extension.

PS.: With this RFC patch, I'm getting now some extra warnings:

/devel/v4l/patchwork/Documentation/media/intro.rst:12: WARNING: undefined 
label: typical_media_device (if the link has no caption the label must precede 
a section header)
/devel/v4l/patchwork/Documentation/media/uapi/dvb/intro.rst:95: WARNING: 
undefined label: stb_components (if the link has no caption the label must 
precede a section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/crop.rst:65: WARNING: 
undefined label: vbi-hsync (if the link has no caption the label must precede a 
section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/dev-raw-vbi.rst:118: WARNING: 
undefined label: vbi-hsync (if the link has no caption the label must precede a 
section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/dev-raw-vbi.rst:138: WARNING: 
undefined label: vbi-525 (if the link has no caption the label must precede a 
section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/dev-raw-vbi.rst:138: WARNING: 
undefined label: vbi-625 (if the link has no caption the label must precede a 
section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/dev-raw-vbi.rst:298: WARNING: 
undefined label: vbi-525 (if the link has no caption the label must precede a 
section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/dev-raw-vbi.rst:298: WARNING: 
undefined label: vbi-625 (if the link has no caption the label must precede a 
section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/dev-subdev.rst:93: WARNING: 
undefined label: pipeline-scaling (if the link has no caption the label must 
precede a section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/dev-subdev.rst:199: WARNING: 
undefined label: pipeline-scaling (if the link has no caption the label must 
precede a section header)
/devel/v4l/patchwork/Documentation/media/uapi/v4l/subdev-formats.rst:1483: 
WARNING: undefined label: bayer-patterns (if the link has no caption the label 
must precede a section header)



 Documentation/media/Makefile   | 47 +-
 Documentation/media/intro.rst  |  9 ++---
 Documentation/media/uapi/dvb/intro.rst |  9 ++---
 Documentation/media/uapi/v4l/crop.rst  |  9 ++---
 Documentation/media/uapi/v4l/dev-raw-vbi.rst   | 25 +---
 Documentation/media/uapi/v4l/dev-subdev.rst| 34 +++-
 Documentation/media/uapi/v4l/field-order.rst   | 14 ---
 Documentation/media/uapi/v4l/pixfmt-nv12mt.rst | 18 -
 Documentation/media/uapi/v4l/selection-api-003.rst |  7 ++--
 Documentation/media/uapi/v4l/subdev-formats.rst|  9 ++---
 10 files changed, 61 insertions(+), 120 deletions(-)

diff --git a/Documentation/media/Makefile b/Documentation/media/Makefile
index 32663602ff25..5bd52ea1eff0 100644
--- a/Documentation/media/Makefile
+++ b/Documentation/media/Makefile
@@ -1,51 +1,6 @@
-# Rules to convert DOT and SVG to Sphinx images
-
-SRC_DIR=$(srctree)/Documentation/media
-
-DOTS = \
-   uapi/v4l/pipeline.dot \
-
-IMAGES = \
-   typical_media_device.svg \
-   uapi/dvb/dvbstb.svg \
-   uapi/v4l/bayer.svg \
-   uapi/v4l/constraints.svg \
-   uapi/v4l/crop.svg \
-   uapi/v4l/fieldseq_bt.svg \
-   uapi/v4l/fieldseq_tb.svg \
-   uapi/v4l/nv12mt.svg \
-   uapi/v4l/nv12mt_example.svg \
-   uapi/v4l/pipeline.svg \
-   uapi/v4l/selection.svg \
-   uapi/v4l/subdev-image-processing-full.svg \
-   uapi/v4l/subdev-image-processing-scaling-multi-source.svg \
-   uapi/v4l/subdev-image-processing-crop.svg \
-   uapi/v4l/vbi_525.svg \
-   uapi/v4l/vbi_625.svg \
-   uapi/v4l/vbi_hsync.svg \
-
-DOTTGT := $(patsubst %.dot,%.svg,$(DOTS))
-IMGDOT := $(patsubst %,$(SRC_DIR)/%,$(DOTTGT))
-
-IMGTGT := $(patsubst %.svg,%.pdf,$(IMAGES))
-IMGPDF := $(patsubst %,$(SRC_DIR)/%,$(IMGTGT))
-
-cmd = $(echo-cmd) $(cmd_$(1))
-
-quiet_cmd_genpdf = GENPDF  $2
-  cmd_genpdf = convert $2 $3
-
-quiet_cmd_gendot = DOT $2
-  cmd_gendot = dot -Tsvg $2 > $3
-
-%.pdf: %.svg
-   @$(call cmd,genpdf,$<,$@)
-
-%.svg: %.dot
-   @$(call cmd,gendot,$<,$@)
-
 # Rules to convert a .h file to inline RST documentation
 
+SRC_DIR=$(srctree)/Documentation/media
 PARSER = $(srctree)/Documentation/sphinx/parse-headers.pl
 UAPI = $(srctree)/include/uapi/linux
 KAPI = $(srctree)/include/linux
diff --git a/Documentation/media/intro.rst b/Documentation/media/intro.rst
index 8f7490c9a8ef..3c0c218c6d08 100644
--- a/Documentation/media/intro.rst
+++ b/Documentation/media/intro.rst
@@ -13,11 +13,10 @@ A typical media device hardware is shown at 
:ref:`typical_media_device`.
 
 .. _typical_media_device:
 
-.. figure::  

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 20:06:39 +0100
Markus Heiser  escreveu:

> Hi Mauro,
> 
> > Tested here with the enclosed patch.  
> 
> great, big step forward making /media/Makefile smaller ...  thanks a lot
> 
> > It crashed:
> > Exception occurred:
> >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
> > dot2format
> >sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
> > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you 
> > want to report the issue to the developers.
> > Please also report this if it was a user error, so that a better error 
> > message can be provided next time.
> > A bug report can be filed in the tracker at 
> > . Thanks!
> > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > make[1]: *** [htmldocs] Error 1
> > Makefile:1450: recipe for target 'htmldocs' failed
> > make: *** [htmldocs] Error 2
> > 
> > Weird enough, it produced a 
> > Documentation/output/media/uapi/v4l/pipeline.svg file.  
> 
> I guess that the dot command writes something to stderr. This is captured 
> by the extension and printed to stderr ...
> 
> +def dot2format(dot_fname, out_fname):
> ...
> +exit_code = 42
> +with open(out_fname, "w") as out:
> +p = subprocess.Popen(
> +cmd, stdout = out, stderr = subprocess.PIPE )
> +nil, err = p.communicate()
> +
> +sys.stderr.write(err)
> +
> +exit_code = p.returncode
> +out.flush()
> +return bool(exit_code == 0)
> 
> >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
> > dot2format
> >sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes  
> 
> Do we need this stderr output? For a first test, uncomment the 
> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> stderr, try:
> 
> -sys.stderr.write(err)
> +sys.stderr.write(str(err))

Yes, this fixed. I actually did:

-sys.stderr.write(err)
+sys.stderr.write(str(err))
+sys.stderr.write("\n")

It is now printing:
b''

I added the \n print to avoid it to be mixed with the "writing output"
prints.

No idea how to make sense from it - but clearly, the error report
logic require some care ;-)

Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 21:16:05 +0100
Markus Heiser  escreveu:

> > Am 02.03.2017 um 20:09 schrieb Mauro Carvalho Chehab 
> > :  

> I leaved a comment at sphinx-doc project:
> 
>   https://github.com/sphinx-doc/sphinx/issues/3212#issuecomment-283756374

Thanks!

> > Maybe one way would be to have a sort of "prepare" script that
> > would be network-dependent, and will install whatever needed to
> > build the docs. If called, make *docs will use the virtualenv.
> > Otherwise, it would print a warning message saying that the doc
> > build is not reliable, but would try to use whatever installed
> > on the machine.  
> 
> Could be a workaround. May Jon has continuative ideas, nothing
> we have to solve today ... give Jon some time.

Sure.

Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 16:34:22 -0300
Mauro Carvalho Chehab  escreveu:

> Em Thu, 2 Mar 2017 20:06:39 +0100
> Markus Heiser  escreveu:
> 
> > Hi Mauro,
> > 
> > > Tested here with the enclosed patch.  
> > 
> > great, big step forward making /media/Makefile smaller ...  thanks a lot
> > 
> > > It crashed:
> > > Exception occurred:
> > >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, 
> > > in dot2format
> > >sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you 
> > > want to report the issue to the developers.
> > > Please also report this if it was a user error, so that a better error 
> > > message can be provided next time.
> > > A bug report can be filed in the tracker at 
> > > . Thanks!
> > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > make[1]: *** [htmldocs] Error 1
> > > Makefile:1450: recipe for target 'htmldocs' failed
> > > make: *** [htmldocs] Error 2
> > > 
> > > Weird enough, it produced a 
> > > Documentation/output/media/uapi/v4l/pipeline.svg file.  
> > 
> > I guess that the dot command writes something to stderr. This is captured 
> > by the extension and printed to stderr ...
> > 
> > +def dot2format(dot_fname, out_fname):
> > ...
> > +exit_code = 42
> > +with open(out_fname, "w") as out:
> > +p = subprocess.Popen(
> > +cmd, stdout = out, stderr = subprocess.PIPE )
> > +nil, err = p.communicate()
> > +
> > +sys.stderr.write(err)
> > +
> > +exit_code = p.returncode
> > +out.flush()
> > +return bool(exit_code == 0)
> > 
> > >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, 
> > > in dot2format
> > >sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes  
> > 
> > Do we need this stderr output? For a first test, uncomment the 
> > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > stderr, try:
> > 
> > -sys.stderr.write(err)
> > +sys.stderr.write(str(err))
> 
> Yes, this fixed. I actually did:
> 
> -sys.stderr.write(err)
> +sys.stderr.write(str(err))
> +sys.stderr.write("\n")
> 
> It is now printing:
>   b''
> 
> I added the \n print to avoid it to be mixed with the "writing output"
> prints.
> 
> No idea how to make sense from it - but clearly, the error report
> logic require some care ;-)

I'm not a Python programmer, but googling about the right syntax for
p.communicate(), I suspect that the fix should be similar to this code
snippet below.

Without the if, this code:

sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")

prints:
Error:None

So, I guess the code below is ok.

Markus, please check. 

Daniel, Feel free to merge it with the original patch if OK.

diff --git a/Documentation/sphinx/kfigure.py b/Documentation/sphinx/kfigure.py
index 32eab0f4cfba..b154c5f17752 100644
--- a/Documentation/sphinx/kfigure.py
+++ b/Documentation/sphinx/kfigure.py
@@ -217,11 +217,12 @@ def dot2format(dot_fname, out_fname):
 with open(out_fname, "w") as out:
 p = subprocess.Popen(
 cmd, stdout = out, stderr = subprocess.PIPE )
-nil, err = p.communicate()
-
-sys.stderr.write(err)
 
 exit_code = p.returncode
+
+if exit_code == 0:
+sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")
+
 out.flush()
 return bool(exit_code == 0)
 
@@ -239,11 +240,12 @@ def svg2pdf(svg_fname, pdf_fname):
 cmd = [convert_cmd, svg_fname, pdf_fname]
 p = subprocess.Popen(
 cmd, stdout = out, stderr = subprocess.PIPE )
-nil, err = p.communicate()
-
-sys.stderr.write(err)
 
 exit_code = p.returncode
+
+if exit_code == 0:
+sys.stderr.write("Error:" + repr(p.communicate()[0]) + "\n")
+
 return bool(exit_code == 0)
 
 

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Daniel Vetter]

On Thu, Mar 02, 2017 at 01:01:36PM -0300, Mauro Carvalho Chehab wrote:
> Em Thu, 2 Mar 2017 16:53:04 +0100
> Daniel Vetter  escreveu:
> 
> > On Thu, Mar 2, 2017 at 4:22 PM, Jonathan Corbet  wrote:
> > > On Thu, 2 Mar 2017 16:11:08 +0100
> > > Daniel Vetter  wrote:
> > >  
> > >> I'll give it a shot at implementing it, but I can't (easily at least) 
> > >> test
> > >> on sphinx 1.3.  
> > >
> > > Virtualenv makes that kind of thing pretty easy to do.  Maybe we should
> > > add a script to create and populate a suitable venv for this kind of
> > > thing.  
> > 
> > Laurent quickly checked v5 of this patch, looks all good now.
> > 
> > > Meanwhile, I've been seeing only parts 1 and 2 of what's clearly a bigger
> > > series.  Do you want me to carry just these two?  
> > 
> > I submitted the entire series to both linux-doc and dri-devel. 
> 
> Sure you sent to linux-doc? I'm not seeing it there.
> 
> I'm not seeing it at the mirrors neither:
>   https://marc.info/?l=linux-doc=1=201703=2
>   https://www.spinics.net/lists/linux-doc/maillist.html

First one I spotted:

https://marc.info/?l=linux-doc=148848308304746=2

So maybe just mail server hiccup that's recovering now?
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC PATCH v4 27/28] x86: Add support to encrypt the kernel in-place

[Tom Lendacky]

On 3/1/2017 11:36 AM, Borislav Petkov wrote:

On Thu, Feb 16, 2017 at 09:48:08AM -0600, Tom Lendacky wrote:

This patch adds the support to encrypt the kernel in-place. This is
done by creating new page mappings for the kernel - a decrypted
write-protected mapping and an encrypted mapping. The kernel is encyrpted


s/encyrpted/encrypted/


by copying the kernel through a temporary buffer.


"... by copying it... "


Ok.





Signed-off-by: Tom Lendacky 
---


...


+ENTRY(sme_encrypt_execute)
+
+#ifdef CONFIG_AMD_MEM_ENCRYPT
+   /*
+* Entry parameters:
+*   RDI - virtual address for the encrypted kernel mapping
+*   RSI - virtual address for the decrypted kernel mapping
+*   RDX - length of kernel
+*   RCX - address of the encryption workarea


 , including:


Ok.




+* - stack page (PAGE_SIZE)
+* - encryption routine page (PAGE_SIZE)
+* - intermediate copy buffer (PMD_PAGE_SIZE)
+*R8 - address of the pagetables to use for encryption
+*/
+
+   /* Set up a one page stack in the non-encrypted memory area */
+   movq%rcx, %rax
+   addq$PAGE_SIZE, %rax
+   movq%rsp, %rbp


%rbp is callee-saved and you're overwriting it here. You need to push it
first.


Yup, I'll re-work the entry code based on this comment and the one
below.




+   movq%rax, %rsp
+   push%rbp
+
+   push%r12
+   push%r13


In general, just do all pushes on function entry and the pops on exit,
like the compiler does.


+   movq%rdi, %r10
+   movq%rsi, %r11
+   movq%rdx, %r12
+   movq%rcx, %r13
+
+   /* Copy encryption routine into the workarea */
+   movq%rax, %rdi
+   leaq.Lencrypt_start(%rip), %rsi
+   movq$(.Lencrypt_stop - .Lencrypt_start), %rcx
+   rep movsb
+
+   /* Setup registers for call */
+   movq%r10, %rdi
+   movq%r11, %rsi
+   movq%r8, %rdx
+   movq%r12, %rcx
+   movq%rax, %r8
+   addq$PAGE_SIZE, %r8
+
+   /* Call the encryption routine */
+   call*%rax
+
+   pop %r13
+   pop %r12
+
+   pop %rsp/* Restore original stack pointer */
+.Lencrypt_exit:


Please put side comments like this here:


Ok, can do.



ENTRY(sme_encrypt_execute)

#ifdef CONFIG_AMD_MEM_ENCRYPT
/*
 * Entry parameters:
 *   RDI - virtual address for the encrypted kernel mapping
 *   RSI - virtual address for the decrypted kernel mapping
 *   RDX - length of kernel
 *   RCX - address of the encryption workarea
 * - stack page (PAGE_SIZE)
 * - encryption routine page (PAGE_SIZE)
 * - intermediate copy buffer (PMD_PAGE_SIZE)
 *R8 - address of the pagetables to use for encryption
 */

/* Set up a one page stack in the non-encrypted memory area */
movq%rcx, %rax  # %rax = workarea
addq$PAGE_SIZE, %rax# %rax += 4096
movq%rsp, %rbp  # stash stack ptr
movq%rax, %rsp  # set new stack
push%rbp# needs to happen before the 
mov %rsp, %rbp

push%r12
push%r13

movq%rdi, %r10  # encrypted kernel
movq%rsi, %r11  # decrypted kernel
movq%rdx, %r12  # kernel length
movq%rcx, %r13  # workarea
...

and so on.

...


diff --git a/arch/x86/kernel/mem_encrypt_init.c 
b/arch/x86/kernel/mem_encrypt_init.c
index 25af15d..07cbb90 100644
--- a/arch/x86/kernel/mem_encrypt_init.c
+++ b/arch/x86/kernel/mem_encrypt_init.c
@@ -16,9 +16,200 @@
 #ifdef CONFIG_AMD_MEM_ENCRYPT

 #include 
+#include 
+
+#include 
+
+extern void sme_encrypt_execute(unsigned long, unsigned long, unsigned long,
+   void *, pgd_t *);


This belongs into mem_encrypt.h. And I think it already came up: please
use names for those params.


Yup, will move it.




+
+#define PGD_FLAGS  _KERNPG_TABLE_NOENC
+#define PUD_FLAGS  _KERNPG_TABLE_NOENC
+#define PMD_FLAGS  __PAGE_KERNEL_LARGE_EXEC
+
+static void __init *sme_pgtable_entry(pgd_t *pgd, void *next_page,
+ void *vaddr, pmdval_t pmd_val)
+{


sme_populate() or so sounds better.


Ok.




+   pud_t *pud;
+   pmd_t *pmd;
+
+   pgd += pgd_index((unsigned long)vaddr);
+   if (pgd_none(*pgd)) {
+   pud = next_page;
+   memset(pud, 0, sizeof(*pud) * PTRS_PER_PUD);
+   native_set_pgd(pgd,
+  native_make_pgd((unsigned long)pud + PGD_FLAGS));


Let it stick out, no need for those "stairs" in the 

Re: [RFC PATCH v4 27/28] x86: Add support to encrypt the kernel in-place

[Borislav Petkov]

On Thu, Mar 02, 2017 at 12:30:31PM -0600, Tom Lendacky wrote:
> The "* 2" here and above is that a PUD and a PMD is needed for both
> the encrypted and decrypted mappings. I'll add a comment to clarify
> that.

Ah, makes sense. Definitely needs a comment.

> Yup, I can do that here too (but need PGDIR_SIZE).

Right, I did test and wanted to write PGDIR_SIZE but then ... I guess
something distracted me :-)

> So next_page is the first free page within the workarea in which a
> pagetable entry (PGD, PUD or PMD) can be created when we are populating
> the new mappings or adding the workarea to the current mapping.  Any
> new pagetable structures that are created will use this value.

Ok, so I guess this needs an overview comment with maybe some ascii
showing how workarea, exec_size, full_size and all those other things
play together.

> Ok, I'll work on the comment.  Something along the line of:
>
> /*
>  * The encrypted mapping of the kernel will use identity mapped
>  * virtual addresses.  A different PGD index/entry must be used to
>  * get different pagetable entries for the decrypted mapping.
>  * Choose the next PGD index and convert it to a virtual address
>  * to be used as the base of the mapping.

Better.

> Except the workarea size includes both the encryption execution
> size and the pagetable structure size.  I'll work on this to try
> and clarify it better.

That's a useful piece of info, yap, the big picture could use some more
explanation.

> Most definitely.  I appreciate the feedback since I'm very close to
> the code and have an understanding of what I'm doing. I'd like to be
> sure that everyone can easily understand what is happening.

Nice!

Thanks.

-- 
Regards/Gruss,
Boris.

Good mailing practices for 400: avoid top-posting and trim the reply.
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 20:52:59 +0100
Daniel Vetter  escreveu:

> On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:
> > Hi Mauro,
> >   
> > > Tested here with the enclosed patch.  
> > 
> > great, big step forward making /media/Makefile smaller ...  thanks a lot
> >   
> > > It crashed:
> > > Exception occurred:
> > >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, 
> > > in dot2format
> > >sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes
> > > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you 
> > > want to report the issue to the developers.
> > > Please also report this if it was a user error, so that a better error 
> > > message can be provided next time.
> > > A bug report can be filed in the tracker at 
> > > . Thanks!
> > > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > > make[1]: *** [htmldocs] Error 1
> > > Makefile:1450: recipe for target 'htmldocs' failed
> > > make: *** [htmldocs] Error 2
> > > 
> > > Weird enough, it produced a 
> > > Documentation/output/media/uapi/v4l/pipeline.svg file.  
> > 
> > I guess that the dot command writes something to stderr. This is captured 
> > by the extension and printed to stderr ...
> > 
> > +def dot2format(dot_fname, out_fname):
> > ...
> > +exit_code = 42
> > +with open(out_fname, "w") as out:
> > +p = subprocess.Popen(
> > +cmd, stdout = out, stderr = subprocess.PIPE )
> > +nil, err = p.communicate()
> > +
> > +sys.stderr.write(err)
> > +
> > +exit_code = p.returncode
> > +out.flush()
> > +return bool(exit_code == 0)
> >   
> > >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, 
> > > in dot2format
> > >sys.stderr.write(err)
> > > TypeError: write() argument must be str, not bytes  
> > 
> > Do we need this stderr output? For a first test, uncomment the 
> > "sys.stderr.write(err)“ in line 222. Or, if we really need the
> > stderr, try:
> > 
> > -sys.stderr.write(err)
> > +sys.stderr.write(str(err))
> > 
> > I this fixes, there is another "sys.stderr.write(err)" in 
> > func svg2pdf(..) which should also fixed ….
> >  
> > +def svg2pdf(svg_fname, pdf_fname):
> > ...
> > +cmd = [convert_cmd, svg_fname, pdf_fname]
> > +p = subprocess.Popen(
> > +cmd, stdout = out, stderr = subprocess.PIPE )
> > +nil, err = p.communicate()
> > +
> > -sys.stderr.write(err)
> > +sys.stderr.write(str(err))
> > +
> > +exit_code = p.returncode
> > +return bool(exit_code == 0)  
> 
> Yes, I very much want stderr to be forward.

Yes, error report is required.

> Without that you don't see
> error output from dot or convert, and that makes it impossible to debug
> anything. If I want a direct forwarding of the bytes, how should I do this
> in python? Capturing stderr and then re-dumping it is kinda silly ...

Markus or some other Python programmer may help us with that.

> Note that I copied this pattern from the kernel-doc extension, seems to
> have worked there.

Maybe it is broken there too then, or this is another python API
that changed over time. Here, I'm testing with:
python3-3.5.2-4.fc25.x86_64

>From here:
https://docs.python.org/2/library/subprocess.html

communicate returns a tuple.

I used repr(p.communicate()[0], on the code snippet I sent, 
as I copied from an example that I found at:

https://stackoverflow.com/questions/33295284/python-subprocess-popen-write-to-stderr

Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Daniel Vetter]

On Thu, Mar 02, 2017 at 08:06:39PM +0100, Markus Heiser wrote:
> Hi Mauro,
> 
> > Tested here with the enclosed patch.
> 
> great, big step forward making /media/Makefile smaller ...  thanks a lot
> 
> > It crashed:
> > Exception occurred:
> >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
> > dot2format
> >sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
> > The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you 
> > want to report the issue to the developers.
> > Please also report this if it was a user error, so that a better error 
> > message can be provided next time.
> > A bug report can be filed in the tracker at 
> > . Thanks!
> > Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> > make[1]: *** [htmldocs] Error 1
> > Makefile:1450: recipe for target 'htmldocs' failed
> > make: *** [htmldocs] Error 2
> > 
> > Weird enough, it produced a 
> > Documentation/output/media/uapi/v4l/pipeline.svg file.
> 
> I guess that the dot command writes something to stderr. This is captured 
> by the extension and printed to stderr ...
> 
> +def dot2format(dot_fname, out_fname):
> ...
> +exit_code = 42
> +with open(out_fname, "w") as out:
> +p = subprocess.Popen(
> +cmd, stdout = out, stderr = subprocess.PIPE )
> +nil, err = p.communicate()
> +
> +sys.stderr.write(err)
> +
> +exit_code = p.returncode
> +out.flush()
> +return bool(exit_code == 0)
> 
> >  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
> > dot2format
> >sys.stderr.write(err)
> > TypeError: write() argument must be str, not bytes
> 
> Do we need this stderr output? For a first test, uncomment the 
> "sys.stderr.write(err)“ in line 222. Or, if we really need the
> stderr, try:
> 
> -sys.stderr.write(err)
> +sys.stderr.write(str(err))
> 
> I this fixes, there is another "sys.stderr.write(err)" in 
> func svg2pdf(..) which should also fixed ….
>  
> +def svg2pdf(svg_fname, pdf_fname):
> ...
> +cmd = [convert_cmd, svg_fname, pdf_fname]
> +p = subprocess.Popen(
> +cmd, stdout = out, stderr = subprocess.PIPE )
> +nil, err = p.communicate()
> +
> -sys.stderr.write(err)
> +sys.stderr.write(str(err))
> +
> +exit_code = p.returncode
> +return bool(exit_code == 0)

Yes, I very much want stderr to be forward. Without that you don't see
error output from dot or convert, and that makes it impossible to debug
anything. If I want a direct forwarding of the bytes, how should I do this
in python? Capturing stderr and then re-dumping it is kinda silly ...

Note that I copied this pattern from the kernel-doc extension, seems to
have worked there.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 17:13:25 +0100
Markus Heiser  escreveu:

> > Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab 
> > :
> >   
> >>> You can test it with virtualenv  
> >>> https://virtualenv.pypa.io/en/stable/userguide/
> >>> 
> >>> In short:
> >>> 
> >>> $ cd kernel-src
> >>> $ virtualenv myenv
> >>> $ source myenv/bin/activate
> >>> $ pip install 'Sphinx==1.3.1'
> >>> $ make 
> >> 
> >> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
> >> virtualenv is broken:
> >> 
> >> writing output... [ 16%] media/intro   
> >>  
> >> Exception occurred:
> >>  File 
> >> "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py",
> >>  line 671, in depart_document
> >>assert not self.context, 'len(context) = %s' % len(self.context)
> >> AssertionError: len(context) = 1
> >> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you 
> >> want to report the issue to the developers.
> >> Please also report this if it was a user error, so that a better error 
> >> message can be provided next time.
> >> A bug report can be filed in the tracker at 
> >> . Thanks!
> >> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> >> make[1]: *** [htmldocs] Error 1
> >> Makefile:1450: recipe for target 'htmldocs' failed
> >> make: *** [htmldocs] Error 2
> >> 
> >> Perhaps it is time for us to move minimal requirements to 1.4?  
> > 
> > Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
> > with 1.5.2 (for htmldocs).
> > 
> > Thanks,
> > Mauro
> > 
> > -
> > 
> > This is the error log with 1.4:
> > 
> > # Sphinx version: 1.4.9  
> 
> >  File 
> > "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py",
> >  line 187, in walkabout
> >visitor.dispatch_departure(self)
> >  File 
> > "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py",
> >  line 1895, in dispatch_departure
> >return method(node)
> >  File 
> > "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py",
> >  line 671, in depart_document
> >assert not self.context, 'len(context) = %s' % len(self.context)
> > AssertionError: len(context) = 1  
> 
> I guess this is a error from newer docutils, so we have to fix docutils 
> version also.
> 
> Sphinx itself specifies "docutils>=0.11"
> 
>   https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52
> 
> So I guess it uses a up to date docutils or the ducutils which are already 
> installed system wide.

The system-wide docutils is this one:

python2-docutils-0.13.1-3.fc25.noarch
python3-docutils-0.13.1-3.fc25.noarch

Btw, I tested also with virtualenv-3/pip3 and the same issue happens there.


Manually installing version 0.11 makes it to work again.

Considering that Sphinx require a specific docutils package for it to
work, perhaps it is time for us to consider to use the virtenv
enchantments at make docs targets :-p



Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 19:16:47 +0100
Markus Heiser  escreveu:

> > Am 02.03.2017 um 17:29 schrieb Mauro Carvalho Chehab 
> > :
> > 
> > Em Thu, 2 Mar 2017 17:13:25 +0100
> > Markus Heiser  escreveu:
> >   
> >>> Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab 
> >>> :
> >>>   
> > You can test it with virtualenv  
> > https://virtualenv.pypa.io/en/stable/userguide/
> > 
> > In short:
> > 
> > $ cd kernel-src
> > $ virtualenv myenv
> > $ source myenv/bin/activate
> > $ pip install 'Sphinx==1.3.1'
> > $ make   
>  
>  Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
>  virtualenv is broken:
>  
>  writing output... [ 16%] media/intro 
> 
>  Exception occurred:
>  File 
>  "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py",
>   line 671, in depart_document
>    assert not self.context, 'len(context) = %s' % len(self.context)
>  AssertionError: len(context) = 1
>  The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you 
>  want to report the issue to the developers.
>  Please also report this if it was a user error, so that a better error 
>  message can be provided next time.
>  A bug report can be filed in the tracker at 
>  . Thanks!
>  Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
>  make[1]: *** [htmldocs] Error 1
>  Makefile:1450: recipe for target 'htmldocs' failed
>  make: *** [htmldocs] Error 2
>  
>  Perhaps it is time for us to move minimal requirements to 1.4?
> >>> 
> >>> Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
> >>> with 1.5.2 (for htmldocs).
> >>> 
> >>> Thanks,
> >>> Mauro
> >>> 
> >>> -
> >>> 
> >>> This is the error log with 1.4:
> >>> 
> >>> # Sphinx version: 1.4.9
> >>   
> >>> File 
> >>> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py",
> >>>  line 187, in walkabout
> >>>   visitor.dispatch_departure(self)
> >>> File 
> >>> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py",
> >>>  line 1895, in dispatch_departure
> >>>   return method(node)
> >>> File 
> >>> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py",
> >>>  line 671, in depart_document
> >>>   assert not self.context, 'len(context) = %s' % len(self.context)
> >>> AssertionError: len(context) = 1
> >> 
> >> I guess this is a error from newer docutils, so we have to fix docutils 
> >> version also.
> >> 
> >> Sphinx itself specifies "docutils>=0.11"
> >> 
> >>  https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52
> >> 
> >> So I guess it uses a up to date docutils or the ducutils which are already 
> >> installed system wide.  
> > 
> > The system-wide docutils is this one:
> > 
> > python2-docutils-0.13.1-3.fc25.noarch
> > python3-docutils-0.13.1-3.fc25.noarch  
> 
> Sorry my mistake, in the traceback we see
> 
> File 
> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py",
>  line 671, in depart_document
>   assert not self.context, 'len(context) = %s' % len(self.context)
> 
> ... which means: system wide installation does not matter.
> 
> > Btw, I tested also with virtualenv-3/pip3 and the same issue happens there.
> > 
> > Manually installing version 0.11 makes it to work again.  
> 
> Yes, its only about "docutils>=0.11" in Sphinx 1.3 dependencies. 
> In Sphinx 1.5 the error:
> 
>   https://github.com/sphinx-doc/sphinx/issues/3212
> 
> is fixed:
> 
>   
> https://github.com/tk0miya/sphinx/commit/73663f63672f22304810ce6bb9787490ad250127
> 
> But this will never be fixed downwards.

Crap. This kind of patch should be backported to Sphinx 1.3/1.4,
or Python's PIP repository should be fixed to require docutils
version to be either 0.11 or 0.12, if one installs version 1.3
or 1.4 of Sphinx.

The way it is, PIP repository is broken!

> All this is about semantic versioning. If you want to promise your
> builds, you have to name which versions of dependencies you support.
> I guess this is nothing new for kernel developers ;)

No. At the Kernel, we do everything possible to not break APIs.

If this were not the case, you wouldn't be able to run Sphinx 1.5 
with legacy Kernel versions ;-)

> The problem is, that PEP440 defines not only ONE version scheme
> 
>  """Some hard to read version identifiers are permitted by
> this scheme in order to better accommodate the wide range
> of versioning practices across existing public and private
> Python projects."""
> 
> In practice, the python projects use slightly different schemes
> which not follow one rule like .. 
> From history packaging in 

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

Hi Mauro,

> Tested here with the enclosed patch.

great, big step forward making /media/Makefile smaller ...  thanks a lot

> It crashed:
> Exception occurred:
>  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
> dot2format
>sys.stderr.write(err)
> TypeError: write() argument must be str, not bytes
> The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you 
> want to report the issue to the developers.
> Please also report this if it was a user error, so that a better error 
> message can be provided next time.
> A bug report can be filed in the tracker at 
> . Thanks!
> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> make[1]: *** [htmldocs] Error 1
> Makefile:1450: recipe for target 'htmldocs' failed
> make: *** [htmldocs] Error 2
> 
> Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg 
> file.

I guess that the dot command writes something to stderr. This is captured 
by the extension and printed to stderr ...

+def dot2format(dot_fname, out_fname):
...
+exit_code = 42
+with open(out_fname, "w") as out:
+p = subprocess.Popen(
+cmd, stdout = out, stderr = subprocess.PIPE )
+nil, err = p.communicate()
+
+sys.stderr.write(err)
+
+exit_code = p.returncode
+out.flush()
+return bool(exit_code == 0)

>  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
> dot2format
>sys.stderr.write(err)
> TypeError: write() argument must be str, not bytes

Do we need this stderr output? For a first test, uncomment the 
"sys.stderr.write(err)“ in line 222. Or, if we really need the
stderr, try:

-sys.stderr.write(err)
+sys.stderr.write(str(err))

I this fixes, there is another "sys.stderr.write(err)" in 
func svg2pdf(..) which should also fixed ….
 
+def svg2pdf(svg_fname, pdf_fname):
...
+cmd = [convert_cmd, svg_fname, pdf_fname]
+p = subprocess.Popen(
+cmd, stdout = out, stderr = subprocess.PIPE )
+nil, err = p.communicate()
+
-sys.stderr.write(err)
+sys.stderr.write(str(err))
+
+exit_code = p.returncode
+return bool(exit_code == 0)

Thanks!

 -- Markus --

> 
> 
> Thanks,
> Mauro
> 
> --
> 
> [PATCH] docs-rst: Don't use explicit Makefile rules to build SVG and
> DOT files
> 
> Now that we have an extension to handle images, use it.
> 
> Signed-off-by: Mauro Carvalho Chehab 
> 
> diff --git a/Documentation/media/Makefile b/Documentation/media/Makefile
> index 32663602ff25..5bd52ea1eff0 100644
> --- a/Documentation/media/Makefile
> +++ b/Documentation/media/Makefile
> @@ -1,51 +1,6 @@
> -# Rules to convert DOT and SVG to Sphinx images
> -
> -SRC_DIR=$(srctree)/Documentation/media
> -
> -DOTS = \
> - uapi/v4l/pipeline.dot \
> -
> -IMAGES = \
> - typical_media_device.svg \
> - uapi/dvb/dvbstb.svg \
> - uapi/v4l/bayer.svg \
> - uapi/v4l/constraints.svg \
> - uapi/v4l/crop.svg \
> - uapi/v4l/fieldseq_bt.svg \
> - uapi/v4l/fieldseq_tb.svg \
> - uapi/v4l/nv12mt.svg \
> - uapi/v4l/nv12mt_example.svg \
> - uapi/v4l/pipeline.svg \
> - uapi/v4l/selection.svg \
> - uapi/v4l/subdev-image-processing-full.svg \
> - uapi/v4l/subdev-image-processing-scaling-multi-source.svg \
> - uapi/v4l/subdev-image-processing-crop.svg \
> - uapi/v4l/vbi_525.svg \
> - uapi/v4l/vbi_625.svg \
> - uapi/v4l/vbi_hsync.svg \
> -
> -DOTTGT := $(patsubst %.dot,%.svg,$(DOTS))
> -IMGDOT := $(patsubst %,$(SRC_DIR)/%,$(DOTTGT))
> -
> -IMGTGT := $(patsubst %.svg,%.pdf,$(IMAGES))
> -IMGPDF := $(patsubst %,$(SRC_DIR)/%,$(IMGTGT))
> -
> -cmd = $(echo-cmd) $(cmd_$(1))
> -
> -quiet_cmd_genpdf = GENPDF  $2
> -  cmd_genpdf = convert $2 $3
> -
> -quiet_cmd_gendot = DOT $2
> -  cmd_gendot = dot -Tsvg $2 > $3
> -
> -%.pdf: %.svg
> - @$(call cmd,genpdf,$<,$@)
> -
> -%.svg: %.dot
> - @$(call cmd,gendot,$<,$@)
> -
> # Rules to convert a .h file to inline RST documentation
> 
> +SRC_DIR=$(srctree)/Documentation/media
> PARSER = $(srctree)/Documentation/sphinx/parse-headers.pl
> UAPI = $(srctree)/include/uapi/linux
> KAPI = $(srctree)/include/linux
> diff --git a/Documentation/media/intro.rst b/Documentation/media/intro.rst
> index 8f7490c9a8ef..5562fba1d51d 100644
> --- a/Documentation/media/intro.rst
> +++ b/Documentation/media/intro.rst
> @@ -13,8 +13,8 @@ A typical media device hardware is shown at 
> :ref:`typical_media_device`.
> 
> .. _typical_media_device:
> 
> -.. figure::  typical_media_device.*
> -:alt:typical_media_device.pdf / typical_media_device.svg
> +.. kernel-figure::  typical_media_device.svg
> +:alt:typical_media_device.svg
> :align:  center
> 
> Typical Media Device
> diff --git a/Documentation/media/uapi/dvb/intro.rst 
> b/Documentation/media/uapi/dvb/intro.rst
> index 2ed5c23102b4..0e76067a5b58 100644
> --- 

[PATCH 5/6] drm/doc: diagram for mode objects and properties

[Daniel Vetter]

Resulted in confusion a few times in the past.

v2: Spelling fix (Eric).

Cc: Eric Anholt 
Acked-by: Eric Anholt 
Cc: Laurent Pinchart 
Cc: Manasi Navare 
Reviewed-by: Gabriel Krisman Bertazi 
Reviewed-by: Laurent Pinchart 
Signed-off-by: Daniel Vetter 
---
 Documentation/gpu/drm-kms.rst | 22 ++
 1 file changed, 22 insertions(+)

diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index b11c537c73f4..1c42448c7aae 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -161,6 +161,28 @@ KMS Core Structures and Functions
 Modeset Base Object Abstraction
 ===
 
+.. kernel-render:: DOT
+   :alt: Mode Objects and Properties
+   :caption: Mode Objects and Properties
+
+   digraph {
+  node [shape=box]
+
+  "drm_property A" -> "drm_mode_object A"
+  "drm_property A" -> "drm_mode_object B"
+  "drm_property B" -> "drm_mode_object A"
+   }
+
+The base structure for all KMS objects is :c:type:`struct drm_mode_object
+`. One of the base services it provides is tracking 
properties,
+which are especially important for the atomic IOCTL (see `Atomic Mode
+Setting`_). The somewhat surprising part here is that properties are not
+directly instantiated on each object, but free-standing mode objects 
themselves,
+represented by :c:type:`struct drm_property `, which only specify
+the type and value range of a property. Any given property can be attached
+multiple times to different objects using :c:func:`drm_object_attach_property()
+`.
+
 .. kernel-doc:: include/drm/drm_mode_object.h
:internal:
 
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

> Am 02.03.2017 um 19:20 schrieb Jonathan Corbet :
> 
> On Thu, 2 Mar 2017 19:16:47 +0100
> Markus Heiser  wrote:
> 
>> This is very easy, if we use a requiremts.txt file where we 
>> stick the versions and run the sphinx in this build in a
>> virtualenv which is build up by this requirements.txt.
>> 
>>  https://pip.pypa.io/en/stable/user_guide/#requirements-files
>> 
>> To summarize, I recommend a Makefile.sphinx cmd which does
>> something like:
>> 
>>  virtualenv output/myenv
>>  source output/myenv/bin/activate
>>  pip install -r requirements.txt
>>  sphinx-build ...
>> 
>> I guess this is something we should discuss with Jon, he
>> is also familiar with it virtualenv. 
> 
> That would perhaps make the build more reliable, but it would also make
> the build dependent on net access to PyPI, and that's not an idea I like a
> whole lot.  We should be able to do a build without going out on the
> network.

Right, there are PROS and CONS. Another point is where to place
the virtualenv. In my example above it is placed in output/ with
I’am not really happy.

> I'm kind of pressed for time, but will try to ponder on this more
> shortly…

My recommendation are mostly from a python developer POV, which is
sometimes diametral to what kernel development needs. You now both
POV in deep so I’am hopeful that you find the right conceptual
answers for those open questions ;)

-- Markus 
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Jonathan Corbet]

On Thu, 2 Mar 2017 19:16:47 +0100
Markus Heiser  wrote:

> This is very easy, if we use a requiremts.txt file where we 
> stick the versions and run the sphinx in this build in a
> virtualenv which is build up by this requirements.txt.
> 
>   https://pip.pypa.io/en/stable/user_guide/#requirements-files
> 
> To summarize, I recommend a Makefile.sphinx cmd which does
> something like:
> 
>   virtualenv output/myenv
>   source output/myenv/bin/activate
>   pip install -r requirements.txt
>   sphinx-build ...
>   
> I guess this is something we should discuss with Jon, he
> is also familiar with it virtualenv. 

That would perhaps make the build more reliable, but it would also make
the build dependent on net access to PyPI, and that's not an idea I like a
whole lot.  We should be able to do a build without going out on the
network.

I'm kind of pressed for time, but will try to ponder on this more
shortly...

jon
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] drm/doc: diagram for mode objects and properties

[Laurent Pinchart]

Hi Daniel,

Thank you for the patch.

On Wednesday 01 Mar 2017 09:27:14 Daniel Vetter wrote:
> Resulted in confusion a few times in the past.
> 
> v2: Spelling fix (Eric).
> 
> Cc: Eric Anholt 
> Acked-by: Eric Anholt 
> Cc: Laurent Pinchart 
> Cc: Manasi Navare 
> Signed-off-by: Daniel Vetter 

Reviewed-by: Laurent Pinchart 

> ---
>  Documentation/gpu/drm-kms.rst | 22 ++
>  1 file changed, 22 insertions(+)
> 
> diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> index 17a4cd5b14fd..a504d9ee4d94 100644
> --- a/Documentation/gpu/drm-kms.rst
> +++ b/Documentation/gpu/drm-kms.rst
> @@ -161,6 +161,28 @@ KMS Core Structures and Functions
>  Modeset Base Object Abstraction
>  ===
> 
> +.. kernel-render:: DOT
> +   :alt: Mode Objects and Properties
> +   :caption: Mode Objects and Properties
> +
> +   digraph {
> +  node [shape=box]
> +
> +  "drm_property A" -> "drm_mode_object A"
> +  "drm_property A" -> "drm_mode_object B"
> +  "drm_property B" -> "drm_mode_object A"
> +   }
> +
> +The base structure for all KMS objects is :c:type:`struct drm_mode_object
> +`. One of the base services it provides is tracking
> properties, +which are especially important for the atomic IOCTL (see
> `Atomic Mode +Setting`_). The somewhat surprising part here is that
> properties are not +directly instantiated on each object, but free-standing
> mode objects themselves, +represented by :c:type:`struct drm_property
> `, which only specify +the type and value range of a
> property. Any given property can be attached +multiple times to different
> objects using :c:func:`drm_object_attach_property()
> +`.
> +
>  .. kernel-doc:: include/drm/drm_mode_object.h
> 
> :internal:

-- 
Regards,

Laurent Pinchart

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

> Am 02.03.2017 um 17:29 schrieb Mauro Carvalho Chehab 
> :
> 
> Em Thu, 2 Mar 2017 17:13:25 +0100
> Markus Heiser  escreveu:
> 
>>> Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab 
>>> :
>>> 
> You can test it with virtualenv  
> https://virtualenv.pypa.io/en/stable/userguide/
> 
> In short:
> 
> $ cd kernel-src
> $ virtualenv myenv
> $ source myenv/bin/activate
> $ pip install 'Sphinx==1.3.1'
> $ make 
 
 Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
 virtualenv is broken:
 
 writing output... [ 16%] media/intro   
  
 Exception occurred:
 File 
 "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py",
  line 671, in depart_document
   assert not self.context, 'len(context) = %s' % len(self.context)
 AssertionError: len(context) = 1
 The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you 
 want to report the issue to the developers.
 Please also report this if it was a user error, so that a better error 
 message can be provided next time.
 A bug report can be filed in the tracker at 
 . Thanks!
 Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
 make[1]: *** [htmldocs] Error 1
 Makefile:1450: recipe for target 'htmldocs' failed
 make: *** [htmldocs] Error 2
 
 Perhaps it is time for us to move minimal requirements to 1.4?  
>>> 
>>> Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
>>> with 1.5.2 (for htmldocs).
>>> 
>>> Thanks,
>>> Mauro
>>> 
>>> -
>>> 
>>> This is the error log with 1.4:
>>> 
>>> # Sphinx version: 1.4.9  
>> 
>>> File 
>>> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py",
>>>  line 187, in walkabout
>>>   visitor.dispatch_departure(self)
>>> File 
>>> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py",
>>>  line 1895, in dispatch_departure
>>>   return method(node)
>>> File 
>>> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py",
>>>  line 671, in depart_document
>>>   assert not self.context, 'len(context) = %s' % len(self.context)
>>> AssertionError: len(context) = 1  
>> 
>> I guess this is a error from newer docutils, so we have to fix docutils 
>> version also.
>> 
>> Sphinx itself specifies "docutils>=0.11"
>> 
>>  https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52
>> 
>> So I guess it uses a up to date docutils or the ducutils which are already 
>> installed system wide.
> 
> The system-wide docutils is this one:
> 
>   python2-docutils-0.13.1-3.fc25.noarch
>   python3-docutils-0.13.1-3.fc25.noarch

Sorry my mistake, in the traceback we see

File 
"/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py",
 line 671, in depart_document
  assert not self.context, 'len(context) = %s' % len(self.context)

... which means: system wide installation does not matter.

> Btw, I tested also with virtualenv-3/pip3 and the same issue happens there.
> 
> Manually installing version 0.11 makes it to work again.

Yes, its only about "docutils>=0.11" in Sphinx 1.3 dependencies. 
In Sphinx 1.5 the error:

  https://github.com/sphinx-doc/sphinx/issues/3212

is fixed:

  
https://github.com/tk0miya/sphinx/commit/73663f63672f22304810ce6bb9787490ad250127

But this will never be fixed downwards.

All this is about semantic versioning. If you want to promise your
builds, you have to name which versions of dependencies you support.
I guess this is nothing new for kernel developers ;)

The problem is, that PEP440 defines not only ONE version scheme

 """Some hard to read version identifiers are permitted by
this scheme in order to better accommodate the wide range
of versioning practices across existing public and private
Python projects."""

In practice, the python projects use slightly different schemes
which not follow one rule like .. 
>From history packaging in Python is the hell, it becomes better, but
the problem with slightly different version schemes still exist.

How can this info help us? Now we know, that we have to stick Sphinx
and docutils by patch-levels we are willing to support. Or with
your words 

> Considering that Sphinx require a specific docutils package for it to
> work, perhaps it is time for us to consider to use the virtenv
> enchantments at make docs targets :-p


This is very easy, if we use a requiremts.txt file where we 
stick the versions and run the sphinx in this build in a
virtualenv which is build up by this requirements.txt.

  https://pip.pypa.io/en/stable/user_guide/#requirements-files

To summarize, I recommend a Makefile.sphinx cmd which does
something like:

  virtualenv 

[PATCH 1/6] doc: Explain light-handed markup preference a bit better

[Daniel Vetter]

We already had a super-short blurb, but worth extending it I think:
We're still pretty far away from anything like a consensus, but
there's clearly a lot of people who prefer an as-light as possible
approach to converting existing .txt files to .rst. Make sure this is
properly taken into account and clear.

Motivated by discussions with Peter and Christoph and others.

v2:
- Mention that existing headings should be kept when converting
  existing .txt files (Mauro).
- Explain that we prefer :: for quoting code, it's easier on the
  eyes (Mauro).
- Explain that blindly converting outdated docs is harmful. Motived
  by comments Peter did in our discussion.

v3: Make the explanations around fixed-width quoting more concise
(Jani).

v4:
- Rebase onto docs-4.10.
- Go with the more terse recommendation from Jani, defer to the much
  more detailed conversion guide Mauro is working on for details.

Cc: Jonathan Corbet 
Cc: linux-doc@vger.kernel.org
Cc: Christoph Hellwig 
Cc: Peter Zijlstra 
Cc: Jani Nikula 
Cc: Mauro Carvalho Chehab 
Signed-off-by: Daniel Vetter 
---
 Documentation/doc-guide/sphinx.rst | 17 -
 1 file changed, 16 insertions(+), 1 deletion(-)

diff --git a/Documentation/doc-guide/sphinx.rst 
b/Documentation/doc-guide/sphinx.rst
index 96fe7ccb2c67..532d65b70500 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -73,7 +73,16 @@ Specific guidelines for the kernel documentation
 
 Here are some specific guidelines for the kernel documentation:
 
-* Please don't go overboard with reStructuredText markup. Keep it simple.
+* Please don't go overboard with reStructuredText markup. Keep it
+  simple. For the most part the documentation should be plain text with
+  just enough consistency in formatting that it can be converted to
+  other formats.
+
+* Please keep the formatting changes minimal when converting existing
+  documentation to reStructuredText.
+
+* Also update the content, not just the formatting, when converting
+  documentation.
 
 * Please stick to this order of heading adornments:
 
@@ -103,6 +112,12 @@ Here are some specific guidelines for the kernel 
documentation:
   the order as encountered."), having the higher levels the same overall makes
   it easier to follow the documents.
 
+* For inserting fixed width text blocks (for code examples, use case
+  examples, etc.), use ``::`` for anything that doesn't really benefit
+  from syntax highlighting, especially short snippets. Use
+  ``.. code-block:: `` for longer code blocks that benefit
+  from highlighting.
+
 
 the C domain
 
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 6/6] drm/doc: atomic overview, with graph

[Daniel Vetter]

I want to split up a few more things and document some details better
(like how exactly to subclass drm_atomic_state). And maybe also split
up the helpers a bit per-topic, but this should be a ok-ish start for
better atomic overview.

v2: Spelling and clarifications (Eric).

v3: Implement suggestion from Gabriel to fix the graph.

v4: Review from Laurent.

Reviewed-by: Laurent Pinchart 
Cc: Gabriel Krisman Bertazi 
Acked-by: Eric Anholt 
Cc: Eric Anholt 
Cc: Laurent Pinchart 
Cc: Harry Wentland 
Signed-off-by: Daniel Vetter 
---
 Documentation/gpu/drm-kms-helpers.rst |  2 +
 Documentation/gpu/drm-kms.rst | 84 ++-
 2 files changed, 85 insertions(+), 1 deletion(-)

diff --git a/Documentation/gpu/drm-kms-helpers.rst 
b/Documentation/gpu/drm-kms-helpers.rst
index 050ebe81d256..ac53c0b893f6 100644
--- a/Documentation/gpu/drm-kms-helpers.rst
+++ b/Documentation/gpu/drm-kms-helpers.rst
@@ -42,6 +42,8 @@ Modeset Helper Reference for Common Vtables
 .. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
:internal:
 
+.. _drm_atomic_helper:
+
 Atomic Modeset Helper Functions Reference
 =
 
diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index 1c42448c7aae..bfecd21a8cdf 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -189,8 +189,90 @@ multiple times to different objects using 
:c:func:`drm_object_attach_property()
 .. kernel-doc:: drivers/gpu/drm/drm_mode_object.c
:export:
 
+Atomic Mode Setting
+===
+
+
+.. kernel-render:: DOT
+   :alt: Mode Objects and Properties
+   :caption: Mode Objects and Properties
+
+   digraph {
+  node [shape=box]
+
+  subgraph cluster_state {
+  style=dashed
+  label="Free-standing state"
+
+  "drm_atomic_state" -> "duplicated drm_plane_state A"
+  "drm_atomic_state" -> "duplicated drm_plane_state B"
+  "drm_atomic_state" -> "duplicated drm_crtc_state"
+  "drm_atomic_state" -> "duplicated drm_connector_state"
+  "drm_atomic_state" -> "duplicated driver private state"
+  }
+
+  subgraph cluster_current {
+  style=dashed
+  label="Current state"
+
+  "drm_device" -> "drm_plane A"
+  "drm_device" -> "drm_plane B"
+  "drm_device" -> "drm_crtc"
+  "drm_device" -> "drm_connector"
+  "drm_device" -> "driver private object"
+
+  "drm_plane A" -> "drm_plane_state A"
+  "drm_plane B" -> "drm_plane_state B"
+  "drm_crtc" -> "drm_crtc_state"
+  "drm_connector" -> "drm_connector_state"
+  "driver private object" -> "driver private state"
+  }
+
+  "drm_atomic_state" -> "drm_device" [label="atomic_commit"]
+  "duplicated drm_plane_state A" -> "drm_device"[style=invis]
+   }
+
+Atomic provides transactional modeset (including planes) updates, but a
+bit differently from the usual transactional approach of try-commit and
+rollback:
+
+- Firstly, no hardware changes are allowed when the commit would fail. This
+  allows us to implement the DRM_MODE_ATOMIC_TEST_ONLY mode, which allows
+  userspace to explore whether certain configurations would work or not.
+
+- This would still allow setting and rollback of just the software state,
+  simplifying conversion of existing drivers. But auditing drivers for
+  correctness of the atomic_check code becomes really hard with that: Rolling
+  back changes in data structures all over the place is hard to get right.
+
+- Lastly, for backwards compatibility and to support all use-cases, atomic
+  updates need to be incremental and be able to execute in parallel. Hardware
+  doesn't always allow it, but where possible plane updates on different CRTCs
+  should not interfere, and not get stalled due to output routing changing on
+  different CRTCs.
+
+Taken all together there's two consequences for the atomic design:
+
+- The overall state is split up into per-object state structures:
+  :c:type:`struct drm_plane_state ` for planes, 
:c:type:`struct
+  drm_crtc_state ` for CRTCs and :c:type:`struct
+  drm_connector_state ` for connectors. These are the only
+  objects with userspace-visible and settable state. For internal state drivers
+  can subclass these structures through embeddeding, or add entirely new state
+  structures for their globally shared hardware functions.
+
+- An atomic update is assembled and validated as an entirely free-standing pile
+  of structures within the :c:type:`drm_atomic_state `
+  container. Again drivers can subclass that container for their own state
+  structure tracking needs. Only when a state is committed is it applied to the
+  driver and modeset objects. This way rolling back an update boils down to
+  releasing 

Re: [PATCH] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu,  2 Mar 2017 16:40:02 +0100
Daniel Vetter  escreveu:

> From: Markus Heiser 
> 
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
> 
> * DOT (http://www.graphviz.org)
> * SVG
> 
> For image handling use the 'image' replacement::
> 
> .. kernel-image::  svg_image.svg
>:alt:simple SVG image
> 
> For figure handling use the 'figure' replacement::
> 
> .. kernel-figure::  svg_image.svg
>:alt:simple SVG image
> 
>SVG image example
> 
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
> 
>   .. kernel-render:: DOT
>  :alt: foobar digraph
>  :caption: Embedded **DOT** (Graphviz) code.
> 
>  digraph foo {
>   "bar" -> "baz";
>  }
> 
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
> 
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
> 
> v2: s/DOC/DOT/ in a few places (by Daniel).
> 
> v3: Simplify stuff a bit (by Daniel):
> 
> - Remove path detection and setup/check code for that. In
>   Documentation/media/Makefile we already simply use these tools,
>   better to have one consolidated check if we want/need one. Also
>   remove the convertsvg support, we require ImageMagick's convert
>   already in the doc build, no need for a 2nd fallback.
> 
> - Use sphinx for depency tracking, remove hand-rolled version.
> 
> - Forward stderr from dot and convert, otherwise debugging issues with
>   the diagrams is impossible.
> 
> v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.
> 
> v5: Use an explicit version check (suggested by Laurent).

Tested here with the enclosed patch. It crashed:


Exception occurred:
  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
dot2format
sys.stderr.write(err)
TypeError: write() argument must be str, not bytes
The full traceback has been saved in /tmp/sphinx-err-_1vahbmg.log, if you want 
to report the issue to the developers.
Please also report this if it was a user error, so that a better error message 
can be provided next time.
A bug report can be filed in the tracker at 
. Thanks!
Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
make[1]: *** [htmldocs] Error 1
Makefile:1450: recipe for target 'htmldocs' failed
make: *** [htmldocs] Error 2

Weird enough, it produced a Documentation/output/media/uapi/v4l/pipeline.svg 
file.

Full trace:

Traceback (most recent call last):
  File 
"/home/mchehab/.local/lib/python3.5/site-packages/sphinx/util/parallel.py", 
line 73, in _process
ret = func(arg)
  File 
"/home/mchehab/.local/lib/python3.5/site-packages/sphinx/builders/__init__.py", 
line 380, in write_process
self.write_doc(docname, doctree)
  File 
"/home/mchehab/.local/lib/python3.5/site-packages/sphinx/builders/html.py", 
line 448, in write_doc
self.docwriter.write(doctree, destination)
  File 
"/home/mchehab/.local/lib/python3.5/site-packages/docutils/writers/__init__.py",
 line 80, in write
self.translate()
  File 
"/home/mchehab/.local/lib/python3.5/site-packages/sphinx/writers/html.py", line 
47, in translate
self.document.walkabout(visitor)
  File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", 
line 174, in walkabout
if child.walkabout(visitor):
  File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", 
line 174, in walkabout
if child.walkabout(visitor):
  File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", 
line 174, in walkabout
if child.walkabout(visitor):
  File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", 
line 166, in walkabout
visitor.dispatch_visit(self)
  File "/home/mchehab/.local/lib/python3.5/site-packages/docutils/nodes.py", 
line 1882, in dispatch_visit
return method(node)
  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 302, in 
visit_kernel_figure
convert_image(img_node, self)
  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 193, in 
convert_image
dot2format(src_fname, dst_fname)
  File "/devel/v4l/patchwork/Documentation/sphinx/kfigure.py", line 222, in 
dot2format
sys.stderr.write(err)
TypeError: write() argument must be str, not bytes


Thanks,
Mauro

--

[PATCH] docs-rst: Don't use explicit Makefile rules to build SVG and
 DOT files

Now that we have an extension to handle images, use it.

Signed-off-by: Mauro Carvalho Chehab 

diff --git a/Documentation/media/Makefile b/Documentation/media/Makefile
index 32663602ff25..5bd52ea1eff0 100644
--- a/Documentation/media/Makefile
+++ 

Re: [RFC PATCH v4 19/28] swiotlb: Add warnings for use of bounce buffers with SME

[Paolo Bonzini]

On 17/02/2017 17:51, Tom Lendacky wrote:
> 
> It's meant just to notify the user about the condition. The user could
> then decide to use an alternative device that supports a greater DMA
> range (I can probably change it to a dev_warn_once() so that a device
> is identified).  I would be nice if I could issue this message once per
> device that experienced this.  I didn't see anything that would do
> that, though.

dev_warn_once would print once only, not once per device.  But if you
leave the dev_warn elsewhere, this can be just pr_warn_once.

Paolo
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

> Am 02.03.2017 um 16:49 schrieb Mauro Carvalho Chehab 
> :
> 
>>> You can test it with virtualenv  
>>> https://virtualenv.pypa.io/en/stable/userguide/
>>> 
>>> In short:
>>> 
>>> $ cd kernel-src
>>> $ virtualenv myenv
>>> $ source myenv/bin/activate
>>> $ pip install 'Sphinx==1.3.1'
>>> $ make   
>> 
>> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
>> virtualenv is broken:
>> 
>> writing output... [ 16%] media/intro 
>>
>> Exception occurred:
>>  File 
>> "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py",
>>  line 671, in depart_document
>>assert not self.context, 'len(context) = %s' % len(self.context)
>> AssertionError: len(context) = 1
>> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want 
>> to report the issue to the developers.
>> Please also report this if it was a user error, so that a better error 
>> message can be provided next time.
>> A bug report can be filed in the tracker at 
>> . Thanks!
>> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
>> make[1]: *** [htmldocs] Error 1
>> Makefile:1450: recipe for target 'htmldocs' failed
>> make: *** [htmldocs] Error 2
>> 
>> Perhaps it is time for us to move minimal requirements to 1.4?
> 
> Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
> with 1.5.2 (for htmldocs).
> 
> Thanks,
> Mauro
> 
> -
> 
> This is the error log with 1.4:
> 
> # Sphinx version: 1.4.9

>  File 
> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py",
>  line 187, in walkabout
>visitor.dispatch_departure(self)
>  File 
> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/nodes.py",
>  line 1895, in dispatch_departure
>return method(node)
>  File 
> "/devel/v4l/patchwork/myenv-1.4/lib/python2.7/site-packages/docutils/writers/_html_base.py",
>  line 671, in depart_document
>assert not self.context, 'len(context) = %s' % len(self.context)
> AssertionError: len(context) = 1

I guess this is a error from newer docutils, so we have to fix docutils version 
also.

Sphinx itself specifies "docutils>=0.11"

  https://github.com/sphinx-doc/sphinx/blob/1.3.1/setup.py#L52

So I guess it uses a up to date docutils or the ducutils which are already 
installed system wide.

— Markus —
 

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Laurent Pinchart]

Hi Daniel and Markus,

On Thursday 02 Mar 2017 16:11:08 Daniel Vetter wrote:
> On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:
> > Am 02.03.2017 um 15:14 schrieb Laurent Pinchart:
> > > On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
> > >> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> > >>> Hi Daniel,
> > >>> 
> > >>> Thank you for the patch.
> > >>> 
> > >>> With this applied, I get
> > >>> 
> > >>> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> > >>> 
> > >>>  SPHINX  htmldocs -->
> > >>>  file:///home/laurent/src/iob/renesas/linux64/Documentation/output
> > >>>  PARSE
> > >>>  
> > >>>include/uapi/linux/videodev2.h
> > >>> 
> > >>> Running Sphinx v1.3.1
> > >>> 
> > >>> Extension error:
> > >>> Could not import extension kfigure (exception: cannot import name
> > >>> patches)
> > >>> make[2]: ***
> > >>> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> > >>> htmldocs] Error 1 make[1]: ***
> > >>> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> > >>> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64'
> > >>> make:
> > >>> *** [Makefile:152: sub-make] Error 2
> > >>> 
> > >>> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to
> > >>> bump
> > >>> the minimum required version I think a notice is needed.
> > >> 
> > >> Ugh. But this also goes completely over my head, no idea whether we
> > >> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
> > >> there's some way to work around this ... Halp?
> > > 
> > > I'm not a Sphinx expert so I don't know, but what I can tell is that
> > > copying the patches.py from Sphinx 1.4 to Documentation/sphinx/ and
> > > modifying kfigure.py to import it from there fixes the build. There's
> > > thus no extra depencency on Sphinx 1.4 (or newer).
> > > 
> > > I'm not sure we want to set a precedent by copying part of the Sphinx
> > > source code to the kernel tree (or inlining the single small function
> > > that the module provides), and I'll let someone more knowledgeable than
> > > me decide how to proceed.
> > 
> > Aargh ... we need virtualenv! For interim something like the following
> > might help. In file Documentation/sphinx/kfigure.py edit the imports
> > 
> > ...
> > from docutils.parsers.rst.directives import images
> > 
> > try:
> > from sphinx.directives.patches import Figure
> > 
> > except ImportError:
> > Figure = images.Figure
> > 
> > ...
> > 
> > And fix the class definition, so it use 'Figure' and no
> > longer 'patch.Figure'::
> > 
> > ...
> > -class KernelFigure(patches.Figure):
> > +class KernelFigure(Figure):
> > ...
> > 
> > Sorry that I have not yet the time to send you a decent and tested
> > patch. Do you like to test my suggestion? / thanks!
> 
> I'll give it a shot at implementing it, but I can't (easily at least) test
> on sphinx 1.3.

I can, and it works.

Tested-by: Laurent Pinchart 

Thank you for the quick fix Markus.

-- 
Regards,

Laurent Pinchart

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 11/32] KVM: MIPS: Add VZ capability

[James Hogan]

Hi Paolo,

On Thu, Mar 02, 2017 at 11:59:28AM +0100, Paolo Bonzini wrote:
> On 02/03/2017 10:36, James Hogan wrote:
> >  - KVM_VM_MIPS_DEFAULT = 2
> > 
> >This will provide the best available KVM implementation (even on
> >older kernels), preferring hardware assisted virtualization over trap
> >& emulate. The KVM_CAP_MIPS_VZ capability should always be checked
> >against known values to determine what type of implementation was
> >chosen.
> > 
> > This is designed to allow the desired implementation (T vs VZ) to be
> > potentially chosen at runtime rather than being fixed in the kernel
> > configuration.
> 
> Can the same kernel run on both TE and VZ?  If not, I'm not sure that
> KVM_VM_MIPS_DEFAULT is a good idea.

It can't right now, though with relocation of the kernel now implemented
in MIPS Linux for KASLR, and hopes for a more generic EVA implementation
(which can require the kernel to be linked in a completely different
segment) it isn't completely infeasible.

Currently the two uses of this I've implemented are:

1) QEMU, which I've implemented using the kvm_type machine callback.
This allows the KVM type to be specified with e.g.
  "-machine malta,accel=kvm,kvm-type=TE"
Otherwise it defaults to using KVM_VM_MIPS_DEFAULT.

When you try and load a kernel (which happens after kvm_init() has
already passed the kvm type into KVM_CREATE_VM) it will check that it
supports the current kernel type.

2) My kvm test application, which uses KVM_VM_MIPS_DEFAULT by default
and hackily maps itself into the guest physical address space to run C
code test cases.

Does that justification sound reasonable?

Cheers
James


signature.asc
Description: Digital signature

[PATCH] docs-rst: automatically convert Graphviz and SVG images

[Daniel Vetter]

From: Markus Heiser 

This patch brings scalable figure, image handling and a concept to
embed *render* markups:

* DOT (http://www.graphviz.org)
* SVG

For image handling use the 'image' replacement::

.. kernel-image::  svg_image.svg
   :alt:simple SVG image

For figure handling use the 'figure' replacement::

.. kernel-figure::  svg_image.svg
   :alt:simple SVG image

   SVG image example

Embed *render* markups (or languages) like Graphviz's **DOT** is
provided by the *render* directive.::

  .. kernel-render:: DOT
 :alt: foobar digraph
 :caption: Embedded **DOT** (Graphviz) code.

 digraph foo {
  "bar" -> "baz";
 }

The *render* directive is a concept to integrate *render* markups and
languages, yet supported markups:

* DOT: render embedded Graphviz's **DOT**
* SVG: render embedded Scalable Vector Graphics (**SVG**)

v2: s/DOC/DOT/ in a few places (by Daniel).

v3: Simplify stuff a bit (by Daniel):

- Remove path detection and setup/check code for that. In
  Documentation/media/Makefile we already simply use these tools,
  better to have one consolidated check if we want/need one. Also
  remove the convertsvg support, we require ImageMagick's convert
  already in the doc build, no need for a 2nd fallback.

- Use sphinx for depency tracking, remove hand-rolled version.

- Forward stderr from dot and convert, otherwise debugging issues with
  the diagrams is impossible.

v4: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
Implement Markus suggestion for backwards compatability with earlier
releases. Laurent reported this, running sphinx 1.3. Solution entirely
untested.

v5: Use an explicit version check (suggested by Laurent).

Cc: Jonathan Corbet 
Cc: linux-doc@vger.kernel.org
Cc: Jani Nikula 
Cc: Mauro Carvalho Chehab 
Cc: Markus Heiser 
Cc: Laurent Pinchart 
Acked-by: Markus Heiser 
Tested-by: Laurent Pinchart 
Signed-off-by: Markus Heiser  (v1)
Signed-off-by: Daniel Vetter 
---
 Documentation/conf.py |   2 +-
 Documentation/doc-guide/hello.dot |   3 +
 Documentation/doc-guide/sphinx.rst|  90 ++-
 Documentation/doc-guide/svg_image.svg |  10 +
 Documentation/process/changes.rst |   7 +-
 Documentation/sphinx/kfigure.py   | 450 ++
 6 files changed, 556 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/doc-guide/hello.dot
 create mode 100644 Documentation/doc-guide/svg_image.svg
 create mode 100644 Documentation/sphinx/kfigure.py

diff --git a/Documentation/conf.py b/Documentation/conf.py
index f6823cf01275..e3f537ce2935 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ from load_config import loadConfig
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 
'kfigure']
 
 # The name of the math extension changed on Sphinx 1.4
 if major == 1 and minor > 3:
diff --git a/Documentation/doc-guide/hello.dot 
b/Documentation/doc-guide/hello.dot
new file mode 100644
index ..504621dfc595
--- /dev/null
+++ b/Documentation/doc-guide/hello.dot
@@ -0,0 +1,3 @@
+graph G {
+  Hello -- World
+}
diff --git a/Documentation/doc-guide/sphinx.rst 
b/Documentation/doc-guide/sphinx.rst
index 532d65b70500..b902744ce7dd 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -34,8 +34,10 @@ format-specific subdirectories under 
``Documentation/output``.
 
 To generate documentation, Sphinx (``sphinx-build``) must obviously be
 installed. For prettier HTML output, the Read the Docs Sphinx theme
-(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is 
also
-needed. All of these are widely available and packaged in distributions.
+(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
+``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
+from ImageMagick (https://www.imagemagick.org). All of these are widely
+available and packaged in distributions.
 
 To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
 variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
@@ -232,3 +234,87 @@ Rendered as:
   * .. _`last row`:
 
 - column 3
+
+
+Figures & Images
+
+
+If you want to add an image, you should use the ``kernel-figure`` and
+``kernel-image`` directives. E.g. to insert a figure with a scalable
+image format use SVG::
+
+.. kernel-figure::  svg_image.svg
+   

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Daniel Vetter]

On Thu, Mar 2, 2017 at 4:22 PM, Jonathan Corbet  wrote:
> On Thu, 2 Mar 2017 16:11:08 +0100
> Daniel Vetter  wrote:
>
>> I'll give it a shot at implementing it, but I can't (easily at least) test
>> on sphinx 1.3.
>
> Virtualenv makes that kind of thing pretty easy to do.  Maybe we should
> add a script to create and populate a suitable venv for this kind of
> thing.

Laurent quickly checked v5 of this patch, looks all good now.

> Meanwhile, I've been seeing only parts 1 and 2 of what's clearly a bigger
> series.  Do you want me to carry just these two?

I submitted the entire series to both linux-doc and dri-devel. But
yeah, probably best if you just pull in the first 2 and send me a
topic pull for drm so I can apply the drm stuff on top in drm-misc.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 16:53:04 +0100
Daniel Vetter  escreveu:

> On Thu, Mar 2, 2017 at 4:22 PM, Jonathan Corbet  wrote:
> > On Thu, 2 Mar 2017 16:11:08 +0100
> > Daniel Vetter  wrote:
> >  
> >> I'll give it a shot at implementing it, but I can't (easily at least) test
> >> on sphinx 1.3.  
> >
> > Virtualenv makes that kind of thing pretty easy to do.  Maybe we should
> > add a script to create and populate a suitable venv for this kind of
> > thing.  
> 
> Laurent quickly checked v5 of this patch, looks all good now.
> 
> > Meanwhile, I've been seeing only parts 1 and 2 of what's clearly a bigger
> > series.  Do you want me to carry just these two?  
> 
> I submitted the entire series to both linux-doc and dri-devel. 

Sure you sent to linux-doc? I'm not seeing it there.

I'm not seeing it at the mirrors neither:
https://marc.info/?l=linux-doc=1=201703=2
https://www.spinics.net/lists/linux-doc/maillist.html

Regards,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 12:45:32 -0300
Mauro Carvalho Chehab  escreveu:

> Em Thu, 2 Mar 2017 16:21:33 +0100
> Markus Heiser  escreveu:
> 
> > Am 02.03.2017 um 16:11 schrieb Daniel Vetter :
> >   
> > > On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:
> > >> Hi Daniel, Laurent
> > >> 
> > >> Am 02.03.2017 um 15:14 schrieb Laurent Pinchart 
> > >> :
> > >> 
> > >>> Hi Daniel,
> > >>> 
> > >>> On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
> >  On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> > > Hi Daniel,
> > > 
> > > Thank you for the patch.
> > > 
> > > With this applied, I get
> > > 
> > > make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> > > 
> > > SPHINX  htmldocs -->
> > > file:///home/laurent/src/iob/renesas/linux64/Documentation/output 
> > > PARSE
> > >   include/uapi/linux/videodev2.h
> > > 
> > > Running Sphinx v1.3.1
> > > 
> > > Extension error:
> > > Could not import extension kfigure (exception: cannot import name 
> > > patches)
> > > make[2]: ***
> > > [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> > > htmldocs] Error 1 make[1]: ***
> > > [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> > > make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' 
> > > make:
> > > *** [Makefile:152: sub-make] Error 2
> > > 
> > > sphinx.directive.patches got introduced in Sphinx 1.4. If you want to 
> > > bump
> > > the minimum required version I think a notice is needed.
> >  
> >  Ugh. But this also goes completely over my head, no idea whether we
> >  must require sphinx 1.4 (it was released Mar 28, 2016), or whether
> >  there's some way to work around this ... Halp?
> > >>> 
> > >>> I'm not a Sphinx expert so I don't know, but what I can tell is that 
> > >>> copying 
> > >>> the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying 
> > >>> kfigure.py to import it from there fixes the build. There's thus no 
> > >>> extra 
> > >>> depencency on Sphinx 1.4 (or newer).
> > >>> 
> > >>> I'm not sure we want to set a precedent by copying part of the Sphinx 
> > >>> source 
> > >>> code to the kernel tree (or inlining the single small function that the 
> > >>> module 
> > >>> provides), and I'll let someone more knowledgeable than me decide how 
> > >>> to 
> > >>> proceed.
> > >> 
> > >> 
> > >> Aargh ... we need virtualenv! For interim something like the following
> > >> might help. In file Documentation/sphinx/kfigure.py edit the imports
> > >> 
> > >> ...
> > >> from docutils.parsers.rst.directives import images
> > >> try:
> > >>from sphinx.directives.patches import Figure
> > >> except ImportError:
> > >>Figure = images.Figure 
> > >> ...
> > >> 
> > >> And fix the class definition, so it use 'Figure' and no
> > >> longer 'patch.Figure'::
> > >> 
> > >> ...
> > >> -class KernelFigure(patches.Figure):
> > >> +class KernelFigure(Figure):
> > >> ...
> > >> 
> > >> Sorry that I have not yet the time to send you a decent and tested
> > >> patch. Do you like to test my suggestion? / thanks!
> > > 
> > > I'll give it a shot at implementing it, but I can't (easily at least) test
> > > on sphinx 1.3.
> > 
> > You can test it with virtualenv  
> > https://virtualenv.pypa.io/en/stable/userguide/
> > 
> > In short:
> > 
> > $ cd kernel-src
> > $ virtualenv myenv
> > $ source myenv/bin/activate
> > $ pip install 'Sphinx==1.3.1'
> > $ make   
> 
> Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
> virtualenv is broken:
> 
> writing output... [ 16%] media/intro  
>   
> Exception occurred:
>   File 
> "/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py",
>  line 671, in depart_document
> assert not self.context, 'len(context) = %s' % len(self.context)
> AssertionError: len(context) = 1
> The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want 
> to report the issue to the developers.
> Please also report this if it was a user error, so that a better error 
> message can be provided next time.
> A bug report can be filed in the tracker at 
> . Thanks!
> Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
> make[1]: *** [htmldocs] Error 1
> Makefile:1450: recipe for target 'htmldocs' failed
> make: *** [htmldocs] Error 2
> 
> Perhaps it is time for us to move minimal requirements to 1.4?

Hmm... the same happened with 1.4.9 inside virtualenv. It built fine
with 1.5.2 (for htmldocs).

Thanks,
Mauro

-

This is the error log with 1.4:

# Sphinx version: 1.4.9
# Python version: 2.7.13 (CPython)
# Docutils version: 0.13.1 release
# 

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Jonathan Corbet]

On Thu, 2 Mar 2017 16:11:08 +0100
Daniel Vetter  wrote:

> I'll give it a shot at implementing it, but I can't (easily at least) test
> on sphinx 1.3.

Virtualenv makes that kind of thing pretty easy to do.  Maybe we should
add a script to create and populate a suitable venv for this kind of
thing.

Meanwhile, I've been seeing only parts 1 and 2 of what's clearly a bigger
series.  Do you want me to carry just these two?

jon
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

Am 02.03.2017 um 16:11 schrieb Daniel Vetter :

> On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:
>> Hi Daniel, Laurent
>> 
>> Am 02.03.2017 um 15:14 schrieb Laurent Pinchart 
>> :
>> 
>>> Hi Daniel,
>>> 
>>> On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
 On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> Hi Daniel,
> 
> Thank you for the patch.
> 
> With this applied, I get
> 
> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> 
> SPHINX  htmldocs -->
> file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
>   include/uapi/linux/videodev2.h
> 
> Running Sphinx v1.3.1
> 
> Extension error:
> Could not import extension kfigure (exception: cannot import name patches)
> make[2]: ***
> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> htmldocs] Error 1 make[1]: ***
> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
> *** [Makefile:152: sub-make] Error 2
> 
> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
> the minimum required version I think a notice is needed.
 
 Ugh. But this also goes completely over my head, no idea whether we
 must require sphinx 1.4 (it was released Mar 28, 2016), or whether
 there's some way to work around this ... Halp?
>>> 
>>> I'm not a Sphinx expert so I don't know, but what I can tell is that 
>>> copying 
>>> the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying 
>>> kfigure.py to import it from there fixes the build. There's thus no extra 
>>> depencency on Sphinx 1.4 (or newer).
>>> 
>>> I'm not sure we want to set a precedent by copying part of the Sphinx 
>>> source 
>>> code to the kernel tree (or inlining the single small function that the 
>>> module 
>>> provides), and I'll let someone more knowledgeable than me decide how to 
>>> proceed.
>> 
>> 
>> Aargh ... we need virtualenv! For interim something like the following
>> might help. In file Documentation/sphinx/kfigure.py edit the imports
>> 
>> ...
>> from docutils.parsers.rst.directives import images
>> try:
>>from sphinx.directives.patches import Figure
>> except ImportError:
>>Figure = images.Figure 
>> ...
>> 
>> And fix the class definition, so it use 'Figure' and no
>> longer 'patch.Figure'::
>> 
>> ...
>> -class KernelFigure(patches.Figure):
>> +class KernelFigure(Figure):
>> ...
>> 
>> Sorry that I have not yet the time to send you a decent and tested
>> patch. Do you like to test my suggestion? / thanks!
> 
> I'll give it a shot at implementing it, but I can't (easily at least) test
> on sphinx 1.3.

You can test it with virtualenv  https://virtualenv.pypa.io/en/stable/userguide/

In short:

$ cd kernel-src
$ virtualenv myenv
$ source myenv/bin/activate
$ pip install 'Sphinx==1.3.1'
$ make 

-- Markus --


> -Daniel
> -- 
> Daniel Vetter
> Software Engineer, Intel Corporation
> http://blog.ffwll.ch

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Mauro Carvalho Chehab]

Em Thu, 2 Mar 2017 16:21:33 +0100
Markus Heiser  escreveu:

> Am 02.03.2017 um 16:11 schrieb Daniel Vetter :
> 
> > On Thu, Mar 02, 2017 at 03:58:36PM +0100, Markus Heiser wrote:  
> >> Hi Daniel, Laurent
> >> 
> >> Am 02.03.2017 um 15:14 schrieb Laurent Pinchart 
> >> :
> >>   
> >>> Hi Daniel,
> >>> 
> >>> On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:  
>  On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:  
> > Hi Daniel,
> > 
> > Thank you for the patch.
> > 
> > With this applied, I get
> > 
> > make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> > 
> > SPHINX  htmldocs -->
> > file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
> >   include/uapi/linux/videodev2.h
> > 
> > Running Sphinx v1.3.1
> > 
> > Extension error:
> > Could not import extension kfigure (exception: cannot import name 
> > patches)
> > make[2]: ***
> > [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> > htmldocs] Error 1 make[1]: ***
> > [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> > make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
> > *** [Makefile:152: sub-make] Error 2
> > 
> > sphinx.directive.patches got introduced in Sphinx 1.4. If you want to 
> > bump
> > the minimum required version I think a notice is needed.  
>  
>  Ugh. But this also goes completely over my head, no idea whether we
>  must require sphinx 1.4 (it was released Mar 28, 2016), or whether
>  there's some way to work around this ... Halp?  
> >>> 
> >>> I'm not a Sphinx expert so I don't know, but what I can tell is that 
> >>> copying 
> >>> the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying 
> >>> kfigure.py to import it from there fixes the build. There's thus no extra 
> >>> depencency on Sphinx 1.4 (or newer).
> >>> 
> >>> I'm not sure we want to set a precedent by copying part of the Sphinx 
> >>> source 
> >>> code to the kernel tree (or inlining the single small function that the 
> >>> module 
> >>> provides), and I'll let someone more knowledgeable than me decide how to 
> >>> proceed.  
> >> 
> >> 
> >> Aargh ... we need virtualenv! For interim something like the following
> >> might help. In file Documentation/sphinx/kfigure.py edit the imports
> >> 
> >> ...
> >> from docutils.parsers.rst.directives import images
> >> try:
> >>from sphinx.directives.patches import Figure
> >> except ImportError:
> >>Figure = images.Figure 
> >> ...
> >> 
> >> And fix the class definition, so it use 'Figure' and no
> >> longer 'patch.Figure'::
> >> 
> >> ...
> >> -class KernelFigure(patches.Figure):
> >> +class KernelFigure(Figure):
> >> ...
> >> 
> >> Sorry that I have not yet the time to send you a decent and tested
> >> patch. Do you like to test my suggestion? / thanks!  
> > 
> > I'll give it a shot at implementing it, but I can't (easily at least) test
> > on sphinx 1.3.  
> 
> You can test it with virtualenv  
> https://virtualenv.pypa.io/en/stable/userguide/
> 
> In short:
> 
> $ cd kernel-src
> $ virtualenv myenv
> $ source myenv/bin/activate
> $ pip install 'Sphinx==1.3.1'
> $ make 

Hmm... at least here, building docs-next with Sphinx 1.3.1 inside a
virtualenv is broken:

writing output... [ 16%] media/intro
Exception occurred:
  File 
"/devel/v4l/patchwork/myenv-1.3/lib/python2.7/site-packages/docutils/writers/_html_base.py",
 line 671, in depart_document
assert not self.context, 'len(context) = %s' % len(self.context)
AssertionError: len(context) = 1
The full traceback has been saved in /tmp/sphinx-err-W7CPtT.log, if you want to 
report the issue to the developers.
Please also report this if it was a user error, so that a better error message 
can be provided next time.
A bug report can be filed in the tracker at 
. Thanks!
Documentation/Makefile.sphinx:69: recipe for target 'htmldocs' failed
make[1]: *** [htmldocs] Error 1
Makefile:1450: recipe for target 'htmldocs' failed
make: *** [htmldocs] Error 2

Perhaps it is time for us to move minimal requirements to 1.4?

Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 4/6] drm/doc: Consistent kerneldoc include order

[Daniel Vetter]

First overview text (if there is any), then headers (since generally
you want to start out with the data structures), then all the other
stuff with functions.

Most of this is pre-shpinx, since with the old docbook only the
overview stuff was pulled in directly. Everything else was put in a
per-section index, so include order didn't really matter.

Acked-by: Eric Anholt 
Signed-off-by: Daniel Vetter 
---
 Documentation/gpu/drm-internals.rst   |  6 +++---
 Documentation/gpu/drm-kms-helpers.rst | 22 +++---
 Documentation/gpu/drm-kms.rst | 24 
 Documentation/gpu/drm-mm.rst  | 24 
 4 files changed, 38 insertions(+), 38 deletions(-)

diff --git a/Documentation/gpu/drm-internals.rst 
b/Documentation/gpu/drm-internals.rst
index e35920db1f4c..29d6bf7bb1ac 100644
--- a/Documentation/gpu/drm-internals.rst
+++ b/Documentation/gpu/drm-internals.rst
@@ -140,12 +140,12 @@ Device Instance and Driver Handling
 .. kernel-doc:: drivers/gpu/drm/drm_drv.c
:doc: driver instance overview
 
-.. kernel-doc:: drivers/gpu/drm/drm_drv.c
-   :export:
-
 .. kernel-doc:: include/drm/drm_drv.h
:internal:
 
+.. kernel-doc:: drivers/gpu/drm/drm_drv.c
+   :export:
+
 Driver Load
 ---
 
diff --git a/Documentation/gpu/drm-kms-helpers.rst 
b/Documentation/gpu/drm-kms-helpers.rst
index 012b6ff3ec3f..050ebe81d256 100644
--- a/Documentation/gpu/drm-kms-helpers.rst
+++ b/Documentation/gpu/drm-kms-helpers.rst
@@ -37,10 +37,10 @@ Modeset Helper Reference for Common Vtables
 ===
 
 .. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
-   :internal:
+   :doc: overview
 
 .. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
-   :doc: overview
+   :internal:
 
 Atomic Modeset Helper Functions Reference
 =
@@ -84,27 +84,27 @@ Legacy CRTC/Modeset Helper Functions Reference
 Simple KMS Helper Reference
 ===
 
+.. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
+   :doc: overview
+
 .. kernel-doc:: include/drm/drm_simple_kms_helper.h
:internal:
 
 .. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
:export:
 
-.. kernel-doc:: drivers/gpu/drm/drm_simple_kms_helper.c
-   :doc: overview
-
 fbdev Helper Functions Reference
 
 
 .. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
:doc: fbdev helpers
 
-.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
-   :export:
-
 .. kernel-doc:: include/drm/drm_fb_helper.h
:internal:
 
+.. kernel-doc:: drivers/gpu/drm/drm_fb_helper.c
+   :export:
+
 Framebuffer CMA Helper Functions Reference
 ==
 
@@ -146,15 +146,15 @@ Bridge Helper Reference
 Panel Helper Reference
 ==
 
+.. kernel-doc:: drivers/gpu/drm/drm_panel.c
+   :doc: drm panel
+
 .. kernel-doc:: include/drm/drm_panel.h
:internal:
 
 .. kernel-doc:: drivers/gpu/drm/drm_panel.c
:export:
 
-.. kernel-doc:: drivers/gpu/drm/drm_panel.c
-   :doc: drm panel
-
 Display Port Helper Functions Reference
 ===
 
diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index bcf475f86e83..b11c537c73f4 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -152,12 +152,12 @@ panels are still in-flux and not really fully sorted out 
yet.
 KMS Core Structures and Functions
 =
 
-.. kernel-doc:: drivers/gpu/drm/drm_mode_config.c
-   :export:
-
 .. kernel-doc:: include/drm/drm_mode_config.h
:internal:
 
+.. kernel-doc:: drivers/gpu/drm/drm_mode_config.c
+   :export:
+
 Modeset Base Object Abstraction
 ===
 
@@ -170,12 +170,12 @@ Modeset Base Object Abstraction
 Atomic Mode Setting Function Reference
 ==
 
-.. kernel-doc:: drivers/gpu/drm/drm_atomic.c
-   :export:
-
 .. kernel-doc:: include/drm/drm_atomic.h
:internal:
 
+.. kernel-doc:: drivers/gpu/drm/drm_atomic.c
+   :export:
+
 CRTC Abstraction
 
 
@@ -200,12 +200,12 @@ Frame Buffer Abstraction
 Frame Buffer Functions Reference
 
 
-.. kernel-doc:: drivers/gpu/drm/drm_framebuffer.c
-   :export:
-
 .. kernel-doc:: include/drm/drm_framebuffer.h
:internal:
 
+.. kernel-doc:: drivers/gpu/drm/drm_framebuffer.c
+   :export:
+
 DRM Format Handling
 ===
 
@@ -508,8 +508,8 @@ operation handler.
 Vertical Blanking and Interrupt Handling Functions Reference
 
 
-.. kernel-doc:: drivers/gpu/drm/drm_irq.c
-   :export:
-
 .. kernel-doc:: include/drm/drm_irq.h
:internal:
+
+.. kernel-doc:: drivers/gpu/drm/drm_irq.c
+   :export:
diff --git a/Documentation/gpu/drm-mm.rst b/Documentation/gpu/drm-mm.rst
index fd35998acefc..d47cbe3a9ef3 100644
--- a/Documentation/gpu/drm-mm.rst

[PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Daniel Vetter]

From: Markus Heiser 

This patch brings scalable figure, image handling and a concept to
embed *render* markups:

* DOT (http://www.graphviz.org)
* SVG

For image handling use the 'image' replacement::

.. kernel-image::  svg_image.svg
   :alt:simple SVG image

For figure handling use the 'figure' replacement::

.. kernel-figure::  svg_image.svg
   :alt:simple SVG image

   SVG image example

Embed *render* markups (or languages) like Graphviz's **DOT** is
provided by the *render* directive.::

  .. kernel-render:: DOT
 :alt: foobar digraph
 :caption: Embedded **DOT** (Graphviz) code.

 digraph foo {
  "bar" -> "baz";
 }

The *render* directive is a concept to integrate *render* markups and
languages, yet supported markups:

* DOT: render embedded Graphviz's **DOT**
* SVG: render embedded Scalable Vector Graphics (**SVG**)

v2: s/DOC/DOT/ in a few places (by Daniel).

v3: Simplify stuff a bit (by Daniel):

- Remove path detection and setup/check code for that. In
  Documentation/media/Makefile we already simply use these tools,
  better to have one consolidated check if we want/need one. Also
  remove the convertsvg support, we require ImageMagick's convert
  already in the doc build, no need for a 2nd fallback.

- Use sphinx for depency tracking, remove hand-rolled version.

- Forward stderr from dot and convert, otherwise debugging issues with
  the diagrams is impossible.

v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
Implement Markus suggestion for backwards compatability with earlier
releases. Laurent reported this, running sphinx 1.3. Solution entirely
untested.

Cc: Jonathan Corbet 
Cc: linux-doc@vger.kernel.org
Cc: Jani Nikula 
Cc: Mauro Carvalho Chehab 
Cc: Markus Heiser 
Cc: Laurent Pinchart 
Signed-off-by: Markus Heiser  (v1)
Signed-off-by: Daniel Vetter 
---
 Documentation/conf.py |   2 +-
 Documentation/doc-guide/hello.dot |   3 +
 Documentation/doc-guide/sphinx.rst|  90 ++-
 Documentation/doc-guide/svg_image.svg |  10 +
 Documentation/process/changes.rst |   7 +-
 Documentation/sphinx/kfigure.py   | 444 ++
 6 files changed, 550 insertions(+), 6 deletions(-)
 create mode 100644 Documentation/doc-guide/hello.dot
 create mode 100644 Documentation/doc-guide/svg_image.svg
 create mode 100644 Documentation/sphinx/kfigure.py

diff --git a/Documentation/conf.py b/Documentation/conf.py
index f6823cf01275..e3f537ce2935 100644
--- a/Documentation/conf.py
+++ b/Documentation/conf.py
@@ -34,7 +34,7 @@ from load_config import loadConfig
 # Add any Sphinx extension module names here, as strings. They can be
 # extensions coming with Sphinx (named 'sphinx.ext.*') or your custom
 # ones.
-extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain']
+extensions = ['kerneldoc', 'rstFlatTable', 'kernel_include', 'cdomain', 
'kfigure']
 
 # The name of the math extension changed on Sphinx 1.4
 if major == 1 and minor > 3:
diff --git a/Documentation/doc-guide/hello.dot 
b/Documentation/doc-guide/hello.dot
new file mode 100644
index ..504621dfc595
--- /dev/null
+++ b/Documentation/doc-guide/hello.dot
@@ -0,0 +1,3 @@
+graph G {
+  Hello -- World
+}
diff --git a/Documentation/doc-guide/sphinx.rst 
b/Documentation/doc-guide/sphinx.rst
index 532d65b70500..b902744ce7dd 100644
--- a/Documentation/doc-guide/sphinx.rst
+++ b/Documentation/doc-guide/sphinx.rst
@@ -34,8 +34,10 @@ format-specific subdirectories under 
``Documentation/output``.
 
 To generate documentation, Sphinx (``sphinx-build``) must obviously be
 installed. For prettier HTML output, the Read the Docs Sphinx theme
-(``sphinx_rtd_theme``) is used if available. For PDF output, ``rst2pdf`` is 
also
-needed. All of these are widely available and packaged in distributions.
+(``sphinx_rtd_theme``) is used if available. For PDF output you'll also need
+``XeLaTeX`` and CairoSVG (http://cairosvg.org) or alternatively ``convert(1)``
+from ImageMagick (https://www.imagemagick.org). All of these are widely
+available and packaged in distributions.
 
 To pass extra options to Sphinx, you can use the ``SPHINXOPTS`` make
 variable. For example, use ``make SPHINXOPTS=-v htmldocs`` to get more verbose
@@ -232,3 +234,87 @@ Rendered as:
   * .. _`last row`:
 
 - column 3
+
+
+Figures & Images
+
+
+If you want to add an image, you should use the ``kernel-figure`` and
+``kernel-image`` directives. E.g. to insert a figure with a scalable
+image format use SVG::
+
+.. kernel-figure::  svg_image.svg
+   :alt:simple SVG image
+
+   SVG image example
+
+.. kernel-figure::  svg_image.svg
+   :alt:simple SVG image
+
+   SVG image example
+
+The kernel figure (and image) 

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Laurent Pinchart]

Hi Daniel,

Thank you for the patch.

On Thursday 02 Mar 2017 16:16:34 Daniel Vetter wrote:
> From: Markus Heiser 
> 
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
> 
> * DOT (http://www.graphviz.org)
> * SVG
> 
> For image handling use the 'image' replacement::
> 
> .. kernel-image::  svg_image.svg
> 
>:alt:simple SVG image
> 
> For figure handling use the 'figure' replacement::
> 
> .. kernel-figure::  svg_image.svg
> 
>:alt:simple SVG image
> 
>SVG image example
> 
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
> 
>   .. kernel-render:: DOT
> 
>  :alt: foobar digraph
>  :caption: Embedded **DOT** (Graphviz) code.
> 
>  digraph foo {
>   "bar" -> "baz";
>  }
> 
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
> 
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
> 
> v2: s/DOC/DOT/ in a few places (by Daniel).
> 
> v3: Simplify stuff a bit (by Daniel):
> 
> - Remove path detection and setup/check code for that. In
>   Documentation/media/Makefile we already simply use these tools,
>   better to have one consolidated check if we want/need one. Also
>   remove the convertsvg support, we require ImageMagick's convert
>   already in the doc build, no need for a 2nd fallback.
> 
> - Use sphinx for depency tracking, remove hand-rolled version.
> 
> - Forward stderr from dot and convert, otherwise debugging issues with
>   the diagrams is impossible.
> 
> v3: Only sphinx 1.4 (released in Mar 2016) has patches.Figure.
> Implement Markus suggestion for backwards compatability with earlier
> releases. Laurent reported this, running sphinx 1.3. Solution entirely
> untested.

Tested-by: Laurent Pinchart 

However, you might want to explicitly check the Sphinx version, as done in 
Documentation/conf.py:

import sphinx

# Get Sphinx version
major, minor, patch = map(int, sphinx.__version__.split("."))

if major == 1 and minor > 3:
...

This will make it clearer what version we depend on, and easier to remove dead 
code when we'll bump the minimum version requirements.

> Cc: Jonathan Corbet 
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula 
> Cc: Mauro Carvalho Chehab 
> Cc: Markus Heiser 
> Cc: Laurent Pinchart 
> Signed-off-by: Markus Heiser  (v1)
> Signed-off-by: Daniel Vetter 
> ---
>  Documentation/conf.py |   2 +-
>  Documentation/doc-guide/hello.dot |   3 +
>  Documentation/doc-guide/sphinx.rst|  90 ++-
>  Documentation/doc-guide/svg_image.svg |  10 +
>  Documentation/process/changes.rst |   7 +-
>  Documentation/sphinx/kfigure.py   | 444 ++
>  6 files changed, 550 insertions(+), 6 deletions(-)
>  create mode 100644 Documentation/doc-guide/hello.dot
>  create mode 100644 Documentation/doc-guide/svg_image.svg
>  create mode 100644 Documentation/sphinx/kfigure.py

-- 
Regards,

Laurent Pinchart

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Markus Heiser]

Hi Daniel, Laurent

Am 02.03.2017 um 15:14 schrieb Laurent Pinchart 
:

> Hi Daniel,
> 
> On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
>> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
>>> Hi Daniel,
>>> 
>>> Thank you for the patch.
>>> 
>>> With this applied, I get
>>> 
>>> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
>>> 
>>>  SPHINX  htmldocs -->
>>>  file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
>>>include/uapi/linux/videodev2.h
>>> 
>>> Running Sphinx v1.3.1
>>> 
>>> Extension error:
>>> Could not import extension kfigure (exception: cannot import name patches)
>>> make[2]: ***
>>> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
>>> htmldocs] Error 1 make[1]: ***
>>> [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
>>> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
>>> *** [Makefile:152: sub-make] Error 2
>>> 
>>> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
>>> the minimum required version I think a notice is needed.
>> 
>> Ugh. But this also goes completely over my head, no idea whether we
>> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
>> there's some way to work around this ... Halp?
> 
> I'm not a Sphinx expert so I don't know, but what I can tell is that copying 
> the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying 
> kfigure.py to import it from there fixes the build. There's thus no extra 
> depencency on Sphinx 1.4 (or newer).
> 
> I'm not sure we want to set a precedent by copying part of the Sphinx source 
> code to the kernel tree (or inlining the single small function that the 
> module 
> provides), and I'll let someone more knowledgeable than me decide how to 
> proceed.


Aargh ... we need virtualenv! For interim something like the following
might help. In file Documentation/sphinx/kfigure.py edit the imports

...
from docutils.parsers.rst.directives import images
try:
from sphinx.directives.patches import Figure
except ImportError:
Figure = images.Figure 
...

And fix the class definition, so it use 'Figure' and no
longer 'patch.Figure'::

...
-class KernelFigure(patches.Figure):
+class KernelFigure(Figure):
...

Sorry that I have not yet the time to send you a decent and tested
patch. Do you like to test my suggestion? / thanks!

-- Markus 
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 3/6] drm/doc: Add KMS overview graphs

[Daniel Vetter]

On Thu, Mar 02, 2017 at 04:34:18PM +0200, Laurent Pinchart wrote:
> Hi Daniel,
> 
> Thank you for the patch.
> 
> On Tuesday 28 Feb 2017 18:13:16 Daniel Vetter wrote:
> > Oh, the shiny and pretties!
> > 
> > Cc: Laurent Pinchart 
> > Signed-off-by: Daniel Vetter 
> 
> Overall this looks good to me, please see below for a few minor issues.
> 
> > ---
> >  Documentation/gpu/drm-kms-helpers.rst |   4 ++
> >  Documentation/gpu/drm-kms.rst | 132 +++
> >  2 files changed, 136 insertions(+)
> > 
> > diff --git a/Documentation/gpu/drm-kms-helpers.rst
> > b/Documentation/gpu/drm-kms-helpers.rst index 03040aa14fe8..012b6ff3ec3f
> > 100644
> > --- a/Documentation/gpu/drm-kms-helpers.rst
> > +++ b/Documentation/gpu/drm-kms-helpers.rst
> > @@ -114,6 +114,8 @@ Framebuffer CMA Helper Functions Reference
> >  .. kernel-doc:: drivers/gpu/drm/drm_fb_cma_helper.c
> > 
> > :export:
> > +.. _drm_bridges:
> > +
> >  Bridges
> >  ===
> > 
> > @@ -139,6 +141,8 @@ Bridge Helper Reference
> >  .. kernel-doc:: drivers/gpu/drm/drm_bridge.c
> > 
> > :export:
> > +.. _drm_panel_helper:
> > +
> >  Panel Helper Reference
> >  ==
> > 
> > diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> > index 4d4068855ec4..87d8162c9a34 100644
> > --- a/Documentation/gpu/drm-kms.rst
> > +++ b/Documentation/gpu/drm-kms.rst
> > @@ -17,6 +17,138 @@ be setup by initializing the following fields.
> > 
> >  Mode Configuration
> 
> Not part of this patch, but this line doesn't feel like it's where it shoul 
> dbe.

Yeah that's an accident from a previous patch, I've removed it.
> 
> > +Overview
> > +
> > +
> > +.. kernel-render:: DOT
> > +   :alt: KMS Display Pipeline
> > +   :caption: KMS Display Pipeline Overview
> > +
> > +   digraph "KMS" {
> > +  node [shape=box]
> > +
> > +  subgraph cluster_static {
> > +  style=dashed
> > +  label="Static Objects"
> > +
> > +  node [bgcolor=grey style=filled]
> > +  "drm_plane A" -> "drm_crtc"
> > +  "drm_plane B" -> "drm_crtc"
> > +  "drm_crtc" -> "drm_encoder A"
> > +  "drm_crtc" -> "drm_encoder B"
> > +  }
> > +
> > +  subgraph cluster_user_created {
> > +  style=dashed
> > +  label="Userspace-Created"
> > +
> > +  node [shape=oval]
> > +  "drm_framebuffer 1" -> "drm_plane A"
> > +  "drm_framebuffer 2" -> "drm_plane B"
> > +  }
> > +
> > +  subgraph cluster_connector {
> > +  style=dashed
> > +  label="Hotpluggable"
> > +
> > +  "drm_encoder A" -> "drm_connector A"
> > +  "drm_encoder B" -> "drm_connector B"
> > +  }
> > +   }
> > +
> > +The basic object structure KMS presents to userspace is fairly simple.
> > +Framebuffers (represented by :c:type:`struct drm_framebuffer
> > `, +see `Frame Buffer Abstraction`_) feed into planes.
> > Multiple (or just one, or +even no) planes
> 
> I'd say "One or multiple (or even no) planes", but that's up to you.
> 
> > feed their pixel data into a
> > CRTC (represented by +:c:type:`struct drm_crtc `, see `CRTC
> > Abstraction`_) for blending. The +precise blending step is explained in
> > more detail in `Plane Composition +Properties`_ and related chapter.
> 
> s/chapter/chapters/ ? Or /related chapter/the related chapter/ ?
> 
> > +
> > +For the output routing the first step are Encoders (represented by
> 
> s/are/is/
> 
> > +:c:type:`struct drm_encoder `, see `Encoder Abstraction`_).
> > Those +are really just internal artifacts of the helper libraries used to
> > implement KMS +drivers. But unfortunately encoders have been exposed to
> 
> s/But u/U/
> 
> (http://blog.oxforddictionaries.com/2012/01/can-i-start-a-sentence-with-a-conjunction/)
> 
> I'd actually move the sentence towards the end of the paragraph and modify it 
> to
> 
> "Unfortunately connectors have been exposed to userspace, so we can't remove 
> them at this point."
> 
> or something similar.
> 
> > userspace. Besides that +they make it unecessarily more complicated for
> > userspace to figure out which +connections between a CRTC and a connector,
> 
> I think you're missing a verb. s/which connections/which connections are 
> possible/ ?
> 
> > and what kind of cloning is +supported, they serve no purpose in the
> > userspace API. Futhermore the exposed +restrictions are often wrongly set
> > by drivers, and in many cases not powerful +enough to express the real
> > restrictions.
> 
> I'd move the "But" sentence here, and possible start a new paragraph.
> 
> > A CRTC can be connected to multiple +Encoders, but for an
> 
> s/but/and/
> 
> > active CRTC there must be at least one encoder.
> > +
> > +The final, and real, endpoint in the display chain is the connector
> > (represented +by :c:type:`struct drm_connector `, see
> > `Connector +Abstraction`_). Connectors can have different possible
> > 

Re: [PATCH] drm/doc: atomic overview, with graph

[Laurent Pinchart]

Hi Daniel,

Thank you for the patch.

On Thursday 02 Mar 2017 08:19:58 Daniel Vetter wrote:
> I want to split up a few more things and document some details better
> (like how exactly to subclass drm_atomic_state). And maybe also split
> up the helpers a bit per-topic, but this should be a ok-ish start for
> better atomic overview.
> 
> v2: Spelling and clarifications (Eric).
> 
> v3: Implement suggestion from Gabriel to fix the graph.
> 
> Cc: Gabriel Krisman Bertazi 
> Acked-by: Eric Anholt 
> Cc: Eric Anholt 
> Cc: Laurent Pinchart 
> Cc: Harry Wentland 
> Signed-off-by: Daniel Vetter 
> ---
>  Documentation/gpu/drm-kms-helpers.rst |  2 +
>  Documentation/gpu/drm-kms.rst | 84 +++-
>  2 files changed, 85 insertions(+), 1 deletion(-)
> 
> diff --git a/Documentation/gpu/drm-kms-helpers.rst
> b/Documentation/gpu/drm-kms-helpers.rst index 050ebe81d256..ac53c0b893f6
> 100644
> --- a/Documentation/gpu/drm-kms-helpers.rst
> +++ b/Documentation/gpu/drm-kms-helpers.rst
> @@ -42,6 +42,8 @@ Modeset Helper Reference for Common Vtables
>  .. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
> 
> :internal:
> +.. _drm_atomic_helper:
> +
>  Atomic Modeset Helper Functions Reference
>  =
> 
> diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
> index a504d9ee4d94..eb9d29865c41 100644
> --- a/Documentation/gpu/drm-kms.rst
> +++ b/Documentation/gpu/drm-kms.rst
> @@ -189,8 +189,90 @@ multiple times to different objects using
> :c:func:`drm_object_attach_property() .. kernel-doc::
> drivers/gpu/drm/drm_mode_object.c
> 
> :export:
> +Atomic Mode Setting
> +===
> +
> +
> +.. kernel-render:: DOT
> +   :alt: Mode Objects and Properties
> +   :caption: Mode Objects and Properties
> +
> +   digraph {
> +  node [shape=box]
> +
> +  subgraph cluster_state {
> +  style=dashed
> +  label="Free-standing state"
> +
> +  "drm_atomic_state" -> "duplicated drm_plane_state A"
> +  "drm_atomic_state" -> "duplicated drm_plane_state B"
> +  "drm_atomic_state" -> "duplicated drm_crtc_state"
> +  "drm_atomic_state" -> "duplicated drm_connector_state"
> +  "drm_atomic_state" -> "duplicated driver private state"
> +  }
> +
> +  subgraph cluster_current {
> +  style=dashed
> +  label="Current state"
> +
> +  "drm_device" -> "drm_plane A"
> +  "drm_device" -> "drm_plane B"
> +  "drm_device" -> "drm_crtc"
> +  "drm_device" -> "drm_connector"
> +  "drm_device" -> "driver private object"
> +
> +  "drm_plane A" -> "drm_plane_state A"
> +  "drm_plane B" -> "drm_plane_state B"
> +  "drm_crtc" -> "drm_crtc_state"
> +  "drm_connector" -> "drm_connector_state"
> +  "driver private object" -> "driver private state"
> +  }
> +
> +  "drm_atomic_state" -> "drm_device" [label="atomic_commit"]
> +  "duplicated drm_plane_state A" -> "drm_device"[style=invis]
> +   }
> +
> +Atomic provides transactional modeset (including planes) updates, but a
> +bit differently from the usual transactional approach of try-commit and
> +rollback:
> +
> +- Firstly, no hardware changes are allowed when the commit would fail. This
> +  allows us to implement the DRM_MODE_ATOMIC_TEST_ONLY mode, which allows
> +  userspace to explore whether certain configurations would work or not. +
> +- This would still allow setting and rollback of just the software state,
> +  simplifying conversion of existing drivers. But auditing drivers for
> +  correctness of the atomic_check code becomes really hard with that:
> Rolling
> +  back changes in data structures all over the place is hard to get right.
> +
> +- Lastly, for backwards compatibility and to support all use-cases, atomic

s/backwards/backward/ (see my comment to patch 3/6, and not that the margin is 
now slightly larger :-))

> +  updates need to be incremental and be able to execute in parallel.
> Hardware
> +  doesn't always allow it, but where possible plane updates on
> different CRTCs
> +  should not interfere, and not get stalled due to output routing changing
> on
> +  different CRTCs.
> +
> +Taken all together there's two consequences for the atomic design:
> +
> +- The overall state is split up into per-object state structures:
> +  :c:type:`struct drm_plane_state ` for planes,
> :c:type:`struct +  drm_crtc_state ` for CRTCs and
> :c:type:`struct
> +  drm_connector_state 

> for connectors. These are the
> only +  objects with userspace-visible and settable state. For internal
> state drivers +  can subclass these structures through embeddeding, or add
> entirely new state +  structures for their globally shared hardware
> functions.
> +
> +- An atomic 

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Laurent Pinchart]

Hi Daniel,

On Thursday 02 Mar 2017 14:54:32 Daniel Vetter wrote:
> On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart wrote:
> > Hi Daniel,
> > 
> > Thank you for the patch.
> > 
> > With this applied, I get
> > 
> > make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
> > 
> >   SPHINX  htmldocs -->
> >   file:///home/laurent/src/iob/renesas/linux64/Documentation/output PARSE
> > include/uapi/linux/videodev2.h
> > 
> > Running Sphinx v1.3.1
> > 
> > Extension error:
> > Could not import extension kfigure (exception: cannot import name patches)
> > make[2]: ***
> > [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70:
> > htmldocs] Error 1 make[1]: ***
> > [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] Error 2
> > make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64' make:
> > *** [Makefile:152: sub-make] Error 2
> > 
> > sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump
> > the minimum required version I think a notice is needed.
> 
> Ugh. But this also goes completely over my head, no idea whether we
> must require sphinx 1.4 (it was released Mar 28, 2016), or whether
> there's some way to work around this ... Halp?

I'm not a Sphinx expert so I don't know, but what I can tell is that copying 
the patches.py from Sphinx 1.4 to Documentation/sphinx/ and modifying 
kfigure.py to import it from there fixes the build. There's thus no extra 
depencency on Sphinx 1.4 (or newer).

I'm not sure we want to set a precedent by copying part of the Sphinx source 
code to the kernel tree (or inlining the single small function that the module 
provides), and I'll let someone more knowledgeable than me decide how to 
proceed.

-- 
Regards,

Laurent Pinchart

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Daniel Vetter]

On Thu, Mar 2, 2017 at 1:26 PM, Laurent Pinchart
 wrote:
> Hi Daniel,
>
> Thank you for the patch.
>
> With this applied, I get
>
> make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
>   SPHINX  htmldocs --> 
> file:///home/laurent/src/iob/renesas/linux64/Documentation/output
>   PARSE   include/uapi/linux/videodev2.h
> Running Sphinx v1.3.1
>
> Extension error:
> Could not import extension kfigure (exception: cannot import name patches)
> make[2]: *** 
> [/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70: 
> htmldocs] Error 1
> make[1]: *** [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] 
> Error 2
> make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64'
> make: *** [Makefile:152: sub-make] Error 2
>
> sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump the
> minimum required version I think a notice is needed.

Ugh. But this also goes completely over my head, no idea whether we
must require sphinx 1.4 (it was released Mar 28, 2016), or whether
there's some way to work around this ... Halp?
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Laurent Pinchart]

Hi Daniel,

Thank you for the patch.

With this applied, I get

make[1]: Entering directory '/home/laurent/src/iob/renesas/linux64'
  SPHINX  htmldocs --> 
file:///home/laurent/src/iob/renesas/linux64/Documentation/output
  PARSE   include/uapi/linux/videodev2.h
Running Sphinx v1.3.1

Extension error:
Could not import extension kfigure (exception: cannot import name patches)
make[2]: *** 
[/home/laurent/src/iob/renesas/linux/Documentation/Makefile.sphinx:70: 
htmldocs] Error 1
make[1]: *** [/home/laurent/src/iob/renesas/linux/Makefile:1453: htmldocs] 
Error 2
make[1]: Leaving directory '/home/laurent/src/iob/renesas/linux64'
make: *** [Makefile:152: sub-make] Error 2

sphinx.directive.patches got introduced in Sphinx 1.4. If you want to bump the
minimum required version I think a notice is needed.

On Tuesday 28 Feb 2017 18:13:15 Daniel Vetter wrote:
> From: Markus Heiser 
> 
> This patch brings scalable figure, image handling and a concept to
> embed *render* markups:
> 
> * DOT (http://www.graphviz.org)
> * SVG
> 
> For image handling use the 'image' replacement::
> 
> .. kernel-image::  svg_image.svg
> 
>:alt:simple SVG image
> 
> For figure handling use the 'figure' replacement::
> 
> .. kernel-figure::  svg_image.svg
> 
>:alt:simple SVG image
> 
>SVG image example
> 
> Embed *render* markups (or languages) like Graphviz's **DOT** is
> provided by the *render* directive.::
> 
>   .. kernel-render:: DOT
> 
>  :alt: foobar digraph
>  :caption: Embedded **DOT** (Graphviz) code.
> 
>  digraph foo {
>   "bar" -> "baz";
>  }
> 
> The *render* directive is a concept to integrate *render* markups and
> languages, yet supported markups:
> 
> * DOT: render embedded Graphviz's **DOT**
> * SVG: render embedded Scalable Vector Graphics (**SVG**)
> 
> v2: s/DOC/DOT/ in a few places (by Daniel).
> 
> v3: Simplify stuff a bit (by Daniel):
> 
> - Remove path detection and setup/check code for that. In
>   Documentation/media/Makefile we already simply use these tools,
>   better to have one consolidated check if we want/need one. Also
>   remove the convertsvg support, we require ImageMagick's convert
>   already in the doc build, no need for a 2nd fallback.
> 
> - Use sphinx for depency tracking, remove hand-rolled version.
> 
> - Forward stderr from dot and convert, otherwise debugging issues with
>   the diagrams is impossible.
> 
> Cc: Jonathan Corbet 
> Cc: linux-doc@vger.kernel.org
> Cc: Jani Nikula 
> Cc: Mauro Carvalho Chehab 
> Signed-off-by: Markus Heiser  (v1)
> Signed-off-by: Daniel Vetter 
> ---
>  Documentation/conf.py |   2 +-
>  Documentation/doc-guide/hello.dot |   3 +
>  Documentation/doc-guide/sphinx.rst|  90 ++-
>  Documentation/doc-guide/svg_image.svg |  10 +
>  Documentation/process/changes.rst |   7 +-
>  Documentation/sphinx/kfigure.py   | 442 ++
>  6 files changed, 548 insertions(+), 6 deletions(-)
>  create mode 100644 Documentation/doc-guide/hello.dot
>  create mode 100644 Documentation/doc-guide/svg_image.svg
>  create mode 100644 Documentation/sphinx/kfigure.py

-- 
Regards,

Laurent Pinchart

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 11/32] KVM: MIPS: Add VZ capability

[Paolo Bonzini]

On 02/03/2017 10:36, James Hogan wrote:
>  - KVM_VM_MIPS_DEFAULT = 2
> 
>This will provide the best available KVM implementation (even on
>older kernels), preferring hardware assisted virtualization over trap
>& emulate. The KVM_CAP_MIPS_VZ capability should always be checked
>against known values to determine what type of implementation was
>chosen.
> 
> This is designed to allow the desired implementation (T vs VZ) to be
> potentially chosen at runtime rather than being fixed in the kernel
> configuration.

Can the same kernel run on both TE and VZ?  If not, I'm not sure that
KVM_VM_MIPS_DEFAULT is a good idea.

Paolo
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 6/6] drm/doc: atomic overview, with graph

[Daniel Vetter]

On Wed, Mar 01, 2017 at 02:42:02PM -0300, Gabriel Krisman Bertazi wrote:
> Daniel Vetter  writes:
> 
> > I've spent quite some time trying to beat DOT into submission, this is the
> > best I can do. The FIXME really is just a hint for someone with more clue
> > to maybe make it better, or if not possible at all, what would look better
> > when doing a proper diagram with .svg or something like that.
> >
> > Assuming no one knows how to fix this, I'd still like to push it - it's
> > still better than nothing imo, you just need to look at the picture
> > full-screen.
> 
> If you add a hidden edge from any of the "duplicated" nodes to the
> drm_device node, the second cluster will be pushed to the third rank
> and do what I think you want.
> 
> Something like:
> 
>  "duplicated drm_plane_state A" -> "drm_device"[style=invis]
> 
> This is the result for me:
> 
> https://people.collabora.com/~krisman/atomic_modesetting.svg
> 
> I think it got a little better.

Yeah, much better, thanks for the suggestion. Still rather small-ish, but
it's a busy graph, so that can't be helped really.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 11/32] KVM: MIPS: Add VZ capability

[James Hogan]

Add a new KVM_CAP_MIPS_VZ capability, and in order to allow MIPS KVM to
support VZ without confusing old users (which expect the trap & emulate
implementation), define and start checking KVM_CREATE_VM type codes.

The codes available are:

 - KVM_VM_MIPS_TE = 0

   This is the current value expected from the user, and will create a
   VM using trap & emulate in user mode, confined to the user mode
   address space. This may in future become unavailable if the kernel is
   only configured to support VZ, in which case the EINVAL error will be
   returned.

 - KVM_VM_MIPS_VZ = 1

   This can be provided when the KVM_CAP_MIPS_VZ capability is available
   to create a VM using VZ, with a fully virtualized guest virtual
   address space. If VZ support is unavailable in the kernel, the EINVAL
   error will be returned (although old kernels without the
   KVM_CAP_MIPS_VZ capability may well succeed and create a trap &
   emulate VM).

 - KVM_VM_MIPS_DEFAULT = 2

   This will provide the best available KVM implementation (even on
   older kernels), preferring hardware assisted virtualization over trap
   & emulate. The KVM_CAP_MIPS_VZ capability should always be checked
   against known values to determine what type of implementation was
   chosen.

This is designed to allow the desired implementation (T vs VZ) to be
potentially chosen at runtime rather than being fixed in the kernel
configuration.

Signed-off-by: James Hogan 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Ralf Baechle 
Cc: Jonathan Corbet 
Cc: linux-m...@linux-mips.org
Cc: k...@vger.kernel.org
Cc: linux-doc@vger.kernel.org
---
 Documentation/virtual/kvm/api.txt | 38 +++-
 arch/mips/kvm/mips.c  |  9 -
 include/uapi/linux/kvm.h  |  6 +-
 3 files changed, 52 insertions(+), 1 deletion(-)

diff --git a/Documentation/virtual/kvm/api.txt 
b/Documentation/virtual/kvm/api.txt
index 069450938b79..bd54d7a30e37 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -115,12 +115,21 @@ will access the virtual machine's physical address space; 
offset zero
 corresponds to guest physical address zero.  Use of mmap() on a VM fd
 is discouraged if userspace memory allocation (KVM_CAP_USER_MEMORY) is
 available.
-You most certainly want to use 0 as machine type.
+You probably want to use 0 as machine type.
 
 In order to create user controlled virtual machines on S390, check
 KVM_CAP_S390_UCONTROL and use the flag KVM_VM_S390_UCONTROL as
 privileged user (CAP_SYS_ADMIN).
 
+To use hardware assisted virtualization on MIPS (VZ ASE) rather than
+the default trap & emulate implementation (which changes the virtual
+memory layout to fit in user mode), check KVM_CAP_MIPS_VZ and use the
+flag KVM_VM_MIPS_VZ.
+
+To use the best available virtualization type on MIPS, use the flag
+KVM_VM_MIPS_DEFAULT and check KVM_CAP_MIPS_VZ on the VM after creation
+to determine exactly which type was chosen.
+
 
 4.3 KVM_GET_MSR_INDEX_LIST
 
@@ -4143,3 +4152,30 @@ This capability, if KVM_CHECK_EXTENSION indicates that 
it is
 available, means that that the kernel can support guests using the
 hashed page table MMU defined in Power ISA V3.00 (as implemented in
 the POWER9 processor), including in-memory segment tables.
+
+8.5 KVM_CAP_MIPS_VZ
+
+Architectures: mips
+
+This capability, if KVM_CHECK_EXTENSION on the main kvm handle indicates that
+it is available, means that full hardware assisted virtualization capabilities
+of the hardware are available for use through KVM. An appropriate
+KVM_VM_MIPS_* type must be passed to KVM_CREATE_VM to create a VM which
+utilises it.
+
+If KVM_CHECK_EXTENSION on a kvm VM handle indicates that this capability is
+available, it means that the VM is using full hardware assisted virtualization
+capabilities of the hardware. This is useful to check after creating a VM with
+KVM_VM_MIPS_DEFAULT.
+
+The value returned by KVM_CHECK_EXTENSION should be compared against known
+values (see below). All other values are reserved. This is to allow for the
+possibility of other hardware assisted virtualization implementations which
+may be incompatible with the MIPS VZ ASE.
+
+ 0: The trap & emulate implementation is in use to run guest code in user
+mode. Guest virtual memory segments are rearranged to fit the guest in the
+user mode address space.
+
+ 1: The MIPS VZ ASE is in use, providing full hardware assisted
+virtualization, including standard guest virtual memory segments.
diff --git a/arch/mips/kvm/mips.c b/arch/mips/kvm/mips.c
index 2a06015930eb..cd07ea27f336 100644
--- a/arch/mips/kvm/mips.c
+++ b/arch/mips/kvm/mips.c
@@ -105,6 +105,15 @@ void kvm_arch_check_processor_compat(void *rtn)
 
 int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 {
+   switch (type) {
+   case KVM_VM_MIPS_DEFAULT:
+   case KVM_VM_MIPS_TE:
+   

[PATCH 12/32] KVM: MIPS: Add 64BIT capability

[James Hogan]

Add a new KVM_CAP_MIPS_64BIT capability to indicate that 64-bit MIPS
guests are available and supported. In this case it should still be
possible to run 32-bit guest code. If not available it won't be possible
to run 64-bit guest code and the instructions may not be available, or
the kernel may not support full context switching of 64-bit registers.

Signed-off-by: James Hogan 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Ralf Baechle 
Cc: Jonathan Corbet 
Cc: linux-m...@linux-mips.org
Cc: k...@vger.kernel.org
Cc: linux-doc@vger.kernel.org
---
 Documentation/virtual/kvm/api.txt | 25 +
 include/uapi/linux/kvm.h  |  1 +
 2 files changed, 26 insertions(+), 0 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt 
b/Documentation/virtual/kvm/api.txt
index bd54d7a30e37..cfe9f8b04b5e 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -4179,3 +4179,28 @@ may be incompatible with the MIPS VZ ASE.
 
  1: The MIPS VZ ASE is in use, providing full hardware assisted
 virtualization, including standard guest virtual memory segments.
+
+8.4 KVM_CAP_MIPS_64BIT
+
+Architectures: mips
+
+This capability indicates the supported architecture type of the guest, i.e. 
the
+supported register and address width.
+
+The values returned when this capability is checked by KVM_CHECK_EXTENSION on a
+kvm VM handle correspond roughly to the CP0_Config.AT register field, and 
should
+be checked specifically against known values (see below). All other values are
+reserved.
+
+ 0: MIPS32 or microMIPS32.
+Both registers and addresses are 32-bits wide.
+It will only be possible to run 32-bit guest code.
+
+ 1: MIPS64 or microMIPS64 with access only to 32-bit compatibility segments.
+Registers are 64-bits wide, but addresses are 32-bits wide.
+64-bit guest code may run but cannot access MIPS64 memory segments.
+It will also be possible to run 32-bit guest code.
+
+ 2: MIPS64 or microMIPS64 with access to all address segments.
+Both registers and addresses are 64-bits wide.
+It will be possible to run 64-bit or 32-bit guest code.
diff --git a/include/uapi/linux/kvm.h b/include/uapi/linux/kvm.h
index f4b450d3c14b..6674b0ee1d82 100644
--- a/include/uapi/linux/kvm.h
+++ b/include/uapi/linux/kvm.h
@@ -889,6 +889,7 @@ struct kvm_ppc_resize_hpt {
 #define KVM_CAP_PPC_MMU_HASH_V3 135
 #define KVM_CAP_IMMEDIATE_EXIT 136
 #define KVM_CAP_MIPS_VZ 137
+#define KVM_CAP_MIPS_64BIT 138
 
 #ifdef KVM_CAP_IRQ_ROUTING
 
-- 
git-series 0.8.10
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 0/32] KVM: MIPS: Add VZ support

[James Hogan]

This series is based on kvm/next (Linus' 4.11 KVM merge) with my recent
hypercall series (which will likely be revised).
My hope is to take this series via the MIPS KVM tree for 4.12, with a
topic branch containing the MIPS architecture changes (patches 1-6).

This series implements basic support for the MIPS Virtualization Module
(generally known as VZ) in KVM, which adds a new "guest" processor mode
with a partial guest Cop0 state and guest TLB to complement the existing
"root" (host) modes & state, allowing for far fewer traps to the
hypervisor compared to the existing Trap & Emulate implementation of KVM
for MIPS, as well as fully virtualizing the guest virtual memory space
to run unmodified kernels.

The MIPS32 and MIPS64 VZ architecture documentation can be found here:
http://wiki.prplfoundation.org/w/images/f/fd/MD00846-2B-VZMIPS32-AFP-01.06.pdf
http://wiki.prplfoundation.org/w/images/f/fe/MD00847-2B-VZMIPS64-AFP-01.06.pdf

We primarily support the ImgTec P5600, P6600, and I6400 cores so far,
including the following VZ / guest hardware features:
- MIPS32 and MIPS64, r5 (VZ requires r5 or later) and r6
- TLBs with GuestID (IMG cores) or Root ASID Dealias (Octeon III)
- Shared physical root/guest TLB (IMG cores)
- FPU / MSA
- Cop0 timer (up to 1GHz for now due to soft timer limit)
- Segmentation control (EVA)
- Hardware page table walker (HTW) both for root and guest TLB

Limitations:
- Support for Cavium Octeon III will follow shortly in a separate series
- Perf counters and watch registers not exposed to guest in this series
  (the patches require some further cleanup)
- Guest microMIPS not supported yet (e.g. M5150)
- Guest XPA not supported yet (e.g. P5600)

We start with some preliminary MIPS changes in patches 1-6, including
some MAAR definition refactoring for XPA and some new guest capabilities
and accessors.

Patches 7-16 make preliminary VZ changes in KVM, including 64-bit MMIO
support, the VZ and 64BIT KVM capabilities, and new implementation
callbacks used by the VZ implementation.

Patches 17-22 get into some meatier KVM changes for VZ, such as
abstracting guest register access and adding VZ entry and TLB management
code.

Patches 23-24 add the bulk of the core VZ support and enable it in the
build system.

Patches 25-30 then go on to implement support for various optional
architectural features, such as segmentation control (EVA) and hardware
page table walkers (HTW). Context switching of the guest state may be
incomplete on certain cores without these patches if the features in
question can't be disabled in the guest context.

Finally patches 31-32 implement support for features that can be easily
disabled at runtime, namely the hardware guest timer (which can be
emulated in software before patch 31) and tracing of guest mode changes
using VZ's CP0_GuestCtl0.MC bit.

Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Ralf Baechle 
Cc: Steven Rostedt 
Cc: Ingo Molnar 
Cc: Paul Burton 
Cc: Jonathan Corbet 
Cc: linux-m...@linux-mips.org
Cc: k...@vger.kernel.org
Cc: linux-doc@vger.kernel.org

James Hogan (32):
  MIPS: Add defs & probing of UFR
  MIPS: Separate MAAR V bit into VL and VH for XPA
  MIPS: Probe guest CP0_UserLocal
  MIPS: Probe guest MVH
  MIPS: Add some missing guest CP0 accessors & defs
  MIPS: asm/tlb.h: Add UNIQUE_GUEST_ENTRYHI() macro
  KVM: MIPS/Emulate: De-duplicate MMIO emulation
  KVM: MIPS/Emulate: Implement 64-bit MMIO emulation
  KVM: MIPS: Update kvm_lose_fpu() for VZ
  KVM: MIPS: Extend counters & events for VZ GExcCodes
  KVM: MIPS: Add VZ capability
  KVM: MIPS: Add 64BIT capability
  KVM: MIPS: Init timer frequency from callback
  KVM: MIPS: Add callback to check extension
  KVM: MIPS: Add hardware_{enable,disable} callback
  KVM: MIPS: Add guest exit exception callback
  KVM: MIPS: Abstract guest CP0 register access for VZ
  KVM: MIPS/Entry: Update entry code to support VZ
  KVM: MIPS/TLB: Add VZ TLB management
  KVM: MIPS/Emulate: Update CP0_Compare emulation for VZ
  KVM: MIPS/Emulate: Drop CACHE emulation for VZ
  KVM: MIPS: Update exit handler for VZ
  KVM: MIPS: Implement VZ support
  KVM: MIPS: Add VZ support to build system
  KVM: MIPS/VZ: Support guest CP0_BadInstr[P]
  KVM: MIPS/VZ: Support guest CP0_[X]ContextConfig
  KVM: MIPS/VZ: Support guest segmentation control
  KVM: MIPS/VZ: Support guest hardware page table walker
  KVM: MIPS/VZ: Support guest load-linked bit
  KVM: MIPS/VZ: Emulate MAARs when necessary
  KVM: MIPS/VZ: Support hardware guest timer
  KVM: MIPS/VZ: Trace guest mode changes

 Documentation/virtual/kvm/api.txt|   81 +-
 arch/mips/include/asm/cpu-features.h |   10 +-
 arch/mips/include/asm/cpu-info.h |2 +-
 arch/mips/include/asm/cpu.h  |1 +-
 arch/mips/include/asm/kvm_host.h |  460 +++-
 arch/mips/include/asm/maar.h |   10 +-
 arch/mips/include/asm/mipsregs.h 

[PATCH 23/32] KVM: MIPS: Implement VZ support

[James Hogan]

Add the main support for the MIPS Virtualization ASE (A.K.A. VZ) to MIPS
KVM. The bulk of this work is in vz.c, with various new state and
definitions elsewhere.

Enough is implemented to be able to run on a minimal VZ core. Further
patches will fill out support for guest features which are optional or
can be disabled.

Signed-off-by: James Hogan 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Ralf Baechle 
Cc: Steven Rostedt 
Cc: Ingo Molnar 
Cc: Jonathan Corbet 
Cc: linux-m...@linux-mips.org
Cc: k...@vger.kernel.org
Cc: linux-doc@vger.kernel.org
---
 Documentation/virtual/kvm/api.txt |2 +-
 arch/mips/include/asm/cpu-info.h  |1 +-
 arch/mips/include/asm/kvm_host.h  |   42 +-
 arch/mips/kernel/time.c   |1 +-
 arch/mips/kvm/interrupt.h |5 +-
 arch/mips/kvm/mips.c  |6 +-
 arch/mips/kvm/mmu.c   |   20 +-
 arch/mips/kvm/tlb.c   |7 +-
 arch/mips/kvm/trace.h |   15 +-
 arch/mips/kvm/vz.c| 2380 ++-
 10 files changed, 2479 insertions(+), 0 deletions(-)
 create mode 100644 arch/mips/kvm/vz.c

diff --git a/Documentation/virtual/kvm/api.txt 
b/Documentation/virtual/kvm/api.txt
index cfe9f8b04b5e..996a388f05c5 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2075,6 +2075,7 @@ registers, find a list below:
   MIPS  | KVM_REG_MIPS_CP0_CONTEXT  | 64
   MIPS  | KVM_REG_MIPS_CP0_USERLOCAL| 64
   MIPS  | KVM_REG_MIPS_CP0_PAGEMASK | 32
+  MIPS  | KVM_REG_MIPS_CP0_PAGEGRAIN| 32
   MIPS  | KVM_REG_MIPS_CP0_WIRED| 32
   MIPS  | KVM_REG_MIPS_CP0_HWRENA   | 32
   MIPS  | KVM_REG_MIPS_CP0_BADVADDR | 64
@@ -2094,6 +2095,7 @@ registers, find a list below:
   MIPS  | KVM_REG_MIPS_CP0_CONFIG4  | 32
   MIPS  | KVM_REG_MIPS_CP0_CONFIG5  | 32
   MIPS  | KVM_REG_MIPS_CP0_CONFIG7  | 32
+  MIPS  | KVM_REG_MIPS_CP0_XCONTEXT | 64
   MIPS  | KVM_REG_MIPS_CP0_ERROREPC | 64
   MIPS  | KVM_REG_MIPS_CP0_KSCRATCH1| 64
   MIPS  | KVM_REG_MIPS_CP0_KSCRATCH2| 64
diff --git a/arch/mips/include/asm/cpu-info.h b/arch/mips/include/asm/cpu-info.h
index 4113796e0ef4..be3b4c25f335 100644
--- a/arch/mips/include/asm/cpu-info.h
+++ b/arch/mips/include/asm/cpu-info.h
@@ -110,6 +110,7 @@ struct cpuinfo_mips {
struct guest_info   guest;
unsigned intgtoffset_mask;
unsigned intguestid_mask;
+   unsigned intguestid_cache;
 } __attribute__((aligned(SMP_CACHE_BYTES)));
 
 extern struct cpuinfo_mips cpu_data[];
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 6c1b5bedf7e9..41d8abc1d6fa 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -10,6 +10,7 @@
 #ifndef __MIPS_KVM_HOST_H__
 #define __MIPS_KVM_HOST_H__
 
+#include 
 #include 
 #include 
 #include 
@@ -73,6 +74,11 @@
 #define KVM_COALESCED_MMIO_PAGE_OFFSET 1
 #define KVM_HALT_POLL_NS_DEFAULT 50
 
+#ifdef CONFIG_KVM_MIPS_VZ
+extern unsigned long GUESTID_MASK;
+extern unsigned long GUESTID_FIRST_VERSION;
+extern unsigned long GUESTID_VERSION_MASK;
+#endif
 
 
 /*
@@ -167,6 +173,8 @@ struct kvm_arch_memory_slot {
 struct kvm_arch {
/* Guest physical mm */
struct mm_struct gpa_mm;
+   /* Mask of CPUs needing GPA ASID flush */
+   cpumask_t asid_flush_mask;
 };
 
 #define N_MIPS_COPROC_REGS 32
@@ -224,6 +232,11 @@ struct mips_coproc {
 #define MIPS_CP0_CONFIG4_SEL   4
 #define MIPS_CP0_CONFIG5_SEL   5
 
+#define MIPS_CP0_GUESTCTL2 10
+#define MIPS_CP0_GUESTCTL2_SEL 5
+#define MIPS_CP0_GTOFFSET  12
+#define MIPS_CP0_GTOFFSET_SEL  7
+
 /* Resume Flags */
 #define RESUME_FLAG_DR (1<<0)  /* Reload guest nonvolatile state? */
 #define RESUME_FLAG_HOST   (1<<1)  /* Resume host? */
@@ -359,7 +372,20 @@ struct kvm_vcpu_arch {
/* Cache some mmu pages needed inside spinlock regions */
struct kvm_mmu_memory_cache mmu_page_cache;
 
+#ifdef CONFIG_KVM_MIPS_VZ
+   /* vcpu's vzguestid is different on each host cpu in an smp system */
+   u32 vzguestid[NR_CPUS];
+
+   /* wired guest TLB entries */
+   struct kvm_mips_tlb *wired_tlb;
+   unsigned int wired_tlb_limit;
+   unsigned int wired_tlb_used;
+#endif
+
+   /* Last CPU the VCPU state was loaded on */
int last_sched_cpu;
+   /* Last CPU the VCPU actually executed guest code on */
+   int last_exec_cpu;
 
/* WAIT executed */
int wait;
@@ -663,6 +689,7 @@ __BUILD_KVM_RW_HW(config4,32, MIPS_CP0_CONFIG,  
 4)
 __BUILD_KVM_RW_HW(config5,32, MIPS_CP0_CONFIG,   5)
 __BUILD_KVM_RW_HW(config6,32, MIPS_CP0_CONFIG,   6)
 __BUILD_KVM_RW_HW(config7,32, MIPS_CP0_CONFIG,   7)
+__BUILD_KVM_RW_HW(xcontext,   l,  

[PATCH 27/32] KVM: MIPS/VZ: Support guest segmentation control

[James Hogan]

Add support for VZ guest CP0_SegCtl0, CP0_SegCtl1, and CP0_SegCtl2
registers, as found on P5600 and P6600 cores. These guest registers need
initialising, context switching, and exposing via the KVM ioctl API when
they are present.

They also require the GVA -> GPA translation code for handling a GVA
root exception to be updated to interpret the segmentation registers and
decode the faulting instruction enough to detect EVA memory access
instructions.

Signed-off-by: James Hogan 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Ralf Baechle 
Cc: Jonathan Corbet 
Cc: linux-m...@linux-mips.org
Cc: k...@vger.kernel.org
Cc: linux-doc@vger.kernel.org
---
 Documentation/virtual/kvm/api.txt |   3 +-
 arch/mips/include/asm/kvm_host.h  |   6 +-
 arch/mips/kvm/vz.c| 242 ++-
 3 files changed, 250 insertions(+), 1 deletion(-)

diff --git a/Documentation/virtual/kvm/api.txt 
b/Documentation/virtual/kvm/api.txt
index 04e5c4dae523..1116becf8d6f 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2078,6 +2078,9 @@ registers, find a list below:
   MIPS  | KVM_REG_MIPS_CP0_XCONTEXTCONFIG| 64
   MIPS  | KVM_REG_MIPS_CP0_PAGEMASK | 32
   MIPS  | KVM_REG_MIPS_CP0_PAGEGRAIN| 32
+  MIPS  | KVM_REG_MIPS_CP0_SEGCTL0  | 64
+  MIPS  | KVM_REG_MIPS_CP0_SEGCTL1  | 64
+  MIPS  | KVM_REG_MIPS_CP0_SEGCTL2  | 64
   MIPS  | KVM_REG_MIPS_CP0_WIRED| 32
   MIPS  | KVM_REG_MIPS_CP0_HWRENA   | 32
   MIPS  | KVM_REG_MIPS_CP0_BADVADDR | 64
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index d3b377dbed36..f986907a7707 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -39,6 +39,9 @@
 #define KVM_REG_MIPS_CP0_XCONTEXTCONFIGMIPS_CP0_64(4, 3)
 #define KVM_REG_MIPS_CP0_PAGEMASK  MIPS_CP0_32(5, 0)
 #define KVM_REG_MIPS_CP0_PAGEGRAIN MIPS_CP0_32(5, 1)
+#define KVM_REG_MIPS_CP0_SEGCTL0   MIPS_CP0_64(5, 2)
+#define KVM_REG_MIPS_CP0_SEGCTL1   MIPS_CP0_64(5, 3)
+#define KVM_REG_MIPS_CP0_SEGCTL2   MIPS_CP0_64(5, 4)
 #define KVM_REG_MIPS_CP0_WIRED MIPS_CP0_32(6, 0)
 #define KVM_REG_MIPS_CP0_HWRENAMIPS_CP0_32(7, 0)
 #define KVM_REG_MIPS_CP0_BADVADDR  MIPS_CP0_64(8, 0)
@@ -675,6 +678,9 @@ __BUILD_KVM_RW_HW(userlocal,  l,  MIPS_CP0_TLB_CONTEXT, 
 2)
 __BUILD_KVM_RW_HW(xcontextconfig, l,  MIPS_CP0_TLB_CONTEXT,  3)
 __BUILD_KVM_RW_HW(pagemask,   l,  MIPS_CP0_TLB_PG_MASK,  0)
 __BUILD_KVM_RW_HW(pagegrain,  32, MIPS_CP0_TLB_PG_MASK,  1)
+__BUILD_KVM_RW_HW(segctl0,l,  MIPS_CP0_TLB_PG_MASK,  2)
+__BUILD_KVM_RW_HW(segctl1,l,  MIPS_CP0_TLB_PG_MASK,  3)
+__BUILD_KVM_RW_HW(segctl2,l,  MIPS_CP0_TLB_PG_MASK,  4)
 __BUILD_KVM_RW_HW(wired,  32, MIPS_CP0_TLB_WIRED,0)
 __BUILD_KVM_RW_HW(hwrena, 32, MIPS_CP0_HWRENA,   0)
 __BUILD_KVM_RW_HW(badvaddr,   l,  MIPS_CP0_BAD_VADDR,0)
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index b1a7485196a6..373b954a9fd3 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -412,6 +412,117 @@ static void kvm_vz_save_timer(struct kvm_vcpu *vcpu)
 }
 
 /**
+ * is_eva_access() - Find whether an instruction is an EVA memory accessor.
+ * @inst:  32-bit instruction encoding.
+ *
+ * Finds whether @inst encodes an EVA memory access instruction, which would
+ * indicate that emulation of it should access the user mode address space
+ * instead of the kernel mode address space. This matters for MUSUK segments
+ * which are TLB mapped for user mode but unmapped for kernel mode.
+ *
+ * Returns:Whether @inst encodes an EVA accessor instruction.
+ */
+static bool is_eva_access(union mips_instruction inst)
+{
+   if (inst.spec3_format.opcode != spec3_op)
+   return false;
+
+   switch (inst.spec3_format.func) {
+   case lwle_op:
+   case lwre_op:
+   case cachee_op:
+   case sbe_op:
+   case she_op:
+   case sce_op:
+   case swe_op:
+   case swle_op:
+   case swre_op:
+   case prefe_op:
+   case lbue_op:
+   case lhue_op:
+   case lbe_op:
+   case lhe_op:
+   case lle_op:
+   case lwe_op:
+   return true;
+   default:
+   return false;
+   }
+}
+
+/**
+ * is_eva_am_mapped() - Find whether an access mode is mapped.
+ * @vcpu:  KVM VCPU state.
+ * @am:3-bit encoded access mode.
+ * @eu:Segment becomes unmapped and uncached when Status.ERL=1.
+ *
+ * Decode @am to find whether it encodes a mapped segment for the current VCPU
+ * state. Where necessary @eu and the actual instruction causing the fault are
+ * taken into account to make the decision.
+ *
+ * Returns:Whether the VCPU faulted on a TLB mapped address.
+ */
+static bool is_eva_am_mapped(struct kvm_vcpu *vcpu, unsigned int 

[PATCH 30/32] KVM: MIPS/VZ: Emulate MAARs when necessary

[James Hogan]

Add emulation of Memory Accessibility Attribute Registers (MAARs) when
necessary. We can't actually do anything with whatever the guest
provides, but it may not be possible to clear Guest.Config5.MRP so we
have to emulate at least a pair of MAARs.

Signed-off-by: James Hogan 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Ralf Baechle 
Cc: Jonathan Corbet 
Cc: Steven Rostedt 
Cc: Ingo Molnar 
Cc: linux-m...@linux-mips.org
Cc: k...@vger.kernel.org
Cc: linux-doc@vger.kernel.org
---
 Documentation/virtual/kvm/api.txt |   5 +-
 arch/mips/include/asm/kvm_host.h  |   5 +-
 arch/mips/include/uapi/asm/kvm.h  |  20 +-
 arch/mips/kvm/trace.h |   2 +-
 arch/mips/kvm/vz.c| 110 ++-
 5 files changed, 137 insertions(+), 5 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt 
b/Documentation/virtual/kvm/api.txt
index a0430186dbd4..c4e43aeadb10 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2114,6 +2114,7 @@ registers, find a list below:
   MIPS  | KVM_REG_MIPS_CP0_KSCRATCH4| 64
   MIPS  | KVM_REG_MIPS_CP0_KSCRATCH5| 64
   MIPS  | KVM_REG_MIPS_CP0_KSCRATCH6| 64
+  MIPS  | KVM_REG_MIPS_CP0_MAAR(0..63)  | 64
   MIPS  | KVM_REG_MIPS_COUNT_CTL| 64
   MIPS  | KVM_REG_MIPS_COUNT_RESUME | 64
   MIPS  | KVM_REG_MIPS_COUNT_HZ | 64
@@ -2180,6 +2181,10 @@ hardware, host kernel, guest, and whether XPA is present 
in the guest, i.e.
 with the RI and XI bits (if they exist) in bits 63 and 62 respectively, and
 the PFNX field starting at bit 30.
 
+MIPS MAARs (see KVM_REG_MIPS_CP0_MAAR(*) above) have the following id bit
+patterns:
+  0x7030  0001 01 
+
 MIPS KVM control registers (see above) have the following id bit patterns:
   0x7030  0002 
 
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 6b7d131565cd..3b381b52fd6c 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -67,6 +67,7 @@
 #define KVM_REG_MIPS_CP0_CONFIG4   MIPS_CP0_32(16, 4)
 #define KVM_REG_MIPS_CP0_CONFIG5   MIPS_CP0_32(16, 5)
 #define KVM_REG_MIPS_CP0_CONFIG7   MIPS_CP0_32(16, 7)
+#define KVM_REG_MIPS_CP0_MAARI MIPS_CP0_64(17, 2)
 #define KVM_REG_MIPS_CP0_XCONTEXT  MIPS_CP0_64(20, 0)
 #define KVM_REG_MIPS_CP0_ERROREPC  MIPS_CP0_64(30, 0)
 #define KVM_REG_MIPS_CP0_KSCRATCH1 MIPS_CP0_64(31, 2)
@@ -391,6 +392,9 @@ struct kvm_vcpu_arch {
struct kvm_mips_tlb *wired_tlb;
unsigned int wired_tlb_limit;
unsigned int wired_tlb_used;
+
+   /* emulated guest MAAR registers */
+   unsigned long maar[6];
 #endif
 
/* Last CPU the VCPU state was loaded on */
@@ -711,6 +715,7 @@ __BUILD_KVM_RW_HW(config4,32, MIPS_CP0_CONFIG,  
 4)
 __BUILD_KVM_RW_HW(config5,32, MIPS_CP0_CONFIG,   5)
 __BUILD_KVM_RW_HW(config6,32, MIPS_CP0_CONFIG,   6)
 __BUILD_KVM_RW_HW(config7,32, MIPS_CP0_CONFIG,   7)
+__BUILD_KVM_RW_SW(maari,  l,  MIPS_CP0_LLADDR,   2)
 __BUILD_KVM_RW_HW(xcontext,   l,  MIPS_CP0_TLB_XCONTEXT, 0)
 __BUILD_KVM_RW_HW(errorepc,   l,  MIPS_CP0_ERROR_PC, 0)
 __BUILD_KVM_RW_HW(kscratch1,  l,  MIPS_CP0_DESAVE,   2)
diff --git a/arch/mips/include/uapi/asm/kvm.h b/arch/mips/include/uapi/asm/kvm.h
index a8a0199bf760..3107095d7f0a 100644
--- a/arch/mips/include/uapi/asm/kvm.h
+++ b/arch/mips/include/uapi/asm/kvm.h
@@ -54,9 +54,14 @@ struct kvm_fpu {
  * Register set = 0: GP registers from kvm_regs (see definitions below).
  *
  * Register set = 1: CP0 registers.
- *  bits[15..8]  - Must be zero.
- *  bits[7..3]   - Register 'rd'  index.
- *  bits[2..0]   - Register 'sel' index.
+ *  bits[15..8]  - COP0 register set.
+ *
+ *  COP0 register set = 0: Main CP0 registers.
+ *   bits[7..3]   - Register 'rd'  index.
+ *   bits[2..0]   - Register 'sel' index.
+ *
+ *  COP0 register set = 1: MAARs.
+ *   bits[7..0]   - MAAR index.
  *
  * Register set = 2: KVM specific registers (see definitions below).
  *
@@ -115,6 +120,15 @@ struct kvm_fpu {
 
 
 /*
+ * KVM_REG_MIPS_CP0 - Coprocessor 0 registers.
+ */
+
+#define KVM_REG_MIPS_MAAR  (KVM_REG_MIPS_CP0 | (1 << 8))
+#define KVM_REG_MIPS_CP0_MAAR(n)   (KVM_REG_MIPS_MAAR | \
+KVM_REG_SIZE_U64 | (n))
+
+
+/*
  * KVM_REG_MIPS_KVM - KVM specific control registers.
  */
 
diff --git a/arch/mips/kvm/trace.h b/arch/mips/kvm/trace.h
index d80d37a1b82e..affde8a2c584 100644
--- a/arch/mips/kvm/trace.h
+++ b/arch/mips/kvm/trace.h
@@ -176,6 +176,8 @@ TRACE_EVENT(kvm_exit,
{ KVM_TRACE_COP0(16, 4),"Config4" },\
{ KVM_TRACE_COP0(16, 5),"Config5" },\
{ KVM_TRACE_COP0(16, 7),"Config7" },\
+   { KVM_TRACE_COP0(17, 1),"MAAR" 

[PATCH 25/32] KVM: MIPS/VZ: Support guest CP0_BadInstr[P]

[James Hogan]

Add support for VZ guest CP0_BadInstr and CP0_BadInstrP registers, as
found on most VZ capable cores. These guest registers need context
switching, and exposing via the KVM ioctl API when they are present.

Signed-off-by: James Hogan 
Cc: Paolo Bonzini 
Cc: "Radim Krčmář" 
Cc: Ralf Baechle 
Cc: Jonathan Corbet 
Cc: linux-m...@linux-mips.org
Cc: k...@vger.kernel.org
Cc: linux-doc@vger.kernel.org
---
 Documentation/virtual/kvm/api.txt |  2 +-
 arch/mips/include/asm/kvm_host.h  |  4 +++-
 arch/mips/kvm/vz.c| 46 -
 3 files changed, 52 insertions(+), 0 deletions(-)

diff --git a/Documentation/virtual/kvm/api.txt 
b/Documentation/virtual/kvm/api.txt
index 996a388f05c5..a654a2133981 100644
--- a/Documentation/virtual/kvm/api.txt
+++ b/Documentation/virtual/kvm/api.txt
@@ -2079,6 +2079,8 @@ registers, find a list below:
   MIPS  | KVM_REG_MIPS_CP0_WIRED| 32
   MIPS  | KVM_REG_MIPS_CP0_HWRENA   | 32
   MIPS  | KVM_REG_MIPS_CP0_BADVADDR | 64
+  MIPS  | KVM_REG_MIPS_CP0_BADINSTR | 32
+  MIPS  | KVM_REG_MIPS_CP0_BADINSTRP| 32
   MIPS  | KVM_REG_MIPS_CP0_COUNT| 32
   MIPS  | KVM_REG_MIPS_CP0_ENTRYHI  | 64
   MIPS  | KVM_REG_MIPS_CP0_COMPARE  | 32
diff --git a/arch/mips/include/asm/kvm_host.h b/arch/mips/include/asm/kvm_host.h
index 41d8abc1d6fa..53c92f34475f 100644
--- a/arch/mips/include/asm/kvm_host.h
+++ b/arch/mips/include/asm/kvm_host.h
@@ -40,6 +40,8 @@
 #define KVM_REG_MIPS_CP0_WIRED MIPS_CP0_32(6, 0)
 #define KVM_REG_MIPS_CP0_HWRENAMIPS_CP0_32(7, 0)
 #define KVM_REG_MIPS_CP0_BADVADDR  MIPS_CP0_64(8, 0)
+#define KVM_REG_MIPS_CP0_BADINSTR  MIPS_CP0_32(8, 1)
+#define KVM_REG_MIPS_CP0_BADINSTRP MIPS_CP0_32(8, 2)
 #define KVM_REG_MIPS_CP0_COUNT MIPS_CP0_32(9, 0)
 #define KVM_REG_MIPS_CP0_ENTRYHI   MIPS_CP0_64(10, 0)
 #define KVM_REG_MIPS_CP0_COMPARE   MIPS_CP0_32(11, 0)
@@ -672,6 +674,8 @@ __BUILD_KVM_RW_HW(pagegrain,  32, MIPS_CP0_TLB_PG_MASK, 
 1)
 __BUILD_KVM_RW_HW(wired,  32, MIPS_CP0_TLB_WIRED,0)
 __BUILD_KVM_RW_HW(hwrena, 32, MIPS_CP0_HWRENA,   0)
 __BUILD_KVM_RW_HW(badvaddr,   l,  MIPS_CP0_BAD_VADDR,0)
+__BUILD_KVM_RW_HW(badinstr,   32, MIPS_CP0_BAD_VADDR,1)
+__BUILD_KVM_RW_HW(badinstrp,  32, MIPS_CP0_BAD_VADDR,2)
 __BUILD_KVM_RW_SW(count,  32, MIPS_CP0_COUNT,0)
 __BUILD_KVM_RW_HW(entryhi,l,  MIPS_CP0_TLB_HI,   0)
 __BUILD_KVM_RW_HW(compare,32, MIPS_CP0_COMPARE,  0)
diff --git a/arch/mips/kvm/vz.c b/arch/mips/kvm/vz.c
index 30c3550926b0..5fd14d35c146 100644
--- a/arch/mips/kvm/vz.c
+++ b/arch/mips/kvm/vz.c
@@ -1220,6 +1220,10 @@ static unsigned long kvm_vz_num_regs(struct kvm_vcpu 
*vcpu)
ret = ARRAY_SIZE(kvm_vz_get_one_regs);
if (cpu_guest_has_userlocal)
++ret;
+   if (cpu_guest_has_badinstr)
+   ++ret;
+   if (cpu_guest_has_badinstrp)
+   ++ret;
ret += __arch_hweight8(cpu_data[0].guest.kscratch_mask);
 
return ret;
@@ -1241,6 +1245,18 @@ static int kvm_vz_copy_reg_indices(struct kvm_vcpu 
*vcpu, u64 __user *indices)
return -EFAULT;
++indices;
}
+   if (cpu_guest_has_badinstr) {
+   index = KVM_REG_MIPS_CP0_BADINSTR;
+   if (copy_to_user(indices, , sizeof(index)))
+   return -EFAULT;
+   ++indices;
+   }
+   if (cpu_guest_has_badinstrp) {
+   index = KVM_REG_MIPS_CP0_BADINSTRP;
+   if (copy_to_user(indices, , sizeof(index)))
+   return -EFAULT;
+   ++indices;
+   }
for (i = 0; i < 6; ++i) {
if (!cpu_guest_has_kscr(i + 2))
continue;
@@ -1326,6 +1342,16 @@ static int kvm_vz_get_one_reg(struct kvm_vcpu *vcpu,
case KVM_REG_MIPS_CP0_BADVADDR:
*v = (long)read_gc0_badvaddr();
break;
+   case KVM_REG_MIPS_CP0_BADINSTR:
+   if (!cpu_guest_has_badinstr)
+   return -EINVAL;
+   *v = read_gc0_badinstr();
+   break;
+   case KVM_REG_MIPS_CP0_BADINSTRP:
+   if (!cpu_guest_has_badinstrp)
+   return -EINVAL;
+   *v = read_gc0_badinstrp();
+   break;
case KVM_REG_MIPS_CP0_COUNT:
*v = kvm_mips_read_count(vcpu);
break;
@@ -1471,6 +1497,16 @@ static int kvm_vz_set_one_reg(struct kvm_vcpu *vcpu,
case KVM_REG_MIPS_CP0_BADVADDR:
write_gc0_badvaddr(v);
break;
+   case KVM_REG_MIPS_CP0_BADINSTR:
+   if (!cpu_guest_has_badinstr)
+   return -EINVAL;
+   write_gc0_badinstr(v);
+   break;
+   case 

[PATCH] drm/doc: atomic overview, with graph

[Daniel Vetter]

I want to split up a few more things and document some details better
(like how exactly to subclass drm_atomic_state). And maybe also split
up the helpers a bit per-topic, but this should be a ok-ish start for
better atomic overview.

v2: Spelling and clarifications (Eric).

v3: Implement suggestion from Gabriel to fix the graph.

Cc: Gabriel Krisman Bertazi 
Acked-by: Eric Anholt 
Cc: Eric Anholt 
Cc: Laurent Pinchart 
Cc: Harry Wentland 
Signed-off-by: Daniel Vetter 
---
 Documentation/gpu/drm-kms-helpers.rst |  2 +
 Documentation/gpu/drm-kms.rst | 84 ++-
 2 files changed, 85 insertions(+), 1 deletion(-)

diff --git a/Documentation/gpu/drm-kms-helpers.rst 
b/Documentation/gpu/drm-kms-helpers.rst
index 050ebe81d256..ac53c0b893f6 100644
--- a/Documentation/gpu/drm-kms-helpers.rst
+++ b/Documentation/gpu/drm-kms-helpers.rst
@@ -42,6 +42,8 @@ Modeset Helper Reference for Common Vtables
 .. kernel-doc:: include/drm/drm_modeset_helper_vtables.h
:internal:
 
+.. _drm_atomic_helper:
+
 Atomic Modeset Helper Functions Reference
 =
 
diff --git a/Documentation/gpu/drm-kms.rst b/Documentation/gpu/drm-kms.rst
index a504d9ee4d94..eb9d29865c41 100644
--- a/Documentation/gpu/drm-kms.rst
+++ b/Documentation/gpu/drm-kms.rst
@@ -189,8 +189,90 @@ multiple times to different objects using 
:c:func:`drm_object_attach_property()
 .. kernel-doc:: drivers/gpu/drm/drm_mode_object.c
:export:
 
+Atomic Mode Setting
+===
+
+
+.. kernel-render:: DOT
+   :alt: Mode Objects and Properties
+   :caption: Mode Objects and Properties
+
+   digraph {
+  node [shape=box]
+
+  subgraph cluster_state {
+  style=dashed
+  label="Free-standing state"
+
+  "drm_atomic_state" -> "duplicated drm_plane_state A"
+  "drm_atomic_state" -> "duplicated drm_plane_state B"
+  "drm_atomic_state" -> "duplicated drm_crtc_state"
+  "drm_atomic_state" -> "duplicated drm_connector_state"
+  "drm_atomic_state" -> "duplicated driver private state"
+  }
+
+  subgraph cluster_current {
+  style=dashed
+  label="Current state"
+
+  "drm_device" -> "drm_plane A"
+  "drm_device" -> "drm_plane B"
+  "drm_device" -> "drm_crtc"
+  "drm_device" -> "drm_connector"
+  "drm_device" -> "driver private object"
+
+  "drm_plane A" -> "drm_plane_state A"
+  "drm_plane B" -> "drm_plane_state B"
+  "drm_crtc" -> "drm_crtc_state"
+  "drm_connector" -> "drm_connector_state"
+  "driver private object" -> "driver private state"
+  }
+
+  "drm_atomic_state" -> "drm_device" [label="atomic_commit"]
+  "duplicated drm_plane_state A" -> "drm_device"[style=invis]
+   }
+
+Atomic provides transactional modeset (including planes) updates, but a
+bit differently from the usual transactional approach of try-commit and
+rollback:
+
+- Firstly, no hardware changes are allowed when the commit would fail. This
+  allows us to implement the DRM_MODE_ATOMIC_TEST_ONLY mode, which allows
+  userspace to explore whether certain configurations would work or not.
+
+- This would still allow setting and rollback of just the software state,
+  simplifying conversion of existing drivers. But auditing drivers for
+  correctness of the atomic_check code becomes really hard with that: Rolling
+  back changes in data structures all over the place is hard to get right.
+
+- Lastly, for backwards compatibility and to support all use-cases, atomic
+  updates need to be incremental and be able to execute in parallel. Hardware
+  doesn't always allow it, but where possible plane updates on different CRTCs
+  should not interfere, and not get stalled due to output routing changing on
+  different CRTCs.
+
+Taken all together there's two consequences for the atomic design:
+
+- The overall state is split up into per-object state structures:
+  :c:type:`struct drm_plane_state ` for planes, 
:c:type:`struct
+  drm_crtc_state ` for CRTCs and :c:type:`struct
+  drm_connector_state 

Re: [PATCH 2/6] docs-rst: automatically convert Graphviz and SVG images

[Daniel Vetter]

On Wed, Mar 01, 2017 at 12:56:36PM -0300, Gabriel Krisman Bertazi wrote:
> Daniel Vetter  writes:
> 
> Hi Daniel,
> 
> > +if dst_fname:
> > +name = dst_fname[len(out_dir) + 1:]
> > +# the builder needs not to copy one more time, so pop it if exists.
> > +translator.builder.images.pop(img_node['uri'], None)
> > +img_node['uri'] = dst_fname
> > +img_node['candidates'] = {'*': dst_fname}
> > +
> > +mkdir(path.dirname(dst_fname))
> > +
> > +if in_ext == '.dot':
> > +verbose('convert DOT to: {out}/' + name)
> > +dot2format(src_fname, dst_fname)
> > +
> > +elif in_ext == '.svg':
> > +verbose('convert SVG to: {out}/' + name)
> > +svg2pdf(src_fname, dst_fname)
> 
> Small nit, but, shouldn't you add dst_fname to img_node only if
> dot2format or svg2pdf was successful?

I kinda assumed they're always successful, since the media documentation
build already requires them.
-Daniel
-- 
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html