Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Linus Torvalds]

On Mon, Nov 5, 2018 at 5:15 AM Martin Schwidefsky
 wrote:
>
> This patch would work for me:

Thanks, applied,

 Linus

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Steven Rostedt]

On Mon, 5 Nov 2018 14:15:35 +0100
Martin Schwidefsky  wrote:
>  
> Follow-up question: the __no_sanitize_address_or_inline define has the 
> 'notrace'
> option that is missing for __no_kasan_or_inline. We need that option for
> __load_psw_mask, if we do the replacement all users of __no_kasan_or_inline
> would get the option as well. This affects __read_once_size_nocheck and
> read_word_at_a_time. Do these function have to be traceable ?

Some history on why I added notrace to inline. It was to make things
more consistent. Since gcc wont add a fentry/mcount call to inlined
functions, having those functions show up in the trace depending on
whether or not gcc honored the "inline" tag made things confusing. And
it even once caused a crash when local_irq_save() stopped being
inlined.

I added notrace to inline so that if you mark something as inline, it
would not show up in the trace regardless of gcc deciding to inline the
function or not.

> 
> This patch would work for me:
> --
> >From 4aaa09fe4b54e930edabac86606dee979b12647c Mon Sep 17 00:00:00 2001  
> From: Martin Schwidefsky 
> Date: Mon, 5 Nov 2018 07:36:28 +0100
> Subject: [PATCH] compiler: remove __no_sanitize_address_or_inline again
> 
> The __no_sanitize_address_or_inline and __no_kasan_or_inline defines
> are almost identical. The only difference is that __no_kasan_or_inline
> does not have the 'notrace' attribute.
> 
> To be able to replace __no_sanitize_address_or_inline with the older
> definition, add 'notrace' to __no_kasan_or_inline and change to two
> users of __no_sanitize_address_or_inline in the s390 code.
> 
> The 'notrace' option is necessary for e.g. the __load_psw_mask function
> in arch/s390/include/asm/processor.h. Without the option it is possible
> to trace __load_psw_mask which leads to kernel stack overflow.

Acked-by: Steven Rostedt (VMware) 

-- Steve

> 
> Signed-off-by: Martin Schwidefsky 
> ---
>  arch/s390/include/asm/processor.h |  4 ++--
>  include/linux/compiler-gcc.h  | 12 
>  include/linux/compiler.h  |  2 +-
>  3 files changed, 3 insertions(+), 15 deletions(-)
> 
> diff --git a/arch/s390/include/asm/processor.h 
> b/arch/s390/include/asm/processor.h
> index 302795c47c06..81038ab357ce 100644
> --- a/arch/s390/include/asm/processor.h
> +++ b/arch/s390/include/asm/processor.h
> @@ -236,7 +236,7 @@ static inline unsigned long current_stack_pointer(void)
>   return sp;
>  }
>  
> -static __no_sanitize_address_or_inline unsigned short stap(void)
> +static __no_kasan_or_inline unsigned short stap(void)
>  {
>   unsigned short cpu_address;
>  
> @@ -330,7 +330,7 @@ static inline void __load_psw(psw_t psw)
>   * Set PSW mask to specified value, while leaving the
>   * PSW addr pointing to the next instruction.
>   */
> -static __no_sanitize_address_or_inline void __load_psw_mask(unsigned long 
> mask)
> +static __no_kasan_or_inline void __load_psw_mask(unsigned long mask)
>  {
>   unsigned long addr;
>   psw_t psw;
> diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
> index c0f5db3a9621..2010493e1040 100644
> --- a/include/linux/compiler-gcc.h
> +++ b/include/linux/compiler-gcc.h
> @@ -143,18 +143,6 @@
>  #define KASAN_ABI_VERSION 3
>  #endif
>  
> -/*
> - * Because __no_sanitize_address conflicts with inlining:
> - *   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
> - * we do one or the other. 
> - */
> -#ifdef CONFIG_KASAN
> -#define __no_sanitize_address_or_inline  
> \
> - __no_sanitize_address __maybe_unused notrace
> -#else
> -#define __no_sanitize_address_or_inline inline
> -#endif
> -
>  #if GCC_VERSION >= 50100
>  #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
>  #endif
> diff --git a/include/linux/compiler.h b/include/linux/compiler.h
> index 18c80cfa4fc4..06396c1cf127 100644
> --- a/include/linux/compiler.h
> +++ b/include/linux/compiler.h
> @@ -189,7 +189,7 @@ void __read_once_size(const volatile void *p, void *res, 
> int size)
>   *   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
>   * '__maybe_unused' allows us to avoid defined-but-not-used warnings.
>   */
> -# define __no_kasan_or_inline __no_sanitize_address __maybe_unused
> +# define __no_kasan_or_inline __no_sanitize_address notrace __maybe_unused
>  #else
>  # define __no_kasan_or_inline __always_inline
>  #endif

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Martin Schwidefsky]

On Mon, 5 Nov 2018 07:02:56 +0100
Martin Schwidefsky  wrote:

> On Fri, 2 Nov 2018 09:09:32 -0700
> Linus Torvalds  wrote:
> 
> > On Fri, Nov 2, 2018 at 2:43 AM Andrey Ryabinin  
> > wrote:  
> > >
> > > You're right, version checks shouldn't matter here. But 
> > > __no_sanitize_address_or_inline
> > > shouldn't have been added in the first place, because we already have 
> > > almost the same
> > >__no_kasan_or_inline:
> > 
> > Ahh, very good.
> > 
> > Vasily, Martin - since __no_sanitize_address_or_inline was added just
> > for s390, would you mind replacing it with __no_kasan_or_inline
> > instead, and testing that in whatever failed before?
> > 
> > Then we can just remove that unnecessary symbol #define entirely..  
> 
> Ok, will do.
 
Follow-up question: the __no_sanitize_address_or_inline define has the 'notrace'
option that is missing for __no_kasan_or_inline. We need that option for
__load_psw_mask, if we do the replacement all users of __no_kasan_or_inline
would get the option as well. This affects __read_once_size_nocheck and
read_word_at_a_time. Do these function have to be traceable ?

This patch would work for me:
--
>From 4aaa09fe4b54e930edabac86606dee979b12647c Mon Sep 17 00:00:00 2001
From: Martin Schwidefsky 
Date: Mon, 5 Nov 2018 07:36:28 +0100
Subject: [PATCH] compiler: remove __no_sanitize_address_or_inline again

The __no_sanitize_address_or_inline and __no_kasan_or_inline defines
are almost identical. The only difference is that __no_kasan_or_inline
does not have the 'notrace' attribute.

To be able to replace __no_sanitize_address_or_inline with the older
definition, add 'notrace' to __no_kasan_or_inline and change to two
users of __no_sanitize_address_or_inline in the s390 code.

The 'notrace' option is necessary for e.g. the __load_psw_mask function
in arch/s390/include/asm/processor.h. Without the option it is possible
to trace __load_psw_mask which leads to kernel stack overflow.

Signed-off-by: Martin Schwidefsky 
---
 arch/s390/include/asm/processor.h |  4 ++--
 include/linux/compiler-gcc.h  | 12 
 include/linux/compiler.h  |  2 +-
 3 files changed, 3 insertions(+), 15 deletions(-)

diff --git a/arch/s390/include/asm/processor.h 
b/arch/s390/include/asm/processor.h
index 302795c47c06..81038ab357ce 100644
--- a/arch/s390/include/asm/processor.h
+++ b/arch/s390/include/asm/processor.h
@@ -236,7 +236,7 @@ static inline unsigned long current_stack_pointer(void)
return sp;
 }
 
-static __no_sanitize_address_or_inline unsigned short stap(void)
+static __no_kasan_or_inline unsigned short stap(void)
 {
unsigned short cpu_address;
 
@@ -330,7 +330,7 @@ static inline void __load_psw(psw_t psw)
  * Set PSW mask to specified value, while leaving the
  * PSW addr pointing to the next instruction.
  */
-static __no_sanitize_address_or_inline void __load_psw_mask(unsigned long mask)
+static __no_kasan_or_inline void __load_psw_mask(unsigned long mask)
 {
unsigned long addr;
psw_t psw;
diff --git a/include/linux/compiler-gcc.h b/include/linux/compiler-gcc.h
index c0f5db3a9621..2010493e1040 100644
--- a/include/linux/compiler-gcc.h
+++ b/include/linux/compiler-gcc.h
@@ -143,18 +143,6 @@
 #define KASAN_ABI_VERSION 3
 #endif
 
-/*
- * Because __no_sanitize_address conflicts with inlining:
- *   https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
- * we do one or the other. 
- */
-#ifdef CONFIG_KASAN
-#define __no_sanitize_address_or_inline
\
-   __no_sanitize_address __maybe_unused notrace
-#else
-#define __no_sanitize_address_or_inline inline
-#endif
-
 #if GCC_VERSION >= 50100
 #define COMPILER_HAS_GENERIC_BUILTIN_OVERFLOW 1
 #endif
diff --git a/include/linux/compiler.h b/include/linux/compiler.h
index 18c80cfa4fc4..06396c1cf127 100644
--- a/include/linux/compiler.h
+++ b/include/linux/compiler.h
@@ -189,7 +189,7 @@ void __read_once_size(const volatile void *p, void *res, 
int size)
  * https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
  * '__maybe_unused' allows us to avoid defined-but-not-used warnings.
  */
-# define __no_kasan_or_inline __no_sanitize_address __maybe_unused
+# define __no_kasan_or_inline __no_sanitize_address notrace __maybe_unused
 #else
 # define __no_kasan_or_inline __always_inline
 #endif
-- 
2.16.4
-- 
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Martin Schwidefsky]

On Fri, 2 Nov 2018 09:09:32 -0700
Linus Torvalds  wrote:

> On Fri, Nov 2, 2018 at 2:43 AM Andrey Ryabinin  
> wrote:
> >
> > You're right, version checks shouldn't matter here. But 
> > __no_sanitize_address_or_inline
> > shouldn't have been added in the first place, because we already have 
> > almost the same
> >__no_kasan_or_inline:  
> 
> Ahh, very good.
> 
> Vasily, Martin - since __no_sanitize_address_or_inline was added just
> for s390, would you mind replacing it with __no_kasan_or_inline
> instead, and testing that in whatever failed before?
> 
> Then we can just remove that unnecessary symbol #define entirely..

Ok, will do.

-- 
blue skies,
   Martin.

"Reality continues to ruin my life." - Calvin.

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Andrey Ryabinin]

On 11/02/2018 07:11 PM, Linus Torvalds wrote:
> On Fri, Nov 2, 2018 at 6:16 AM Andrey Ryabinin  
> wrote:
>>
>> On 11/02/2018 04:46 AM, Linus Torvalds wrote:
>>>
>>> So I _think_ the KASAN config should have a
>>>
>>> depends on CC_IS_GCC && GCC_VERSION >= 40902
>>>
>>> on it, but maybe there is something I'm missing.
>>
>> I'd rather use cc-option instead of version check, since we also support 
>> clang.
> 
> That would be even better, but I thought the requirement for 4.9.2
> came not from the option existing, but because of some bugs getting
> fixed?

It looks unusual but -fsanitize=kernel-address really showed up only in 4.9.2.
It was actually backported from 5.0 branch to 4.9 for whatever reason.

> 
> But if we can do it without version checks, that would be lovely.
> 
> Linus
> 

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Linus Torvalds]

On Fri, Nov 2, 2018 at 6:16 AM Andrey Ryabinin  wrote:
>
> On 11/02/2018 04:46 AM, Linus Torvalds wrote:
> >
> > So I _think_ the KASAN config should have a
> >
> > depends on CC_IS_GCC && GCC_VERSION >= 40902
> >
> > on it, but maybe there is something I'm missing.
>
> I'd rather use cc-option instead of version check, since we also support 
> clang.

That would be even better, but I thought the requirement for 4.9.2
came not from the option existing, but because of some bugs getting
fixed?

But if we can do it without version checks, that would be lovely.

Linus

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Linus Torvalds]

On Fri, Nov 2, 2018 at 2:43 AM Andrey Ryabinin  wrote:
>
> You're right, version checks shouldn't matter here. But 
> __no_sanitize_address_or_inline
> shouldn't have been added in the first place, because we already have almost 
> the same
>__no_kasan_or_inline:

Ahh, very good.

Vasily, Martin - since __no_sanitize_address_or_inline was added just
for s390, would you mind replacing it with __no_kasan_or_inline
instead, and testing that in whatever failed before?

Then we can just remove that unnecessary symbol #define entirely..

Thanks,

 Linus

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Andrey Ryabinin]

On 11/02/2018 04:46 AM, Linus Torvalds wrote:
> On Thu, Nov 1, 2018 at 10:06 AM Linus Torvalds
>  wrote:
>>
>> The logic for using __no_sanitize_address *used* to be
>>
>> #if GCC_VERSION >= 40902
> 
> Ok, looking around, I think this has less to do with the attribute
> being recognized, and simply just being because KASAN itself wants
> gcc-4.9.2.
> 
> I'm actually not seeing that KASAN dependency in the Kconfig scripts
> (and it probably _should_ be now that we can just add compiler version
> dependencies there), but that explains why the gcc version check is
> different from "gcc supports the attribute".
> 
> Anyway, I decided to do the merge by just getting rid of the
> GCC_VERSION check around __no_sanitize_address_or_inline entirely. If
> you enable KASAN, then a function with that marking just won't be
> marked inline.
> 
> End result: pulled. I'm as confused as you are as to why
> __no_sanitize_address_or_inline is in the gcc header, but I guess it
> ends up being the same issue: KASAN depends on gcc even if that
> dependency doesn't seem to be spelled out in lib/Kconfig.kasan.
> 
> So I _think_ the KASAN config should have a
> 
> depends on CC_IS_GCC && GCC_VERSION >= 40902
> 
> on it, but maybe there is something I'm missing.
> 

I'd rather use cc-option instead of version check, since we also support clang.

It should be possible to express compiler requirements for subfeatures
like stack/inline instrumentation in Kconfig. But that would be not that 
trivial, 
see the scripts/Makefile.kasan

> But from a pull standpoint, I don't want to mess with those
> (unrelated) issues, so I just kept the merge resolution as simple and
> straightforward as possible.
> 
> Miguel, please do double-check the merge (it's not pushed out yet, I'm
> doing the usual build tests etc first).
> 
> Linus
> 

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Miguel Ojeda]

On Fri, Nov 2, 2018 at 2:52 AM Linus Torvalds
 wrote:
>
> Anyway, I decided to do the merge by just getting rid of the
> GCC_VERSION check around __no_sanitize_address_or_inline entirely. If
> you enable KASAN, then a function with that marking just won't be
> marked inline.

I was a bit confused when reading the gcc bug reports, i.e. why gcc
did *not* complain in 4.9 but it did in 5.1 (when it was supposed to
complain also in 4.9). It turns out that gcc 5.1 takes into account
who is the actual caller due to this change:

+  cgraph_node *caller = e->caller->global.inlined_to
+   ? e->caller->global.inlined_to : e->caller;
...
-  else if (!sanitize_attrs_match_for_inline_p (e->caller->decl, callee->decl))
+  else if (!sanitize_attrs_match_for_inline_p (caller->decl, callee->decl))

change; e.g. this:

#define really_inline inline __attribute__((always_inline))
#define no_sanitize __attribute__((no_sanitize_address))

really_inline void f() {}
really_inline void g() { f(); }
no_sanitize void h() { g(); }

Complains in gcc 4.9 -O0, 5.1 -O0 and 5.1 -O2; but *not* in 4.9 -O2.
https://godbolt.org/z/kNApqk

Anyway, this is orthogonal but in case it clarifies that for someone else...

> Miguel, please do double-check the merge (it's not pushed out yet, I'm
> doing the usual build tests etc first).

I was sleeping, didn't manage to see it (in your GitHub, I guess?).

I did the merge myself, and arrived at the same thing as you. I
quickly inspected the rest and seems fine. By the way, I spotted an
extra space at:

+ * we do one or the other.

Cheers,
Miguel

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Andrey Ryabinin]

On 11/01/2018 08:06 PM, Linus Torvalds wrote:
> On Mon, Oct 22, 2018 at 3:59 AM Miguel Ojeda
>  wrote:
>>
>> Here it is the Compiler Attributes series/tree, which tries to disentangle
>> the include/linux/compiler*.h headers and bring them up to date.
> 
> I've finally emptied the "normal" pull queues, and am looking at the
> odd ones that I felt I needed to review more in-depth.
> 
> This looked fine to me, and I already pulled, but when looking at the
> conflict for __no_sanitize_address_or_inline, I also noticed that you
> actually changed the gcc version logic.
> 
> The logic for using __no_sanitize_address *used* to be
> 
> #if GCC_VERSION >= 40902
> 
> but you have changed it to
> 
>   # define __GCC4_has_attribute___no_sanitize_address__ (__GNUC_MINOR__ >= 8)
> 
> so now it's gcc-4.8+ rather than gcc-4.9.2+.

As you said in other email - "this has less to do with the attribute
being recognized, and simply just being because KASAN itself wants
gcc-4.9.2."

gcc < 4.9.2 simply doesn't have -fsanitize=kernel-address.
But no_sanitize attribute originally comes with user space ASAN 
(-fsanitize=address)
which exist in earlier GCCs like 4.8.
Checking against 4.8 gcc should be fine. If we compile the kernel with gcc 4.8
it will be compiled without instrumentation, so the attribute won't have any 
effect.

> 
> I'm not sure how much that matters (maybe the original check for 4.9.2
> was just a random pick by Andrey? Added to cc), but together with the
> movement to  that looks like it also
> wouldn't want the CONFIG_KASAN tests, I wonder what the right merge
> resolution would be.
> 
> Yes, I see the resolution in linux-next, and I think that one is odd
> and dubious, and now it *mixes* that old test of gcc-4.9.2 with the
> different test in compiler-attributes.
> 
> I'm _inclined_ to just do
> 
> #ifdef CONFIG_KASAN
>  #define __no_sanitize_address_or_inline \
>   __no_sanitize_address __maybe_unused notrace
> #else
>  #define __no_sanitize_address_or_inline inline
> #endif
> 
> without any compiler versions at all, because I don't think it matters
> (maybe we won't get address sanitation, and we will not get inlining
> either, but do we care?).

You're right, version checks shouldn't matter here. But 
__no_sanitize_address_or_inline
shouldn't have been added in the first place, because we already have almost 
the same __no_kasan_or_inline:

#ifdef CONFIG_KASAN
/*
 * We can't declare function 'inline' because __no_sanitize_address confilcts
 * with inlining. Attempt to inline it may cause a build failure.
 *  https://gcc.gnu.org/bugzilla/show_bug.cgi?id=67368
 * '__maybe_unused' allows us to avoid defined-but-not-used warnings.
 */
# define __no_kasan_or_inline __no_sanitize_address __maybe_unused
#else
# define __no_kasan_or_inline __always_inline
#endif

There are small differences like absence of notrace, but notrace could be added
to the function together with __no_kasan_or_inline.
And inline is *not* redefined to __always_inline  only on x86 when 
CONFIG_OPTIMIZE_INLINING=y

So there is absolutely no reason to have both __no_kasan_or_inline and 
__no_sanitize_address_or_inline.

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Linus Torvalds]

On Thu, Nov 1, 2018 at 10:06 AM Linus Torvalds
 wrote:
>
> The logic for using __no_sanitize_address *used* to be
>
> #if GCC_VERSION >= 40902

Ok, looking around, I think this has less to do with the attribute
being recognized, and simply just being because KASAN itself wants
gcc-4.9.2.

I'm actually not seeing that KASAN dependency in the Kconfig scripts
(and it probably _should_ be now that we can just add compiler version
dependencies there), but that explains why the gcc version check is
different from "gcc supports the attribute".

Anyway, I decided to do the merge by just getting rid of the
GCC_VERSION check around __no_sanitize_address_or_inline entirely. If
you enable KASAN, then a function with that marking just won't be
marked inline.

End result: pulled. I'm as confused as you are as to why
__no_sanitize_address_or_inline is in the gcc header, but I guess it
ends up being the same issue: KASAN depends on gcc even if that
dependency doesn't seem to be spelled out in lib/Kconfig.kasan.

So I _think_ the KASAN config should have a

depends on CC_IS_GCC && GCC_VERSION >= 40902

on it, but maybe there is something I'm missing.

But from a pull standpoint, I don't want to mess with those
(unrelated) issues, so I just kept the merge resolution as simple and
straightforward as possible.

Miguel, please do double-check the merge (it's not pushed out yet, I'm
doing the usual build tests etc first).

Linus

Re: [GIT PULL] Compiler Attributes for v4.20-rc1

[Miguel Ojeda]

On Thu, Nov 1, 2018 at 6:06 PM Linus Torvalds
 wrote:
>
> I'm not sure how much that matters (maybe the original check for 4.9.2
> was just a random pick by Andrey? Added to cc), but together with the
> movement to  that looks like it also
> wouldn't want the CONFIG_KASAN tests, I wonder what the right merge
> resolution would be.

Good catch. I don't recall any special logic when I did that change,
so most likely I simply did like for the rest of the attributes and
took a look at when it was first supported (and documentation in gcc's
docs) in order to implement __has_attribute by hand.

But indeed, it *may* be that there is an (undocumented) problem
between 4.8 <= gcc < 4.9.2 with it. If so, we should document it down
and fix it. Andrey?

> Yes, I see the resolution in linux-next, and I think that one is odd
> and dubious, and now it *mixes* that old test of gcc-4.9.2 with the
> different test in compiler-attributes.

I missed that conflict completely, my bad (I did not miss all of them,
at least; one required fixing).

Hm at a quick look, why is it only on compiler-gcc.h? It should
either have a corresponding #define elsewhere or just be put directly
in another common header, no? (Adding Vasily & Martin to CC.)

> But I'm also unsure whether you meant for the "__has_attribute()" test
> to be usable outside the linux/compiler_attributes.h file, in which
> case I could just do
>
>   #if defined(CONFIG_KASAN) && __has_attribute(__no_sanitize_address__)
>
> instead.

I think that (using __has_attribute() outside) may be a good idea: I
wanted to keep compiler_attributes.h as simple as possible by avoiding
#ifdefs inside that header (except for __has_attribute itself), as an
attempt to avoid going back to the mess of #ifdefs we had previously.
Basically, keeping the attributes in compiler_attributes.h that do not
depend on complex logic. So using __has_attribute *outside* the header
actually goes well with that principle, because it helps keeping stuff
out of it if they depend on other config options; without having to
rely on GCC_VERSION either.

[By the way, in case it clarifies: note that "optional" in that file
actually is a bit of a misnomer. I meant to say "optional" as in "not
supported by all compilers, so conditionally defined" ("optionally"
defined); rather than "optional" in the sense of "code still works
without the attribute". It caught Rasmus in one of his patches sent a
few days ago on top of this tree, so I want to change it or explain it
to avoid confusion.]

Cheers,
Miguel

Re: git pull

[Linus Torvalds]

On Tue, Nov 14, 2017 at 1:46 PM, Linus Torvalds
 wrote:
> And then people can do this:
>
>   [url "ssh://g...@gitolite.kernel.org"]
>   insteadOf = https://git.kernel.org
>   insteadOf = http://git.kernel.org
>   insteadOf = git://git.kernel.org
>
> which makes git.kernel.org addresses use ssh, and avoid the whole
> possible DNS spoofing problem.

So credit goes for Konstantin for pointing that out, and I actually
used it this merge window.

A few notes for other people who end up doing this:

 (a) ssh is slower, and the gitolite machine is not as reachable.

 (b) it affects your merge commit message.

As to (a), yes it's noticeable, but the extra couple of seconds isn't
really that big of a deal. Depending on exactly where you are, though,
you might end up wanting to use https:// to the public servers
instead.

But (b) actually ends up being annoying, because I don't like my merge
commits to contain references to repositories that aren't actually
available unless you have a kernel.org account.

I tried to edit things up by hand, but honestly, that just meant that
I forgot about 50% of the time. Do a

git log --author=Torvalds --grep=ssh://gitolite

to see my shameful lack of actually fixing up the end result.

Happily, once you realize that you have the attention span of a
slightly retarded chipmunk, and that you keep on forgetting to fix
things up, you hopefully also go "I'm a moron, but I can compensate
for that automatically".

Which is simple. Just create a .git/hooks/prepare-commit-msg file that contains

  #!/bin/sh
  sed -i 's|ssh://gitolite.kernel.org/|git://git.kernel.org/|g' "$1"

and make it executable, and git will do that commit message editing for you.

Tadaa! Now you don't look like quite the tool that I did.

  Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull

[Michael Ellerman]

Linus Torvalds  writes:

> On Tue, Nov 14, 2017 at 1:33 PM, Tobin C. Harding  wrote:
>>
>> Linus do you care what protocol? I'm patching Documentation and since
>> the point is creating pull requests for you 'some people' don't matter.
>
> I actually tend to prefer the regular git:// protocol and signed tags.
>
> It's true that https should have the proper certificate and perhaps
> help with DNS spoofing, but I'm not convinced that git won't just
> accept self-signed random certs, and I basically don't think we should
> trust that.

git does not accept self-signed certs by default, at least in recent
versions.

Though you can do a trust-on-first-use type thing, by downloading the
cert and telling git where to find it.

So https does provide additional security vs git:// IMHO. There is some
verification of the server and your data is encrypted on the wire.

It's not like it would be trivial to MITM a git fetch to insert a
malicious Makefile change, but it's also not *hard*.

> In contrast, using ssh I would actually trust, but it's not convenient
> and involves people sending things that aren't necessarily publicly
> available.
>
> So instead, I prefer just using git:// and not trying to fool people
> into thinking the protocol is secure - the security should come from
> the signed tag.

That's true, but only when you're pulling a signed tag, which for most
people is not the common case.

...
> That said, I actually would prefer even kernel.org repositories to
> just send pull requests with signed tags, despite the protocol itself
> being secure for that (and only that).

Which you mention here.

cheers
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull

[Linus Torvalds]

On Tue, Nov 14, 2017 at 1:33 PM, Tobin C. Harding  wrote:
>
> Linus do you care what protocol? I'm patching Documentation and since
> the point is creating pull requests for you 'some people' don't matter.

I actually tend to prefer the regular git:// protocol and signed tags.

It's true that https should have the proper certificate and perhaps
help with DNS spoofing, but I'm not convinced that git won't just
accept self-signed random certs, and I basically don't think we should
trust that.

In contrast, using ssh I would actually trust, but it's not convenient
and involves people sending things that aren't necessarily publicly
available.

So instead, I prefer just using git:// and not trying to fool people
into thinking the protocol is secure - the security should come from
the signed tag.

And then people can do this:

  [url "ssh://g...@gitolite.kernel.org"]
  insteadOf = https://git.kernel.org
  insteadOf = http://git.kernel.org
  insteadOf = git://git.kernel.org

which makes git.kernel.org addresses use ssh, and avoid the whole
possible DNS spoofing problem.

That said, I actually would prefer even kernel.org repositories to
just send pull requests with signed tags, despite the protocol itself
being secure for that (and only that).

Other hosts I will simply not trust without it because I can't do the above.

Side note: there's an unrelated advantage of using "git://" over
"https://;. It means that people who do automation see that it's a git
repo. It also means, for example, that people that highlight https://
URL's and perhaps use them for spam marking hopefully don't do that
with git:// format.

  Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull

[Tobin C. Harding]

On Tue, Nov 14, 2017 at 12:05:00PM +0100, Greg Kroah-Hartman wrote:
> Adding lkml and linux-doc mailing lists...
> 
> On Tue, Nov 14, 2017 at 10:11:55AM +1100, Tobin C. Harding wrote:
> > Hi Greg,
> > 
> > This is totally asking a favour, feel free to ignore. How do you format
> > your [GIT PULL] emails to Linus? Do you create a tag and then run a git
> > command to get the email?
> > 
> > I tried to do it manually and failed pretty hard (as you no doubt will
> > notice on LKML).
> 
> Well, I think you got it right the third time, so nice job :)

Lucky. Three strikes and your out isn't it?

> Anyway, this actually came up at the kernel summit / maintainer meeting
> a few weeks ago, in that "how do I make a good pull request to Linus" is
> something we need to document.
> 
> Here's what I do, and it seems to work well, so maybe we should turn it
> into the start of the documentation for how to do it.

Patch to come.

> ---
> 
> To start with, put your changes on a branch, hopefully one named in a
> semi-useful way (I use 'char-misc-next' for my char/misc driver patches
> to be merged into linux-next).  That is the branch you wish to tag and
> have Linus pull from.
> 
> Name the tag with something useful that you can understand if you run
> across it in a few weeks, and something that will be "unique".
> Continuing the example of my char-misc tree, for the patches to be sent
> to Linus for the 4.15-rc1 merge window, I would name the tag
> 'char-misc-4.15-rc1':
>   git tag -u KEY_ID -s char-misc-4.15-rc1 char-misc-next
> 
> that will create a signed tag called 'char-misc-4.15-rc1' based on the
> last commit in the char-misc-next branch, and sign it with my gpg key
> KEY_ID (replace KEY_ID with your own gpg key id.)
> 
> When you run the above command, git will drop you into an editor and ask
> you to describe the tag.  In this case, you are describing a pull
> request, so outline what is contained here, why it should be merged, and
> what, if any, testing has happened to it.  All of this information will
> end up in the tag itself, and then in the merge commit that Linus makes,
> so write it up well, as it will be in the kernel tree for forever.
> 
> An example pull request of mine might look like:
>   Char/Misc patches for 4.15-rc1
> 
>   Here is the big char/misc patch set for the 4.15-rc1 merge
>   window.  Contained in here is the normal set of new functions
>   added to all of these crazy drivers, as well as the following
>   brand new subsystems:
>   - time_travel_controller: Finally a set of drivers for
> the latest time travel bus architecture that provides
> i/o to the CPU before it asked for it, allowing
> uninterrupted processing
>   - relativity_shifters: due to the affect that the
> time_travel_controllers have on the overall system,
> there was a need for a new set of relativity shifter
> drivers to accommodate the newly formed black holes
> that would threaten to suck CPUs into them.  This
> subsystem handles this in a way to successfully
> neutralize the problems.  There is a Kconfig option to
> force these to be enabled when needed, so problems
> should not occur.
> 
>   All of these patches have been successfully tested in the latest
>   linux-next releases, and the original problems that it found
>   have all been resolved (apologies to anyone living near Canberra
>   for the lack of the Kconfig options in the earlier versions of
>   the linux-next tree creations.)
> 
>   Signed-off-by: Your-name-here 
> 
> 
> The tag message format is just like a git commit id.  One line at the
> top for a "summary subject" and be sure to sign-off at the bottom.
> 
> Now that you have a local signed tag, you need to push it up to where it
> can be retrieved by others:
>   git push origin char-misc-4.15-rc1
> pushes the char-misc-4.15-rc1 tag to where the 'origin' repo is located.
> 
> The last thing to do is create the pull request message.  Git handily
> will do this for you with the 'git request-pull' command, but it needs a
> bit of help determining what you want to pull, and what to base the pull
> against (to show the correct changes to be pulled and the diffstat.)
> 
> I use the following command to generate a pull request:
>   git request-pull master 
> git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ 
> char-misc-4.15-rc1
> 
> This is asking git to compare the difference from the
> 'char-misc-4.15-rc1' tag location, to the head of the 'master' branch
> (which in my case points to the last location in Linus's tree that I
> diverged from, usually a -rc release) and to use the git:// protocol to
> pull from.  If you wish to use https://, that can be used here instead
> as well (but note that some people 

Re: git pull

[Tobin C. Harding]

Added Linus to To: header.

On Tue, Nov 14, 2017 at 12:05:00PM +0100, Greg Kroah-Hartman wrote:
> Adding lkml and linux-doc mailing lists...
> 
> On Tue, Nov 14, 2017 at 10:11:55AM +1100, Tobin C. Harding wrote:
> > Hi Greg,
> > 
> > This is totally asking a favour, feel free to ignore. How do you format
> > your [GIT PULL] emails to Linus? Do you create a tag and then run a git
> > command to get the email?
> > 
> > I tried to do it manually and failed pretty hard (as you no doubt will
> > notice on LKML).
> 
> Well, I think you got it right the third time, so nice job :)
> 
> Anyway, this actually came up at the kernel summit / maintainer meeting
> a few weeks ago, in that "how do I make a good pull request to Linus" is
> something we need to document.
> 
> Here's what I do, and it seems to work well, so maybe we should turn it
> into the start of the documentation for how to do it.
> 
> ---
> 
> To start with, put your changes on a branch, hopefully one named in a
> semi-useful way (I use 'char-misc-next' for my char/misc driver patches
> to be merged into linux-next).  That is the branch you wish to tag and
> have Linus pull from.
> 
> Name the tag with something useful that you can understand if you run
> across it in a few weeks, and something that will be "unique".
> Continuing the example of my char-misc tree, for the patches to be sent
> to Linus for the 4.15-rc1 merge window, I would name the tag
> 'char-misc-4.15-rc1':
>   git tag -u KEY_ID -s char-misc-4.15-rc1 char-misc-next
> 
> that will create a signed tag called 'char-misc-4.15-rc1' based on the
> last commit in the char-misc-next branch, and sign it with my gpg key
> KEY_ID (replace KEY_ID with your own gpg key id.)
> 
> When you run the above command, git will drop you into an editor and ask
> you to describe the tag.  In this case, you are describing a pull
> request, so outline what is contained here, why it should be merged, and
> what, if any, testing has happened to it.  All of this information will
> end up in the tag itself, and then in the merge commit that Linus makes,
> so write it up well, as it will be in the kernel tree for forever.
> 
> An example pull request of mine might look like:
>   Char/Misc patches for 4.15-rc1
> 
>   Here is the big char/misc patch set for the 4.15-rc1 merge
>   window.  Contained in here is the normal set of new functions
>   added to all of these crazy drivers, as well as the following
>   brand new subsystems:
>   - time_travel_controller: Finally a set of drivers for
> the latest time travel bus architecture that provides
> i/o to the CPU before it asked for it, allowing
> uninterrupted processing
>   - relativity_shifters: due to the affect that the
> time_travel_controllers have on the overall system,
> there was a need for a new set of relativity shifter
> drivers to accommodate the newly formed black holes
> that would threaten to suck CPUs into them.  This
> subsystem handles this in a way to successfully
> neutralize the problems.  There is a Kconfig option to
> force these to be enabled when needed, so problems
> should not occur.
> 
>   All of these patches have been successfully tested in the latest
>   linux-next releases, and the original problems that it found
>   have all been resolved (apologies to anyone living near Canberra
>   for the lack of the Kconfig options in the earlier versions of
>   the linux-next tree creations.)
> 
>   Signed-off-by: Your-name-here 
> 
> 
> The tag message format is just like a git commit id.  One line at the
> top for a "summary subject" and be sure to sign-off at the bottom.
> 
> Now that you have a local signed tag, you need to push it up to where it
> can be retrieved by others:
>   git push origin char-misc-4.15-rc1
> pushes the char-misc-4.15-rc1 tag to where the 'origin' repo is located.
> 
> The last thing to do is create the pull request message.  Git handily
> will do this for you with the 'git request-pull' command, but it needs a
> bit of help determining what you want to pull, and what to base the pull
> against (to show the correct changes to be pulled and the diffstat.)
> 
> I use the following command to generate a pull request:
>   git request-pull master 
> git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ 
> char-misc-4.15-rc1
> 
> This is asking git to compare the difference from the
> 'char-misc-4.15-rc1' tag location, to the head of the 'master' branch
> (which in my case points to the last location in Linus's tree that I
> diverged from, usually a -rc release) and to use the git:// protocol to
> pull from.  If you wish to use https://, that can be used here instead
> as well (but note that some people behind firewalls will have 

Re: git pull

[Linus Torvalds]

On Tue, Nov 14, 2017 at 3:05 AM, Greg Kroah-Hartman
 wrote:
>
> Name the tag with something useful that you can understand if you run
> across it in a few weeks, and something that will be "unique".
> Continuing the example of my char-misc tree, for the patches to be sent
> to Linus for the 4.15-rc1 merge window, I would name the tag
> 'char-misc-4.15-rc1':
> git tag -u KEY_ID -s char-misc-4.15-rc1 char-misc-next

Side note: since you _usually_ would use the same key for the same
project, just set it once with

git config user.signingkey "keyname"

and if you use the same key for everything, just add "--global".

Or just edit your .git/config or ~/.gitconfig file by hand, it's
designed to be human-readable and writable, and not some garbage like
XML:

   [torvalds@i7 ~]$ head -4 .gitconfig
   [user]
name = Linus Torvalds
email = torva...@linux-foundation.org
signingkey = torva...@linux-foundation.org

it's not really all that complicated ;)

Then you don't  need that "-u KEY_ID" when you sign things.

Anyway, at least to me, the important part is the *message*. I want to
understand what I'm pulling, and why I should pull it. I also want to
use that message as the message for the merge, so it should not just
make sense to me, but make sense as a historical record too.

Note that if there is something odd about the pull request, that
should very much be in the explanation. If you're touching files that
you don't maintain, explain _why_. I will see it in the diffstat
anyway, and if you didn't mention it, I'll just be extra suspicious.
And when you send me new stuff after the merge window (or even
bug-fixes, but ones that look scary), explain not just what they do
and why they do it, but explain the _timing_. What happened that this
didn't go through the merge window..

I will take both what you write in the email pull request _and_ in the
signed tag, so depending on your workflow, you can either describe
your work in the signed tag (which will also automatically make it
into the pull request email), or you can make the signed tag just a
placeholder with nothing interesting in it, and describe the work
later when you actually send me the pull request.

And yes, I will edit the message. Partly because I tend to do just
trivial formatting (the whole indentation and quoting etc), but partly
because part of the message may make sense for me at pull time
(describing the conflicts and your personal issues for sending it
right now), but may not make sense in the context of a merge commit
message, so I will try to make it all make sense. I will also fix any
speeling mistaeks and bad grammar I notice, particularly for
non-native speakers (but also for native ones ;^). But I may miss
some, or even add some.

   Linus
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull

[Greg Kroah-Hartman]

On Tue, Nov 14, 2017 at 01:00:14PM +0100, Ulf Hansson wrote:
> [...]
> 
> >
> > An example pull request of mine might look like:
> > Char/Misc patches for 4.15-rc1
> >
> > Here is the big char/misc patch set for the 4.15-rc1 merge
> > window.  Contained in here is the normal set of new functions
> > added to all of these crazy drivers, as well as the following
> > brand new subsystems:
> > - time_travel_controller: Finally a set of drivers for
> >   the latest time travel bus architecture that provides
> >   i/o to the CPU before it asked for it, allowing
> >   uninterrupted processing
> > - relativity_shifters: due to the affect that the
> >   time_travel_controllers have on the overall system,
> >   there was a need for a new set of relativity shifter
> >   drivers to accommodate the newly formed black holes
> >   that would threaten to suck CPUs into them.  This
> >   subsystem handles this in a way to successfully
> >   neutralize the problems.  There is a Kconfig option to
> >   force these to be enabled when needed, so problems
> >   should not occur.
> >
> > All of these patches have been successfully tested in the latest
> > linux-next releases, and the original problems that it found
> > have all been resolved (apologies to anyone living near Canberra
> > for the lack of the Kconfig options in the earlier versions of
> > the linux-next tree creations.)
> >
> > Signed-off-by: Your-name-here 
> >
> >
> > The tag message format is just like a git commit id.  One line at the
> > top for a "summary subject" and be sure to sign-off at the bottom.
> 
> I don't add my s-o-b to signed tags for pull requests, but perhaps I should.
> 
> However, I think most maintainers don't use it, and neither does it
> seems like Linus is preserving the tag when he does the pull.

The text of the tag is in the merge commit, but you are right, the
signed-off-by doesn't seem to be in the merge commit, I guess Linus's
workflow removes them.  I know I keep them in there if present for pull
requests that people send to me.

thanks,

greg k-h
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull

[Ulf Hansson]

[...]

>
> An example pull request of mine might look like:
> Char/Misc patches for 4.15-rc1
>
> Here is the big char/misc patch set for the 4.15-rc1 merge
> window.  Contained in here is the normal set of new functions
> added to all of these crazy drivers, as well as the following
> brand new subsystems:
> - time_travel_controller: Finally a set of drivers for
>   the latest time travel bus architecture that provides
>   i/o to the CPU before it asked for it, allowing
>   uninterrupted processing
> - relativity_shifters: due to the affect that the
>   time_travel_controllers have on the overall system,
>   there was a need for a new set of relativity shifter
>   drivers to accommodate the newly formed black holes
>   that would threaten to suck CPUs into them.  This
>   subsystem handles this in a way to successfully
>   neutralize the problems.  There is a Kconfig option to
>   force these to be enabled when needed, so problems
>   should not occur.
>
> All of these patches have been successfully tested in the latest
> linux-next releases, and the original problems that it found
> have all been resolved (apologies to anyone living near Canberra
> for the lack of the Kconfig options in the earlier versions of
> the linux-next tree creations.)
>
> Signed-off-by: Your-name-here 
>
>
> The tag message format is just like a git commit id.  One line at the
> top for a "summary subject" and be sure to sign-off at the bottom.

I don't add my s-o-b to signed tags for pull requests, but perhaps I should.

However, I think most maintainers don't use it, and neither does it
seems like Linus is preserving the tag when he does the pull.

[...]

Kind regards
Uffe
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: git pull

[Greg Kroah-Hartman]

Adding lkml and linux-doc mailing lists...

On Tue, Nov 14, 2017 at 10:11:55AM +1100, Tobin C. Harding wrote:
> Hi Greg,
> 
> This is totally asking a favour, feel free to ignore. How do you format
> your [GIT PULL] emails to Linus? Do you create a tag and then run a git
> command to get the email?
> 
> I tried to do it manually and failed pretty hard (as you no doubt will
> notice on LKML).

Well, I think you got it right the third time, so nice job :)

Anyway, this actually came up at the kernel summit / maintainer meeting
a few weeks ago, in that "how do I make a good pull request to Linus" is
something we need to document.

Here's what I do, and it seems to work well, so maybe we should turn it
into the start of the documentation for how to do it.

---

To start with, put your changes on a branch, hopefully one named in a
semi-useful way (I use 'char-misc-next' for my char/misc driver patches
to be merged into linux-next).  That is the branch you wish to tag and
have Linus pull from.

Name the tag with something useful that you can understand if you run
across it in a few weeks, and something that will be "unique".
Continuing the example of my char-misc tree, for the patches to be sent
to Linus for the 4.15-rc1 merge window, I would name the tag
'char-misc-4.15-rc1':
git tag -u KEY_ID -s char-misc-4.15-rc1 char-misc-next

that will create a signed tag called 'char-misc-4.15-rc1' based on the
last commit in the char-misc-next branch, and sign it with my gpg key
KEY_ID (replace KEY_ID with your own gpg key id.)

When you run the above command, git will drop you into an editor and ask
you to describe the tag.  In this case, you are describing a pull
request, so outline what is contained here, why it should be merged, and
what, if any, testing has happened to it.  All of this information will
end up in the tag itself, and then in the merge commit that Linus makes,
so write it up well, as it will be in the kernel tree for forever.

An example pull request of mine might look like:
Char/Misc patches for 4.15-rc1

Here is the big char/misc patch set for the 4.15-rc1 merge
window.  Contained in here is the normal set of new functions
added to all of these crazy drivers, as well as the following
brand new subsystems:
- time_travel_controller: Finally a set of drivers for
  the latest time travel bus architecture that provides
  i/o to the CPU before it asked for it, allowing
  uninterrupted processing
- relativity_shifters: due to the affect that the
  time_travel_controllers have on the overall system,
  there was a need for a new set of relativity shifter
  drivers to accommodate the newly formed black holes
  that would threaten to suck CPUs into them.  This
  subsystem handles this in a way to successfully
  neutralize the problems.  There is a Kconfig option to
  force these to be enabled when needed, so problems
  should not occur.

All of these patches have been successfully tested in the latest
linux-next releases, and the original problems that it found
have all been resolved (apologies to anyone living near Canberra
for the lack of the Kconfig options in the earlier versions of
the linux-next tree creations.)

Signed-off-by: Your-name-here 


The tag message format is just like a git commit id.  One line at the
top for a "summary subject" and be sure to sign-off at the bottom.

Now that you have a local signed tag, you need to push it up to where it
can be retrieved by others:
git push origin char-misc-4.15-rc1
pushes the char-misc-4.15-rc1 tag to where the 'origin' repo is located.

The last thing to do is create the pull request message.  Git handily
will do this for you with the 'git request-pull' command, but it needs a
bit of help determining what you want to pull, and what to base the pull
against (to show the correct changes to be pulled and the diffstat.)

I use the following command to generate a pull request:
git request-pull master 
git://git.kernel.org/pub/scm/linux/kernel/git/gregkh/char-misc.git/ 
char-misc-4.15-rc1

This is asking git to compare the difference from the
'char-misc-4.15-rc1' tag location, to the head of the 'master' branch
(which in my case points to the last location in Linus's tree that I
diverged from, usually a -rc release) and to use the git:// protocol to
pull from.  If you wish to use https://, that can be used here instead
as well (but note that some people behind firewalls will have problems
with https git pulls).

If the char-misc-4.15-rc1 tag is not present in the repo that I am
asking to be pulled from, git will complain saying it is not there, a
handy way to remember to actually push it to a public location.

The 

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Kishon Vijay Abraham I]

Hi Bjorn,

On Wednesday 12 April 2017 01:04 AM, Bjorn Helgaas wrote:
> On Mon, Apr 10, 2017 at 10:43:28AM -0500, Bjorn Helgaas wrote:
>> On Wed, Apr 05, 2017 at 02:22:20PM +0530, Kishon Vijay Abraham I wrote:
>>> Hi Bjorn,
>>>
>>> Please find the pull request for PCI endpoint support below. I've
>>> also included all the history here.
>>
>> Thanks, I applied these (with v7 of the first patch) to pci/host-designware
>> for v4.12.
> 
> Ok, sorry, I screwed this up.  I think my branch actually had v5, not
> v6.  But I *think* I fixed it.  Here's the diff from my branch to your
> git tree.  Apparently you haven't pushed the v7 patch there, so I
> *think* the diff below is the diff between v6 and v7 of that first
> patch.

I just checked your pci/host-designware branch and it looks correct. Thanks for
sorting this out.

Cheers
Kishon

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Bjorn Helgaas]

On Mon, Apr 10, 2017 at 10:43:28AM -0500, Bjorn Helgaas wrote:
> On Wed, Apr 05, 2017 at 02:22:20PM +0530, Kishon Vijay Abraham I wrote:
> > Hi Bjorn,
> > 
> > Please find the pull request for PCI endpoint support below. I've
> > also included all the history here.
> 
> Thanks, I applied these (with v7 of the first patch) to pci/host-designware
> for v4.12.

Ok, sorry, I screwed this up.  I think my branch actually had v5, not
v6.  But I *think* I fixed it.  Here's the diff from my branch to your
git tree.  Apparently you haven't pushed the v7 patch there, so I
*think* the diff below is the diff between v6 and v7 of that first
patch.

$ git diff pci/host-designware a5c85ba45c96
diff --git a/drivers/pci/endpoint/pci-epc-core.c 
b/drivers/pci/endpoint/pci-epc-core.c
index caa7be10e473..9ae9e59b2a74 100644
--- a/drivers/pci/endpoint/pci-epc-core.c
+++ b/drivers/pci/endpoint/pci-epc-core.c
@@ -83,7 +83,6 @@ struct pci_epc *pci_epc_get(const char *epc_name)
goto err;
}
 
-   class_dev_iter_exit();
get_device(>dev);
return epc;
}
diff --git a/drivers/pci/endpoint/pci-epf-core.c 
b/drivers/pci/endpoint/pci-epf-core.c
index 6877d6a5bcc9..92db7dcd911c 100644
--- a/drivers/pci/endpoint/pci-epf-core.c
+++ b/drivers/pci/endpoint/pci-epf-core.c
@@ -40,10 +40,8 @@ static struct device_type pci_epf_type;
  */
 void pci_epf_linkup(struct pci_epf *epf)
 {
-   if (!epf->driver) {
+   if (!epf->driver)
dev_WARN(>dev, "epf device not bound to driver\n");
-   return;
-   }
 
epf->driver->ops->linkup(epf);
 }
@@ -59,10 +57,8 @@ EXPORT_SYMBOL_GPL(pci_epf_linkup);
  */
 void pci_epf_unbind(struct pci_epf *epf)
 {
-   if (!epf->driver) {
+   if (!epf->driver)
dev_WARN(>dev, "epf device not bound to driver\n");
-   return;
-   }
 
epf->driver->ops->unbind(epf);
module_put(epf->driver->owner);
@@ -78,10 +74,8 @@ EXPORT_SYMBOL_GPL(pci_epf_unbind);
  */
 int pci_epf_bind(struct pci_epf *epf)
 {
-   if (!epf->driver) {
+   if (!epf->driver)
dev_WARN(>dev, "epf device not bound to driver\n");
-   return -EINVAL;
-   }
 
if (!try_module_get(epf->driver->owner))
return -EAGAIN;
@@ -233,7 +227,7 @@ struct pci_epf *pci_epf_create(const char *name)
epf->name = kstrdup(func_name, GFP_KERNEL);
if (!epf->name) {
ret = -ENOMEM;
-   goto free_func_name;
+   goto free_epf;
}
 
dev = >dev;
@@ -255,8 +249,6 @@ struct pci_epf *pci_epf_create(const char *name)
 put_dev:
put_device(dev);
kfree(epf->name);
-
-free_func_name:
kfree(func_name);
 
 free_epf:
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Bjorn Helgaas]

On Wed, Apr 05, 2017 at 02:22:20PM +0530, Kishon Vijay Abraham I wrote:
> Hi Bjorn,
> 
> Please find the pull request for PCI endpoint support below. I've
> also included all the history here.

Thanks, I applied these (with v7 of the first patch) to pci/host-designware
for v4.12.

> Changes from v5:
> *) remove #syscon-cells property added in v5 and used 
>of_parse_phandle_with_fixed_args
> *) fix compilation error in make.cross ARCH=blackfin that was because
>pci_endpoint_test.c driver depends on COMPILE_TEST.
> 
> Changes from v4:
> *) add #syscon-cells property and used of_parse_phandle_with_args
>to perform a configuration in syscon module (as suggested by
>Rob Herring)
> *) Remove unnecessary white space.
> 
> Changes from v3:
> *) fixed a typo and adapted to https://lkml.org/lkml/2017/3/13/562.
> 
> Changes from v2:
> *) changed the configfs structure as suggested by Christoph Hellwig. With
>this change the framework creates configfs entry for EP function driver
>and EP controller. Previously these entries have to be created by the
>the user. (Haven't changed the epc core or epf core except for invoking
>configfs APIs to create entries for EP function driver and EP controller.
>That's mostly because the EP function device can still be created by
>directly invoking the epf core API without using configfs).
> *) Now the user has to use configfs entry 'start' to start the link.
>This was previously done by the function driver. However in the case of
>multi function EP, the function driver shouldn't start the link.
> 
> Changes from v1:
> *) The preparation patches for adding EP support is removed and is sent
>separately
> *) Added device ID for DRA74x/DRA72x and used it instead of
>using "PCI_ANY_ID"
> *) Added userguide for PCI endpoint test function
> 
> Major Improvements from RFC:
>  *) support multi-function devices (hw supported not virtual)
>  *) Access host side buffers
>  *) Raise MSI interrupts
>  *) Add user space program to use the host side PCI driver
>  *) Adapt all other users of designware to use the new design (only
> compile tested. Since I have only dra7xx boards, the new design
> has only been tested in dra7xx. I'd require the help of others
> to test the platforms they have access to).
> 
> This is based on
> git://git.kernel.org/pub/scm/linux/kernel/git/helgaas/pci.git
> pci/host-designware
> 
> Thanks
> Kishon
> 
> The following changes since commit 7ea64dcf602c21b3e5062ca90111ca4459fab403:
> 
>   __end__ (2017-04-04 15:29:37 -0500)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/kishon/pci-endpoint.git 
> tags/pci-endpoint-for-4.12
> 
> for you to fetch changes up to a5c85ba45c9682456077d7277196e91f8ea5fd5c:
> 
>   ARM: DRA7: clockdomain: Change the CLKTRCTRL of CM_PCIE_CLKSTCTRL to 
> SW_WKUP (2017-04-05 14:05:28 +0530)
> 
> 
> pci: endpoint: for 4.12
> 
>  *) Add PCI endpoint core layer
>  *) Modify designware and dra7xx driver to be configured in EP mode
>  *) Add a PCI endpoint *test* function driver and corresponding host
> driver
> 
> Signed-off-by: Kishon Vijay Abraham I 
> 
> 
> Kishon Vijay Abraham I (23):
>   PCI: endpoint: Add EP core layer to enable EP controller and EP 
> functions
>   Documentation: PCI: Guide to use PCI Endpoint Core Layer
>   PCI: endpoint: Introduce configfs entry for configuring EP functions
>   Documentation: PCI: Guide to use PCI endpoint configfs
>   PCI: endpoint: Create configfs entry for EPC device and EPF driver
>   Documentation: PCI: Add specification for the *PCI test* function device
>   PCI: endpoint: functions: Add an EP function to test PCI
>   Documentation: PCI: Add binding documentation for pci-test endpoint 
> function
>   PCI: dwc: designware: Add EP mode support
>   dt-bindings: PCI: Add DT bindings for PCI designware EP mode
>   PCI: dwc: dra7xx: Facilitate wrapper and MSI interrupts to be enabled 
> independently
>   PCI: dwc: dra7xx: Add EP mode support
>   dt-bindings: PCI: dra7xx: Add DT bindings for PCI dra7xx EP mode
>   PCI: dwc: dra7xx: Workaround for errata id i870
>   dt-bindings: PCI: dra7xx: Add DT bindings to enable unaligned access
>   PCI: Add device IDs for DRA74x and DRA72x
>   misc: Add host side PCI driver for PCI test function device
>   Documentation: misc-devices: Add Documentation for pci-endpoint-test 
> driver
>   tools: PCI: Add a userspace tool to test PCI endpoint
>   tools: PCI: Add sample test script to invoke pcitest
>   Documentation: PCI: Add userguide for PCI endpoint test function
>   MAINTAINERS: Add PCI Endpoint maintainer
>   ARM: DRA7: clockdomain: Change the CLKTRCTRL of CM_PCIE_CLKSTCTRL to 
> SW_WKUP
> 
>  Documentation/PCI/00-INDEX 

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Kishon Vijay Abraham I]

Hi Bjorn,

On Wednesday 05 April 2017 02:06 AM, Bjorn Helgaas wrote:
> On Mon, Mar 27, 2017 at 03:14:56PM +0530, Kishon Vijay Abraham I wrote:
>> Hi Bjorn,
>>
>> Please find the pull request for PCI endpoint support below. I've
>> also included all the history here.
> 
> I tentatively applied this to pci/host-designware with the mostly trival
> textual changes below.  If you post the series again, please include them.

Sure, I've used pci/host-designware as the base for the next revision.
> 
> I saw some acks to prior revisions, but few of them were included in this
> series.   Can you collect them up?  If there are no other substantial
> changes, I can insert them into my branch manually.

Yeah, looks like I've missed including Joao Pinto's Acked-by in some of the
patches. I'll include them in the next revision.

Thanks
Kishon
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Bjorn Helgaas]

On Mon, Mar 27, 2017 at 03:14:56PM +0530, Kishon Vijay Abraham I wrote:
> Hi Bjorn,
> 
> Please find the pull request for PCI endpoint support below. I've
> also included all the history here.

I tentatively applied this to pci/host-designware with the mostly trival
textual changes below.  If you post the series again, please include them.

I saw some acks to prior revisions, but few of them were included in this
series.   Can you collect them up?  If there are no other substantial
changes, I can insert them into my branch manually.

Bjorn



diff --git a/Documentation/PCI/00-INDEX b/Documentation/PCI/00-INDEX
index fd533c79fa19..00c9a90b6f38 100644
--- a/Documentation/PCI/00-INDEX
+++ b/Documentation/PCI/00-INDEX
@@ -15,10 +15,10 @@ pcieaer-howto.txt
 endpoint/pci-endpoint.txt
- guide to add endpoint controller driver and endpoint function driver.
 endpoint/pci-endpoint-cfs.txt
-   - guide to use configfs to configure the pci endpoint function.
+   - guide to use configfs to configure the PCI endpoint function.
 endpoint/pci-test-function.txt
-   - specification of *pci test* function device.
+   - specification of *PCI test* function device.
 endpoint/pci-test-howto.txt
- userguide for PCI endpoint test function.
 endpoint/function/binding/
-   - binding documentation for pci endpoint function
+   - binding documentation for PCI endpoint function
diff --git a/Documentation/PCI/endpoint/function/binding/pci-test.txt 
b/Documentation/PCI/endpoint/function/binding/pci-test.txt
index c44fc18d78cc..3b68b955fb50 100644
--- a/Documentation/PCI/endpoint/function/binding/pci-test.txt
+++ b/Documentation/PCI/endpoint/function/binding/pci-test.txt
@@ -13,5 +13,5 @@ cache_line_size: don't care
 subsys_vendor_id : don't care
 subsys_id   : don't care
 interrupt_pin   : Should be 1 - INTA, 2 - INTB, 3 - INTC, 4 -INTD
-msi_interrupts  : Should be 1 to 32 depending on the number of msi interrupts
+msi_interrupts  : Should be 1 to 32 depending on the number of MSI interrupts
   to test
diff --git a/Documentation/PCI/endpoint/pci-endpoint-cfs.txt 
b/Documentation/PCI/endpoint/pci-endpoint-cfs.txt
index 8b2a8280b131..d740f29960a4 100644
--- a/Documentation/PCI/endpoint/pci-endpoint-cfs.txt
+++ b/Documentation/PCI/endpoint/pci-endpoint-cfs.txt
@@ -1,10 +1,10 @@
CONFIGURING PCI ENDPOINT USING CONFIGFS
 Kishon Vijay Abraham I 
 
-The PCI Endpoint Core exposes configfs entry (pci_ep) in order to configure the
-PCI endpoint function and in order to bind the endpoint function
+The PCI Endpoint Core exposes configfs entry (pci_ep) to configure the
+PCI endpoint function and to bind the endpoint function
 with the endpoint controller. (For introducing other mechanisms to
-configure the PCI Endpoint Function refer [1]).
+configure the PCI Endpoint Function refer to [1]).
 
 *) Mounting configfs
 
diff --git a/Documentation/PCI/endpoint/pci-endpoint.txt 
b/Documentation/PCI/endpoint/pci-endpoint.txt
index 4a3e4388b37b..9b1d66829290 100644
--- a/Documentation/PCI/endpoint/pci-endpoint.txt
+++ b/Documentation/PCI/endpoint/pci-endpoint.txt
@@ -2,27 +2,27 @@
Kishon Vijay Abraham I 
 
 This document is a guide to use the PCI Endpoint Framework in order to create
-endpoint controller driver, endpoint function driver and using configfs
+endpoint controller driver, endpoint function driver, and using configfs
 interface to bind the function driver to the controller driver.
 
 1. Introduction
 
-*Linux* has a comprehensive PCI subsystem to support PCI controllers that
+Linux has a comprehensive PCI subsystem to support PCI controllers that
 operates in Root Complex mode. The subsystem has capability to scan PCI bus,
-assign memory resources and irq resources, load PCI driver (based on
-vendorid, deviceid), support other services like hot-plug, power management,
+assign memory resources and IRQ resources, load PCI driver (based on
+vendor ID, device ID), support other services like hot-plug, power management,
 advanced error reporting and virtual channels.
 
-However PCI controller IPs integrated in certain SoC is capable of operating
+However the PCI controller IP integrated in some SoCs is capable of operating
 either in Root Complex mode or Endpoint mode. PCI Endpoint Framework will
-add endpoint mode support in *Linux*. This will help to run Linux in an
+add endpoint mode support in Linux. This will help to run Linux in an
 EP system which can have a wide variety of use cases from testing or
-validation, co-processor accelerator etc..
+validation, co-processor accelerator, etc.
 
 2. PCI Endpoint Core
 
-The PCI Endpoint Core layer comprises of 3 components: the Endpoint Controller
-library, the Endpoint Function library and the configfs layer to bind the
+The PCI Endpoint Core layer comprises 3 components: the Endpoint Controller
+library, the Endpoint Function library, and the 

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Niklas Cassel]

On 03/29/2017 02:36 PM, Kishon Vijay Abraham I wrote:
(snip)
>>>
>>> FWIW:
>>> I've tested Kishon's tag pci-endpoint-for-4.12
>>> and PCIe on artpec6 SoC is still working fine.
>>
>> Thanks for testing it.
>>>
>>> I also included the DRA7xx PCIe driver in my
>>> kernel so that pcie-designware-ep.c gets built.
>>>
>>> My only worry is that the code in pcie-designware-ep.c
>>> is not compile tested if DRA7xx is not selected
>>> (as it is the only driver using PCIE_DW_EP at
>>> the moment).
>>
>> yeah, we should plan to include COMPILE_TEST in all pci drivers but I guess
>> there is some problem with non-ARM builds [1]. As Bjorn mentioned in the
>> thread, we could add #ifdef ARM and then include COMPILE_TEST.
> 
> I think I misunderstood your concern. yeah, there is no direct way to compile
> pcie-designware-ep.c without selecting DRA7xx.

There is no way to compile pcie-designware-host.c + pcie-designware.c
without selecting a driver that selects PCIE_DW_HOST,
so I guess the same "problem" applies there.

As long as kbuild builds PCI_DRA7XX_EP (and any driver that selects
PCIE_DW_HOST), we should be fine.
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Kishon Vijay Abraham I]

Hi,

On Wednesday 29 March 2017 05:40 PM, Kishon Vijay Abraham I wrote:
> Hi Niklas,
> 
> On Wednesday 29 March 2017 05:12 PM, Niklas Cassel wrote:
>> On 03/27/2017 11:44 AM, Kishon Vijay Abraham I wrote:
>>> Hi Bjorn,
>>>
>>> Please find the pull request for PCI endpoint support below. I've
>>> also included all the history here.
>>>
>>> Changes from v4:
>>> *) add #syscon-cells property and used of_parse_phandle_with_args
>>>to perform a configuration in syscon module (as suggested by
>>>Rob Herring)
>>> *) Remove unnecessary white space.
>>>
>>> Changes from v3:
>>> *) fixed a typo and adapted to https://lkml.org/lkml/2017/3/13/562.
>>>
>>> Changes from v2:
>>> *) changed the configfs structure as suggested by Christoph Hellwig. With
>>>this change the framework creates configfs entry for EP function driver
>>>and EP controller. Previously these entries have to be created by the
>>>the user. (Haven't changed the epc core or epf core except for invoking
>>>configfs APIs to create entries for EP function driver and EP controller.
>>>That's mostly because the EP function device can still be created by
>>>directly invoking the epf core API without using configfs).
>>> *) Now the user has to use configfs entry 'start' to start the link.
>>>This was previously done by the function driver. However in the case of
>>>multi function EP, the function driver shouldn't start the link.
>>>
>>> Changes from v1:
>>> *) The preparation patches for adding EP support is removed and is sent
>>>separately
>>> *) Added device ID for DRA74x/DRA72x and used it instead of
>>>using "PCI_ANY_ID"
>>> *) Added userguide for PCI endpoint test function
>>>
>>> Major Improvements from RFC:
>>>  *) support multi-function devices (hw supported not virtual)
>>>  *) Access host side buffers
>>>  *) Raise MSI interrupts
>>>  *) Add user space program to use the host side PCI driver
>>>  *) Adapt all other users of designware to use the new design (only
>>> compile tested. Since I have only dra7xx boards, the new design
>>> has only been tested in dra7xx. I'd require the help of others
>>> to test the platforms they have access to).
>>>
>>> This series has been developed over 4.11-rc1 + [1]
>>> [1] -> https://lkml.org/lkml/2017/3/13/562
>>>
>>> Let me know if this has to be re-based to some of your branch.
>>>
>>> Thanks
>>> Kishon
>>>
>>> The following changes since commit 623e87fec8ab7867fb51b3079196bd10718a60ce:
>>>
>>>   PCI: dwc: dra7xx: Push request_irq call to the bottom of probe 
>>> (2017-03-22 20:35:30 +0530)
>>>
>>> are available in the git repository at:
>>>
>>>   git://git.kernel.org/pub/scm/linux/kernel/git/kishon/pci-endpoint.git 
>>> tags/pci-endpoint-for-4.12
>>>
>>> for you to fetch changes up to e98bf80074be4654faae42fe0f5a622a776b6fdd:
>>>
>>>   ARM: DRA7: clockdomain: Change the CLKTRCTRL of CM_PCIE_CLKSTCTRL to 
>>> SW_WKUP (2017-03-27 15:08:22 +0530)
>>>
>>> 
>>
>> FWIW:
>> I've tested Kishon's tag pci-endpoint-for-4.12
>> and PCIe on artpec6 SoC is still working fine.
> 
> Thanks for testing it.
>>
>> I also included the DRA7xx PCIe driver in my
>> kernel so that pcie-designware-ep.c gets built.
>>
>> My only worry is that the code in pcie-designware-ep.c
>> is not compile tested if DRA7xx is not selected
>> (as it is the only driver using PCIE_DW_EP at
>> the moment).
> 
> yeah, we should plan to include COMPILE_TEST in all pci drivers but I guess
> there is some problem with non-ARM builds [1]. As Bjorn mentioned in the
> thread, we could add #ifdef ARM and then include COMPILE_TEST.

I think I misunderstood your concern. yeah, there is no direct way to compile
pcie-designware-ep.c without selecting DRA7xx.

Thanks
Kishon
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Kishon Vijay Abraham I]

Hi Niklas,

On Wednesday 29 March 2017 05:12 PM, Niklas Cassel wrote:
> On 03/27/2017 11:44 AM, Kishon Vijay Abraham I wrote:
>> Hi Bjorn,
>>
>> Please find the pull request for PCI endpoint support below. I've
>> also included all the history here.
>>
>> Changes from v4:
>> *) add #syscon-cells property and used of_parse_phandle_with_args
>>to perform a configuration in syscon module (as suggested by
>>Rob Herring)
>> *) Remove unnecessary white space.
>>
>> Changes from v3:
>> *) fixed a typo and adapted to https://lkml.org/lkml/2017/3/13/562.
>>
>> Changes from v2:
>> *) changed the configfs structure as suggested by Christoph Hellwig. With
>>this change the framework creates configfs entry for EP function driver
>>and EP controller. Previously these entries have to be created by the
>>the user. (Haven't changed the epc core or epf core except for invoking
>>configfs APIs to create entries for EP function driver and EP controller.
>>That's mostly because the EP function device can still be created by
>>directly invoking the epf core API without using configfs).
>> *) Now the user has to use configfs entry 'start' to start the link.
>>This was previously done by the function driver. However in the case of
>>multi function EP, the function driver shouldn't start the link.
>>
>> Changes from v1:
>> *) The preparation patches for adding EP support is removed and is sent
>>separately
>> *) Added device ID for DRA74x/DRA72x and used it instead of
>>using "PCI_ANY_ID"
>> *) Added userguide for PCI endpoint test function
>>
>> Major Improvements from RFC:
>>  *) support multi-function devices (hw supported not virtual)
>>  *) Access host side buffers
>>  *) Raise MSI interrupts
>>  *) Add user space program to use the host side PCI driver
>>  *) Adapt all other users of designware to use the new design (only
>> compile tested. Since I have only dra7xx boards, the new design
>> has only been tested in dra7xx. I'd require the help of others
>> to test the platforms they have access to).
>>
>> This series has been developed over 4.11-rc1 + [1]
>> [1] -> https://lkml.org/lkml/2017/3/13/562
>>
>> Let me know if this has to be re-based to some of your branch.
>>
>> Thanks
>> Kishon
>>
>> The following changes since commit 623e87fec8ab7867fb51b3079196bd10718a60ce:
>>
>>   PCI: dwc: dra7xx: Push request_irq call to the bottom of probe (2017-03-22 
>> 20:35:30 +0530)
>>
>> are available in the git repository at:
>>
>>   git://git.kernel.org/pub/scm/linux/kernel/git/kishon/pci-endpoint.git 
>> tags/pci-endpoint-for-4.12
>>
>> for you to fetch changes up to e98bf80074be4654faae42fe0f5a622a776b6fdd:
>>
>>   ARM: DRA7: clockdomain: Change the CLKTRCTRL of CM_PCIE_CLKSTCTRL to 
>> SW_WKUP (2017-03-27 15:08:22 +0530)
>>
>> 
> 
> FWIW:
> I've tested Kishon's tag pci-endpoint-for-4.12
> and PCIe on artpec6 SoC is still working fine.

Thanks for testing it.
> 
> I also included the DRA7xx PCIe driver in my
> kernel so that pcie-designware-ep.c gets built.
> 
> My only worry is that the code in pcie-designware-ep.c
> is not compile tested if DRA7xx is not selected
> (as it is the only driver using PCIE_DW_EP at
> the moment).

yeah, we should plan to include COMPILE_TEST in all pci drivers but I guess
there is some problem with non-ARM builds [1]. As Bjorn mentioned in the
thread, we could add #ifdef ARM and then include COMPILE_TEST.

Thanks
Kishon

[1] -> http://www.spinics.net/lists/linux-pci/msg58134.html

Thanks
Kishon
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] PCI: Support for configurable PCI endpoint

[Niklas Cassel]

On 03/27/2017 11:44 AM, Kishon Vijay Abraham I wrote:
> Hi Bjorn,
> 
> Please find the pull request for PCI endpoint support below. I've
> also included all the history here.
> 
> Changes from v4:
> *) add #syscon-cells property and used of_parse_phandle_with_args
>to perform a configuration in syscon module (as suggested by
>Rob Herring)
> *) Remove unnecessary white space.
> 
> Changes from v3:
> *) fixed a typo and adapted to https://lkml.org/lkml/2017/3/13/562.
> 
> Changes from v2:
> *) changed the configfs structure as suggested by Christoph Hellwig. With
>this change the framework creates configfs entry for EP function driver
>and EP controller. Previously these entries have to be created by the
>the user. (Haven't changed the epc core or epf core except for invoking
>configfs APIs to create entries for EP function driver and EP controller.
>That's mostly because the EP function device can still be created by
>directly invoking the epf core API without using configfs).
> *) Now the user has to use configfs entry 'start' to start the link.
>This was previously done by the function driver. However in the case of
>multi function EP, the function driver shouldn't start the link.
> 
> Changes from v1:
> *) The preparation patches for adding EP support is removed and is sent
>separately
> *) Added device ID for DRA74x/DRA72x and used it instead of
>using "PCI_ANY_ID"
> *) Added userguide for PCI endpoint test function
> 
> Major Improvements from RFC:
>  *) support multi-function devices (hw supported not virtual)
>  *) Access host side buffers
>  *) Raise MSI interrupts
>  *) Add user space program to use the host side PCI driver
>  *) Adapt all other users of designware to use the new design (only
> compile tested. Since I have only dra7xx boards, the new design
> has only been tested in dra7xx. I'd require the help of others
> to test the platforms they have access to).
> 
> This series has been developed over 4.11-rc1 + [1]
> [1] -> https://lkml.org/lkml/2017/3/13/562
> 
> Let me know if this has to be re-based to some of your branch.
> 
> Thanks
> Kishon
> 
> The following changes since commit 623e87fec8ab7867fb51b3079196bd10718a60ce:
> 
>   PCI: dwc: dra7xx: Push request_irq call to the bottom of probe (2017-03-22 
> 20:35:30 +0530)
> 
> are available in the git repository at:
> 
>   git://git.kernel.org/pub/scm/linux/kernel/git/kishon/pci-endpoint.git 
> tags/pci-endpoint-for-4.12
> 
> for you to fetch changes up to e98bf80074be4654faae42fe0f5a622a776b6fdd:
> 
>   ARM: DRA7: clockdomain: Change the CLKTRCTRL of CM_PCIE_CLKSTCTRL to 
> SW_WKUP (2017-03-27 15:08:22 +0530)
> 
> 

FWIW:
I've tested Kishon's tag pci-endpoint-for-4.12
and PCIe on artpec6 SoC is still working fine.

I also included the DRA7xx PCIe driver in my
kernel so that pcie-designware-ep.c gets built.

My only worry is that the code in pcie-designware-ep.c
is not compile tested if DRA7xx is not selected
(as it is the only driver using PCIE_DW_EP at
the moment).
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Mauro Carvalho Chehab]

Em Thu, 30 Jun 2016 11:25:11 +0200
Markus Heiser  escreveu:

> Am 29.06.2016 um 22:57 schrieb Mauro Carvalho Chehab 
> :
> 
> > Em Wed, 29 Jun 2016 11:52:09 -0600
> > Jonathan Corbet  escreveu:
> >   
> >>> 2. What is the best way to ship these migrations
> >>> 
> >>> or better I asked, what is your recommendation for a
> >>> migration strategy. Jani says, that this better belongs
> >>> to authors, but I have a doubt that we end with the
> >>> migration in the next years, if we wait about every author.
> >>> I think, supporting both infrastructures - the xml and the
> >>> reST - over a long period is not the best option. What is
> >>> your recommendation on this?
> >> 
> >> I think we need to give maintainers the first shot at doing the conversion;
> >> in any case, I don't think we can just force it through without their
> >> cooperation.  And, honestly, while we're still groping around in this
> >> space, I think it's fine if we don't have lots of conversions right away.
> >> The ones that go more slowly will benefit from what we learn with the easy
> >> ones.
> >> 
> >> You could certainly talk to maintainers and see if they would like
> >> assistance with specific books.  Helping Mauro to get his tables done
> >> without going totally nuts would be a great first step, IMO.  
> > 
> > Yeah, going to each maintainer is the best way, as we're the ones
> > that will be bothered if something goes wrong on the documentation
> > (and code) that we maintain.  
> 
> Later if I have more time, may I will pick up those books who need
> only small correction e.g. "debugobjects" .. but when the time comes
> I will contact the maintainer and Jon' first.
> 
> > In the specific case of media stuff, the first step is to get the
> > flat-tree support and to put the migrated documentation on a separate
> > directory. Unfortunately, I have two sets of topic branches for 4.8,
> > and they both touch at the media docbook (and nothing prevents that
> > similar things to happen on 4.9 or any other version).
> > 
> > So, my plan is to keep both reST and docbook on Kernel for a while,
> > until be sure that everything is properly migrated.
> > 
> > Yet, I'm not sure if we should keep the migration scripts at the
> > Kernel tree, or if the best is to keep them in separate, as I
> > intend to not take more than one Kernel version to finish the
> > conversion. At least for media docbook, this will be a one-time
> > conversion.  
> 
> If I understood you right: the scripts, which are made the
> migration itself should not committed to the kernel tree (IMHO).
> Currently the scripts are available at my POC, may I could
> separate them into a python package which could be installed
> via python's pip. So if one wants to use this procedure of
> migration, he could install the package locally, make the
> conversion and after finishing, he uninstall the package. 

Sounds like a plan.

> 
> > So, I really appreciate if you could send me the patches with
> > the converted media documentation that you did. I'll merge it
> > after Jonathan applies the flat-tree patch on his tree, and
> > start reviewing and fixing the documentation over the main
> > branch.  
> 
> Yes, first flat-table to Jon', then the converted media book
> to Mauro, after this the man-page builder to Jon ... as long
> as there are no man-pages in media it should be smoothly.

OK.

> But I need a bit time, hopefully, end next week you got the
> media patch from me.

Ok, but please notice that the end of the next week is probably too late
for 4.8. Next week, we'll be on -rc6, and most maintainers freeze their
trees during -rc7, except for bug fixes. Ok, as this is just documentation
and should not cause regressions, I may open an exception, if it won't
cause any troubles to Jon.

> 
> -- Markus --
> 
> > I'll need to track the topic branch changes at the
> > Docbook, to apply them again at the master tree, once they
> > gets merged during the 4.8 window. Thankfully, there aren't
> > complex elements on such changes (as far as I remember).
> > 
> > If everything goes right, by -rc2 or -rc3 I'll likely be dropping
> > the DocBook/media from my tree.
> > 
> > --  
> > Thanks,
> > Mauro  
> 


-- 
Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Markus Heiser]

Am 29.06.2016 um 22:57 schrieb Mauro Carvalho Chehab :

> Em Wed, 29 Jun 2016 11:52:09 -0600
> Jonathan Corbet  escreveu:
> 
>>> 2. What is the best way to ship these migrations
>>> 
>>> or better I asked, what is your recommendation for a
>>> migration strategy. Jani says, that this better belongs
>>> to authors, but I have a doubt that we end with the
>>> migration in the next years, if we wait about every author.
>>> I think, supporting both infrastructures - the xml and the
>>> reST - over a long period is not the best option. What is
>>> your recommendation on this?  
>> 
>> I think we need to give maintainers the first shot at doing the conversion;
>> in any case, I don't think we can just force it through without their
>> cooperation.  And, honestly, while we're still groping around in this
>> space, I think it's fine if we don't have lots of conversions right away.
>> The ones that go more slowly will benefit from what we learn with the easy
>> ones.
>> 
>> You could certainly talk to maintainers and see if they would like
>> assistance with specific books.  Helping Mauro to get his tables done
>> without going totally nuts would be a great first step, IMO.
> 
> Yeah, going to each maintainer is the best way, as we're the ones
> that will be bothered if something goes wrong on the documentation
> (and code) that we maintain.

Later if I have more time, may I will pick up those books who need
only small correction e.g. "debugobjects" .. but when the time comes
I will contact the maintainer and Jon' first.

> In the specific case of media stuff, the first step is to get the
> flat-tree support and to put the migrated documentation on a separate
> directory. Unfortunately, I have two sets of topic branches for 4.8,
> and they both touch at the media docbook (and nothing prevents that
> similar things to happen on 4.9 or any other version).
> 
> So, my plan is to keep both reST and docbook on Kernel for a while,
> until be sure that everything is properly migrated.
> 
> Yet, I'm not sure if we should keep the migration scripts at the
> Kernel tree, or if the best is to keep them in separate, as I
> intend to not take more than one Kernel version to finish the
> conversion. At least for media docbook, this will be a one-time
> conversion.

If I understood you right: the scripts, which are made the
migration itself should not committed to the kernel tree (IMHO).
Currently the scripts are available at my POC, may I could
separate them into a python package which could be installed
via python's pip. So if one wants to use this procedure of
migration, he could install the package locally, make the
conversion and after finishing, he uninstall the package. 

> So, I really appreciate if you could send me the patches with
> the converted media documentation that you did. I'll merge it
> after Jonathan applies the flat-tree patch on his tree, and
> start reviewing and fixing the documentation over the main
> branch.

Yes, first flat-table to Jon', then the converted media book
to Mauro, after this the man-page builder to Jon ... as long
as there are no man-pages in media it should be smoothly.

But I need a bit time, hopefully, end next week you got the
media patch from me.

-- Markus --

> I'll need to track the topic branch changes at the
> Docbook, to apply them again at the master tree, once they
> gets merged during the 4.8 window. Thankfully, there aren't
> complex elements on such changes (as far as I remember).
> 
> If everything goes right, by -rc2 or -rc3 I'll likely be dropping
> the DocBook/media from my tree.
> 
> --  
> Thanks,
> Mauro

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Markus Heiser]

Hi Jonathan,

Am 29.06.2016 um 19:52 schrieb Jonathan Corbet :

> On Wed, 29 Jun 2016 19:35:46 +0200
> Markus Heiser  wrote:
> 
>>> I would love it if you would take the flat-table and man-page work,
>>> separate them out, and make them work with the *existing* Sphinx-based
>>> scheme.  If you can do it soon, we can maybe get it into 4.8.  Can you
>>> focus on that for now, please?  
>> 
>> Yes, I will send you flat-table request in the next days.
> 
> I'm glad to hear that.  One request: please post it as a patch, rather than
> as a pull request; that makes it easier for everybody to review it.
> 
>>> As for the rest, what we have now is certainly far from perfect; we're
>>> figuring a lot of this out as we go.  Incremental improvements are
>>> welcome, and each will be evaluated independently.  Please help us to
>>> make the kernel's documentation better that way.  
>> 
>> I'am willing to do so, but I need some help / suggestions:
>> 
>> 1. I have this extensions in the scripts/site-python/linuxdoc.
>>   What do you recommend, how could I split this up in a patch
>>   series which is more evaluated.
>> 
>> .. I wrote to Jani, that my approach was chaotic in the past
>> and I'am sorry for this. But now I'am sitting in front of this
>> bulk of source and I'am bit helpless how to split ... I will
>> try to make it more elaborate, but it will be helpfull if 
>> you point me the right direction ... 
> 
> Try to break it down as much as possible so that each patch represents a
> single logical change.  Each bit that you can break out reduces the problem
> space a bit, and often helps with the rest.  If possible, I'd like to
> suggest starting with the man-page generation, since that's a hole in the
> current system.  I'll have to fill it if you don't :)

Give me a bit time, I will do it. At first flat-table, then man-page.


> Please note that I'd really like to see this stuff done without big changes
> like the wholesale replacement of kernel-doc with a version in a different
> language.  Someday we might want to make a change like that, but one step
> at a time.

mmh, OK ... it will be "the long run" for me ... I will take it (again). The 
replacement makes many things much easier and has this big features; to parse 
only once (not on every kernel-doc directive / one error log, not n times same 
error messages) and a rich interface to control the reST output fine grained (
and Snippets, and a NullTranslator as a lint for free and .. and) ... it is a
good working base ... no need for breadcrumbs or other tricky workarounds ... 

OK, I will start improving the perl script insofar it is needed (the reST out
has to be more structural, you will see it / if it comes to the man-page 
builder)
... may be later I could persuade you, that it is a "dead end street" ... if the
language python is the problem, I could maintain these modules (15 years 
practice).

Regards 

  --Markus--


>> 2. What is the best way to ship these migrations
>> 
>> or better I asked, what is your recommendation for a
>> migration strategy. Jani says, that this better belongs
>> to authors, but I have a doubt that we end with the
>> migration in the next years, if we wait about every author.
>> I think, supporting both infrastructures - the xml and the
>> reST - over a long period is not the best option. What is
>> your recommendation on this?
> 
> I think we need to give maintainers the first shot at doing the conversion;
> in any case, I don't think we can just force it through without their
> cooperation.  And, honestly, while we're still groping around in this
> space, I think it's fine if we don't have lots of conversions right away.
> The ones that go more slowly will benefit from what we learn with the easy
> ones.
> 
> You could certainly talk to maintainers and see if they would like
> assistance with specific books.  Helping Mauro to get his tables done
> without going totally nuts would be a great first step, IMO.
> 
> That said, if you're wanting to convert documents, there is a set of older
> ones in the docbook directory that have no current maintainer and will
> never move over on their own.  kernel-hacking.tmpl is an obvious example.
> The problem with these, of course, is that they are *way* out of date in
> general, and really need attention beyond just a format conversion.  I
> won't say one has to happen before the other, but I am unsure that we will
> really benefit from convert-and-forget-again efforts.
> 
> Thanks,
> 
> jon

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Mauro Carvalho Chehab]

Em Wed, 29 Jun 2016 11:52:09 -0600
Jonathan Corbet  escreveu:

> > 2. What is the best way to ship these migrations
> > 
> > or better I asked, what is your recommendation for a
> > migration strategy. Jani says, that this better belongs
> > to authors, but I have a doubt that we end with the
> > migration in the next years, if we wait about every author.
> > I think, supporting both infrastructures - the xml and the
> > reST - over a long period is not the best option. What is
> > your recommendation on this?  
> 
> I think we need to give maintainers the first shot at doing the conversion;
> in any case, I don't think we can just force it through without their
> cooperation.  And, honestly, while we're still groping around in this
> space, I think it's fine if we don't have lots of conversions right away.
> The ones that go more slowly will benefit from what we learn with the easy
> ones.
> 
> You could certainly talk to maintainers and see if they would like
> assistance with specific books.  Helping Mauro to get his tables done
> without going totally nuts would be a great first step, IMO.

Yeah, going to each maintainer is the best way, as we're the ones
that will be bothered if something goes wrong on the documentation
(and code) that we maintain.

In the specific case of media stuff, the first step is to get the
flat-tree support and to put the migrated documentation on a separate
directory. Unfortunately, I have two sets of topic branches for 4.8,
and they both touch at the media docbook (and nothing prevents that
similar things to happen on 4.9 or any other version).

So, my plan is to keep both reST and docbook on Kernel for a while,
until be sure that everything is properly migrated.

Yet, I'm not sure if we should keep the migration scripts at the
Kernel tree, or if the best is to keep them in separate, as I
intend to not take more than one Kernel version to finish the
conversion. At least for media docbook, this will be a one-time
conversion.

So, I really appreciate if you could send me the patches with
the converted media documentation that you did. I'll merge it
after Jonathan applies the flat-tree patch on his tree, and
start reviewing and fixing the documentation over the main
branch. I'll need to track the topic branch changes at the
Docbook, to apply them again at the master tree, once they
gets merged during the 4.8 window. Thankfully, there aren't
complex elements on such changes (as far as I remember).

If everything goes right, by -rc2 or -rc3 I'll likely be dropping
the DocBook/media from my tree.

--  
Thanks,
Mauro
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Jonathan Corbet]

On Wed, 29 Jun 2016 19:35:46 +0200
Markus Heiser  wrote:

> > I would love it if you would take the flat-table and man-page work,
> > separate them out, and make them work with the *existing* Sphinx-based
> > scheme.  If you can do it soon, we can maybe get it into 4.8.  Can you
> > focus on that for now, please?  
> 
> Yes, I will send you flat-table request in the next days.

I'm glad to hear that.  One request: please post it as a patch, rather than
as a pull request; that makes it easier for everybody to review it.

> > As for the rest, what we have now is certainly far from perfect; we're
> > figuring a lot of this out as we go.  Incremental improvements are
> > welcome, and each will be evaluated independently.  Please help us to
> > make the kernel's documentation better that way.  
> 
> I'am willing to do so, but I need some help / suggestions:
> 
> 1. I have this extensions in the scripts/site-python/linuxdoc.
>What do you recommend, how could I split this up in a patch
>series which is more evaluated.
> 
> .. I wrote to Jani, that my approach was chaotic in the past
> and I'am sorry for this. But now I'am sitting in front of this
> bulk of source and I'am bit helpless how to split ... I will
> try to make it more elaborate, but it will be helpfull if 
> you point me the right direction ... 

Try to break it down as much as possible so that each patch represents a
single logical change.  Each bit that you can break out reduces the problem
space a bit, and often helps with the rest.  If possible, I'd like to
suggest starting with the man-page generation, since that's a hole in the
current system.  I'll have to fill it if you don't :)

Please note that I'd really like to see this stuff done without big changes
like the wholesale replacement of kernel-doc with a version in a different
language.  Someday we might want to make a change like that, but one step
at a time.

> 2. What is the best way to ship these migrations
> 
> or better I asked, what is your recommendation for a
> migration strategy. Jani says, that this better belongs
> to authors, but I have a doubt that we end with the
> migration in the next years, if we wait about every author.
> I think, supporting both infrastructures - the xml and the
> reST - over a long period is not the best option. What is
> your recommendation on this?

I think we need to give maintainers the first shot at doing the conversion;
in any case, I don't think we can just force it through without their
cooperation.  And, honestly, while we're still groping around in this
space, I think it's fine if we don't have lots of conversions right away.
The ones that go more slowly will benefit from what we learn with the easy
ones.

You could certainly talk to maintainers and see if they would like
assistance with specific books.  Helping Mauro to get his tables done
without going totally nuts would be a great first step, IMO.

That said, if you're wanting to convert documents, there is a set of older
ones in the docbook directory that have no current maintainer and will
never move over on their own.  kernel-hacking.tmpl is an obvious example.
The problem with these, of course, is that they are *way* out of date in
general, and really need attention beyond just a format conversion.  I
won't say one has to happen before the other, but I am unsure that we will
really benefit from convert-and-forget-again efforts.

Thanks,

jon
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Markus Heiser]

Hi Jonathan,

Am 29.06.2016 um 18:24 schrieb Jonathan Corbet :

> Hi, Markus,
> 
> I was glad to hear from you, but I have to agree with Jani: this is not
> how things are done.  Consider this one line:
> 
>> 706 files changed, 123369 insertions(+), 752 deletions(-)
> 
> Something like that will be a huge red flag to any kernel maintainer!
> 
> In the kernel community, we have spent the last 25 years figuring out a
> development model that is based on gradual, incremental changes, each of
> which can be reviewed on its own merits.  It does *not* encompass
> wholesale replacements of existing code — even in situations where that
> code was not just merged after a year of discussions, negotiations, and
> false starts.

Yes, my mistake, as I wrote to Jani.

> I simply cannot accept this pull request.
> 
> Markus, we all very much want your help in this work.  You have expertise
> and energy that could really push the documentation effort forward.  But
> it needs to be done the way kernel developers do it: cooperatively,
> incrementally, and always mindful of the entire community's needs.
> 
> I would love it if you would take the flat-table and man-page work,
> separate them out, and make them work with the *existing* Sphinx-based
> scheme.  If you can do it soon, we can maybe get it into 4.8.  Can you
> focus on that for now, please?

Yes, I will send you flat-table request in the next days.

> As for the rest, what we have now is certainly far from perfect; we're
> figuring a lot of this out as we go.  Incremental improvements are
> welcome, and each will be evaluated independently.  Please help us to
> make the kernel's documentation better that way.

I'am willing to do so, but I need some help / suggestions:

1. I have this extensions in the scripts/site-python/linuxdoc.
   What do you recommend, how could I split this up in a patch
   series which is more evaluated.

.. I wrote to Jani, that my approach was chaotic in the past
and I'am sorry for this. But now I'am sitting in front of this
bulk of source and I'am bit helpless how to split ... I will
try to make it more elaborate, but it will be helpfull if 
you point me the right direction ... 

2. What is the best way to ship these migrations

or better I asked, what is your recommendation for a
migration strategy. Jani says, that this better belongs
to authors, but I have a doubt that we end with the
migration in the next years, if we wait about every author.
I think, supporting both infrastructures - the xml and the
reST - over a long period is not the best option. What is
your recommendation on this? 

Regards

  -- Markus --
> 
> Thanks,
> 
> jon

--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Markus Heiser]

Am 29.06.2016 um 15:15 schrieb Jani Nikula :

> On Wed, 29 Jun 2016, Markus Heiser  wrote:
>> Am 28.06.2016 um 21:05 schrieb Jani Nikula :
>>> Perhaps you misunderstood, I don't know. When we ask you to rebase your
>>> work on something, in this case docs-next, it generally means, accept
>>> what is there, and iteratively build and extend upon it, *not* rip out
>>> and rewrite everything from scratch.
>>> 
>>> Several people have spent a non-insignificant amount of time to review
>>> and polish what is in docs-next currently. We've converted gpu
>>> documentation on top, in drm-next, and set up autobuilders for it. We've
>>> ironed out python2 vs python3 issues. We've fixed kernel-doc comments
>>> here and there. Written documentation for the whole thing. Generally
>>> tested the stuff in various environments. Etc, etc. That is the baseline
>>> now, and it should be improved on iteratively, not destructively. That's
>>> just sane engineering.
> 
> Please just read the above again, and try to let it sink in.

I know what you mean, read on about this "unlucky situation" ..

> 
>>> On the actual content (and really, this is orthogonal to the above),
>>> I've repeatedly told you that I disagree with your approach to having
>>> several configuration files, having the distinction between "books" and
>>> other files, rewriting kernel-doc in python, having both rst and
>>> "vintage" kernel-doc comments, converting all the docbook files in one
>>> big lump. I won't repeat my rationale here, I've said it all before, but
>>> sadly I don't see any of that addressed.
> 
>> Take it as what it is: 
>> 
>>  a complete replacement of the XML toolchain.
>> 
>> IMHO, most of what you mentioned are assumptions, so my first 
>> question is:
>> 
>>  have you pulled and tested?
> 
> I have looked at the patches and read the commit messages. If your work
> was reasonably based on docs-next, iteratively improving on what is
> there, we could have a sensible discussion on the relative merits of
> each commit and change. Now, it all depends on being a full rewrite. It
> depends on throwing out everything we've done so far. There isn't a
> single commit that's a change or an improvement to existing code.
> 
> I'm obviously biased because I've done the bulk of the Sphinx work in
> docs-next. Your pull request feels like a complicated way to tell me you
> think it's all crap. I'll try to let that slide.
> 
>> Since your solution is not a full replacement (no man-page builder) and 
>> does not produce structural markup  
>> 
>> IMHO pull my solution, if you have any remarks, let me know. E.g if
>> you think it is better to use ":no-header:" as default on DOC: 
>> sections, I implement it and test it against the complete docs.
> 
> You have plenty of good stuff in there. The annoying thing is that you
> present it in a take-it-all-or-leave-it-all kind of way.

Yes, this was my mistake I'am sorry about.

I started 4 or 5 month ago with the migration in my POC [1] I posted
it on the ML, to show those who had doubt in reST & sphinx that it is
the right decision to switch from DocBook to reST. But the only one how
gives feedback about this was Mauro. We discussed some points and he
explained me some of the developer's requirements, this was productive
and the flat-table is one of the results.

My mistake: after I finished the migration, I continued to
develop the sphinx extensions in the POC and not on the docs-next.
Then I switched to base on a 4.7rc tag... yes, complete chaotic 
Great mistake of mine not to base on the docs-next from the beginning.
... This is my first work in the kernel community and I was not familiar
with the organizational structures. I'am very sorry about and I will
improve myself on this.

In the further course you made huge steps forward and I thought
it is the best to let you work to get productive on that what you
have. While you needed to get productive, I needed more time,
it was "unlucky situation" ... 

side node: It is not really a good alternative, but the history is
present in the POC [1]. E.g. here is one patch from you I ported
to the python kernel-doc module: 

 
https://github.com/return42/sphkerneldoc/commit/9ac8fc9023400d26a6a0b6e7f741e1bf788d2326

Yes, I read through all your patches, made tests with your perl
script and compared it with the python one ... I got huge
benefit from your patches -- the situation and my strategy
was unlucky, but your investigation is not lost! -- and
to fix kernel-doc comments is always good.

> For example, it would be trivial to add the flat table directive to the
> existing configuration file, but instead you opt to rewrite the entire
> file.

The config file "replacement" is because, that my configuration
with sub-project is different to the config file from the sphinx 
template. I also added more comments, dropped settings who are 
not relevant and I moved settings like project, 

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Jonathan Corbet]

Hi, Markus,

I was glad to hear from you, but I have to agree with Jani: this is not
how things are done.  Consider this one line:

> 706 files changed, 123369 insertions(+), 752 deletions(-)

Something like that will be a huge red flag to any kernel maintainer!

In the kernel community, we have spent the last 25 years figuring out a
development model that is based on gradual, incremental changes, each of
which can be reviewed on its own merits.  It does *not* encompass
wholesale replacements of existing code — even in situations where that
code was not just merged after a year of discussions, negotiations, and
false starts.

I simply cannot accept this pull request.

Markus, we all very much want your help in this work.  You have expertise
and energy that could really push the documentation effort forward.  But
it needs to be done the way kernel developers do it: cooperatively,
incrementally, and always mindful of the entire community's needs.

I would love it if you would take the flat-table and man-page work,
separate them out, and make them work with the *existing* Sphinx-based
scheme.  If you can do it soon, we can maybe get it into 4.8.  Can you
focus on that for now, please?

As for the rest, what we have now is certainly far from perfect; we're
figuring a lot of this out as we go.  Incremental improvements are
welcome, and each will be evaluated independently.  Please help us to
make the kernel's documentation better that way.

Thanks,

jon
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Jani Nikula]

On Wed, 29 Jun 2016, Markus Heiser  wrote:
> Am 28.06.2016 um 21:05 schrieb Jani Nikula :
>> Perhaps you misunderstood, I don't know. When we ask you to rebase your
>> work on something, in this case docs-next, it generally means, accept
>> what is there, and iteratively build and extend upon it, *not* rip out
>> and rewrite everything from scratch.
>> 
>> Several people have spent a non-insignificant amount of time to review
>> and polish what is in docs-next currently. We've converted gpu
>> documentation on top, in drm-next, and set up autobuilders for it. We've
>> ironed out python2 vs python3 issues. We've fixed kernel-doc comments
>> here and there. Written documentation for the whole thing. Generally
>> tested the stuff in various environments. Etc, etc. That is the baseline
>> now, and it should be improved on iteratively, not destructively. That's
>> just sane engineering.

Please just read the above again, and try to let it sink in.

>> On the actual content (and really, this is orthogonal to the above),
>> I've repeatedly told you that I disagree with your approach to having
>> several configuration files, having the distinction between "books" and
>> other files, rewriting kernel-doc in python, having both rst and
>> "vintage" kernel-doc comments, converting all the docbook files in one
>> big lump. I won't repeat my rationale here, I've said it all before, but
>> sadly I don't see any of that addressed.

> Take it as what it is: 
>
>a complete replacement of the XML toolchain.
>
> IMHO, most of what you mentioned are assumptions, so my first 
> question is:
>
>have you pulled and tested?

I have looked at the patches and read the commit messages. If your work
was reasonably based on docs-next, iteratively improving on what is
there, we could have a sensible discussion on the relative merits of
each commit and change. Now, it all depends on being a full rewrite. It
depends on throwing out everything we've done so far. There isn't a
single commit that's a change or an improvement to existing code.

I'm obviously biased because I've done the bulk of the Sphinx work in
docs-next. Your pull request feels like a complicated way to tell me you
think it's all crap. I'll try to let that slide.

> Since your solution is not a full replacement (no man-page builder) and 
> does not produce structural markup  
>
> IMHO pull my solution, if you have any remarks, let me know. E.g if
> you think it is better to use ":no-header:" as default on DOC: 
> sections, I implement it and test it against the complete docs.

You have plenty of good stuff in there. The annoying thing is that you
present it in a take-it-all-or-leave-it-all kind of way.

For example, it would be trivial to add the flat table directive to the
existing configuration file, but instead you opt to rewrite the entire
file. You could base your kernel-doc directive extension on the existing
one, but instead you rewrite it. And then add it in the configuration
file rewrite. And so on and so on.

I'll need to focus on other things now, and it's a good time to let
others chime in. It would have been nice to see iterative improvements
from you. Perhaps they could have made it to v4.8, the merge window
being just a few weeks away.

BR,
Jani.


-- 
Jani Nikula, Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [GIT PULL] doc: sphinx-4.8 DocBook to reST movement on Jon's docs-next

[Jani Nikula]

On Tue, 28 Jun 2016, Markus Heiser  wrote:
> Hi Jonathan, hi Mauro,
>
> here is my DocBook to reST movement on top of Jon's docs-next branch. It 
> includes:
>
> * kernel-doc parser & directive
> * flat-table directive
> * man page builder 'kernel-doc-man'
> * the kernel-doc-HOWTO
> * guides for starters (reST-nano-HOWTO.rst & template-book)
> * A index-page which bundles the lose Documentation/*.rst files.
> * sub sphinx-projects aka *books* under Documentation/*/conf.py
> * a full DocBook-XML to reST migration (*books* matching
>  Documentation/books_migrated/*/conf.py) described in [2]. The migration is
>  based on the DocBook-XML content at commit 17defc2 from Johnatan's docs-next
>  branch.
>
> I decided to comment out the pdf creation in the Makefile.reST, rst2pdf is to
> fragil see [2] (pdf is WIP).
>
> The output of html and man is included in the common %doc targets, to get
> more information about reST builds use:
>
>   make books-help 
>
> Any comments are welcome.

Perhaps you misunderstood, I don't know. When we ask you to rebase your
work on something, in this case docs-next, it generally means, accept
what is there, and iteratively build and extend upon it, *not* rip out
and rewrite everything from scratch.

Several people have spent a non-insignificant amount of time to review
and polish what is in docs-next currently. We've converted gpu
documentation on top, in drm-next, and set up autobuilders for it. We've
ironed out python2 vs python3 issues. We've fixed kernel-doc comments
here and there. Written documentation for the whole thing. Generally
tested the stuff in various environments. Etc, etc. That is the baseline
now, and it should be improved on iteratively, not destructively. That's
just sane engineering.

On the actual content (and really, this is orthogonal to the above),
I've repeatedly told you that I disagree with your approach to having
several configuration files, having the distinction between "books" and
other files, rewriting kernel-doc in python, having both rst and
"vintage" kernel-doc comments, converting all the docbook files in one
big lump. I won't repeat my rationale here, I've said it all before, but
sadly I don't see any of that addressed.

IMHO the most productive thing you could do right now is to send out
just the "flat table directive" patch. That's not controversial, and
would still have a chance to make it to v4.8.

BR,
Jani.

>
> -- Markus --
>
> [1] http://return42.github.io/sphkerneldoc/articles/dbtools.html
> [2] https://github.com/rst2pdf/rst2pdf/issues/556#issuecomment-228779542
>
>
> The following changes since commit 17defc282fe6e6ac93edbad8873ce89ef86b2490:
>
>  Documentation: add meta-documentation for Sphinx and kernel-doc (2016-06-24 
> 06:55:28 -0600)
>
> are available in the git repository at:
>
>  https://github.com/return42/linux.git docs-next/linux-doc-reST
>
> for you to fetch changes up to 81bd813a599f8582570a735302ab660e20c7f442:
>
>  doc-rst: full DocBook-XML to reST migration (docs-next) (2016-06-28 17:15:38 
> +0200)
>
> 
>
> Markus Heiser (15):
>  python: add scripts/site-packages
>  kernel-doc-HOWTO: add kernel-doc specification
>  doc-rst: add python package linuxdoc
>  doc-rst: kernel-doc parser - inital python implementation
>  doc-rst: kernel-doc directive - initial implementation
>  doc-rst: flat-table directive - initial implementation
>  doc-rst: kernel-doc-man sphinx builder - initial implementation
>  doc-rst: add basic sphinx-build infrastructure
>  doc-rst: add template book "Get started with reST"
>  doc-rst: removed Jani's kernel-documentation.rst and index.rst
>  doc-rst: infrastructure for *loose reST articles*
>  doc-rst: add build-html decription to book-help.
>  doc-rst: kernel-doc parser assert absolute pathname
>  doc-rst: infrastructure for migrated DocBook-XML
>  doc-rst: full DocBook-XML to reST migration (docs-next)
>

-- 
Jani Nikula, Intel Open Source Technology Center
--
To unsubscribe from this list: send the line "unsubscribe linux-doc" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html