Re: [RFC v6 01/62] powerpc: Free up four 64K PTE bits in 4K backed HPTE pages

[Ram Pai]

On Thu, Jul 20, 2017 at 11:21:51AM +0530, Aneesh Kumar K.V wrote:
> 
> .
> 
> > /*
> > @@ -116,8 +104,8 @@ int __hash_page_4K(unsigned long ea, unsigned long 
> > access, unsigned long vsid,
> >  * On hash insert failure we use old pte value and we don't
> >  * want slot information there if we have a insert failure.
> >  */
> > -   old_pte &= ~(H_PAGE_HASHPTE | H_PAGE_F_GIX | H_PAGE_F_SECOND);
> > -   new_pte &= ~(H_PAGE_HASHPTE | H_PAGE_F_GIX | H_PAGE_F_SECOND);
> > +   old_pte &= ~(H_PAGE_HASHPTE);
> > +   new_pte &= ~(H_PAGE_HASHPTE);
> > goto htab_insert_hpte;
> > }
> 
> With the current path order and above hunk we will breaks the bisect I guess. 
> With the above, when
> we convert a 64k hpte to 4khpte, since this is the first patch, we
> should clear that H_PAGE_F_GIX and H_PAGE_F_SECOND. We still use them
> for 64k. I guess you should move this hunk to second patch.

true. it should move to the next patch. Will fix it.
RP

Re: [RFC v6 01/62] powerpc: Free up four 64K PTE bits in 4K backed HPTE pages

[Aneesh Kumar K.V]

.

>   /*
> @@ -116,8 +104,8 @@ int __hash_page_4K(unsigned long ea, unsigned long 
> access, unsigned long vsid,
>* On hash insert failure we use old pte value and we don't
>* want slot information there if we have a insert failure.
>*/
> - old_pte &= ~(H_PAGE_HASHPTE | H_PAGE_F_GIX | H_PAGE_F_SECOND);
> - new_pte &= ~(H_PAGE_HASHPTE | H_PAGE_F_GIX | H_PAGE_F_SECOND);
> + old_pte &= ~(H_PAGE_HASHPTE);
> + new_pte &= ~(H_PAGE_HASHPTE);
>   goto htab_insert_hpte;
>   }

With the current path order and above hunk we will breaks the bisect I guess. 
With the above, when
we convert a 64k hpte to 4khpte, since this is the first patch, we
should clear that H_PAGE_F_GIX and H_PAGE_F_SECOND. We still use them
for 64k. I guess you should move this hunk to second patch.


-aneesh

[RFC v6 01/62] powerpc: Free up four 64K PTE bits in 4K backed HPTE pages

[Ram Pai]

Rearrange 64K PTE bits to  free  up  bits 3, 4, 5  and  6,
in the 4K backed HPTE pages.These bits continue to be used
for 64K backed HPTE pages in this patch, but will be freed
up in the next patch. The  bit  numbers are big-endian  as
defined in the ISA3.0

The patch does the following change to the 4k htpe backed
64K PTE's format.

H_PAGE_BUSY moves from bit 3 to bit 9 (B bit in the figure
below)
V0 which occupied bit 4 is not used anymore.
V1 which occupied bit 5 is not used anymore.
V2 which occupied bit 6 is not used anymore.
V3 which occupied bit 7 is not used anymore.

Before the patch, the 4k backed 64k PTE format was as follows

 0 1 2 3 4  5  6  7  8 9 10...63
 : : : : :  :  :  :  : : ::
 v v v v v  v  v  v  v v vv

,-,-,-,-,--,--,--,--,-,-,-,-,-,--,-,-,-,
|x|x|x|B|V0|V1|V2|V3|x| | |x|x||x|x|x|x| <- primary pte
'_'_'_'_'__'__'__'__'_'_'_'_'_''_'_'_'_'
|S|G|I|X|S |G |I |X |S|G|I|X|..|S|G|I|X| <- secondary pte
'_'_'_'_'__'__'__'__'_'_'_'_'__'_'_'_'_'

After the patch, the 4k backed 64k PTE format is as follows

 0 1 2 3 4  5  6  7  8 9 10...63
 : : : : :  :  :  :  : : ::
 v v v v v  v  v  v  v v vv

,-,-,-,-,--,--,--,--,-,-,-,-,-,--,-,-,-,
|x|x|x| |  |  |  |  |x|B| |x|x||.|.|.|.| <- primary pte
'_'_'_'_'__'__'__'__'_'_'_'_'_''_'_'_'_'
|S|G|I|X|S |G |I |X |S|G|I|X|..|S|G|I|X| <- secondary pte
'_'_'_'_'__'__'__'__'_'_'_'_'__'_'_'_'_'

the four  bits S,G,I,X (one quadruplet per 4k HPTE) that
cache  the  hash-bucket  slot  value, is initialized  to
1,1,1,1 indicating -- an invalid slot.   If  a HPTE gets
cached in a   slot(i.e 7th  slot  of  secondary hash
bucket), it is  released  immediately. In  other  words,
even  though    is   a valid slot  value in the hash
bucket, we consider it invalid and  release the slot and
the HPTE.  This  gives  us  the opportunity to determine
the validity of S,G,I,X  bits  based on its contents and
not on any of the bits V0,V1,V2 or V3 in the primary PTE

When   we  release  aHPTEcached in the  slot
we alsorelease  a  legitimate   slot  in the primary
hash bucket  and  unmap  its  corresponding  HPTE.  This
is  to  ensure   that  we do get a HPTE cached in a slot
of the primary hash bucket, the next time we retry.

Though  treating    slot  as  invalid,  reduces  the
number of  available  slots  in the hash bucket and  may
have  an  effect   on the performance, the probabilty of
hitting a  slot is extermely low.

Compared  to  the  current   scheme, the above described
scheme  reduces  the  number of false hash table updates
significantly   andhas  the   added   advantage   of
releasing  four  valuable  PTE bits for other purpose.

NOTE:even though bits 3, 4, 5, 6, 7 are  not  used  when
the  64K  PTE is backed by 4k HPTE,  they continue to be
used  if  the  PTE  gets  backed  by 64k HPTE.  The next
patch will decouple that aswell, and truely  release the
bits.

This idea was jointly developed by Paul Mackerras,
Aneesh, Michael Ellermen and myself.

4K PTE format remains unchanged currently.

The patch does the following code changes
a) PTE flags are split between 64k and 4k  header files.
b) __hash_page_4K()  is  reimplemented   to reflect the
   above logic.

Signed-off-by: Ram Pai 
---
 arch/powerpc/include/asm/book3s/64/hash-4k.h  |2 +
 arch/powerpc/include/asm/book3s/64/hash-64k.h |8 +--
 arch/powerpc/include/asm/book3s/64/hash.h |1 -
 arch/powerpc/mm/hash64_64k.c  |   78 -
 arch/powerpc/mm/hash_utils_64.c   |4 +-
 5 files changed, 57 insertions(+), 36 deletions(-)

diff --git a/arch/powerpc/include/asm/book3s/64/hash-4k.h 
b/arch/powerpc/include/asm/book3s/64/hash-4k.h
index 0c4e470..f959c00 100644
--- a/arch/powerpc/include/asm/book3s/64/hash-4k.h
+++ b/arch/powerpc/include/asm/book3s/64/hash-4k.h
@@ -16,6 +16,8 @@
 #define H_PUD_TABLE_SIZE   (sizeof(pud_t) << H_PUD_INDEX_SIZE)
 #define H_PGD_TABLE_SIZE   (sizeof(pgd_t) << H_PGD_INDEX_SIZE)
 
+#define H_PAGE_BUSY_RPAGE_RSV1 /* software: PTE & hash are busy */
+
 /* PTE flags to conserve for HPTE identification */
 #define _PAGE_HPTEFLAGS (H_PAGE_BUSY | H_PAGE_HASHPTE | \
 H_PAGE_F_SECOND | H_PAGE_F_GIX)
diff --git a/arch/powerpc/include/asm/book3s/64/hash-64k.h 
b/arch/powerpc/include/asm/book3s/64/hash-64k.h
index 9732837..62e580c 100644
--- a/arch/powerpc/include/asm/book3s/64/hash-64k.h
+++ b/arch/powerpc/include/asm/book3s/64/hash-64k.h
@@ -12,18 +12,14 @@
  */
 #define H_PAGE_COMBO   _RPAGE_RPN0 /* this is a combo 4k page */
 #define H_PAGE_4K_PFN  _RPAGE_RPN1 /* PFN is for a single 4k page */
+#define