Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Bobby Eshleman]

On Sat, Jan 25, 2020 at 05:11:18PM +, Andrew Cooper wrote:
> On 25/01/2020 03:26, Bobby Eshleman wrote:
> > On Fri, Jan 24, 2020 at 01:41:38PM +, Andrew Cooper wrote:
> >> Other areas where you can reduce the minimum build would be to put some
> >> defaults into the defconfig, such as disabling grant tables and mem
> >> access.  There are almost certainly others which will help, so have a
> >> search around menuconfig.
> > Taking note of these, I can definitely do that.
> 
> To fore-warn you, there probably will be more things like this which
> become apparent on further review.
> 
> While I don't expect you to do all the cleanup (some will almost
> certainly be quicker for others to do), I would like to take the
> opportunity to do the obvious bits of cleanup, given the rare
> opportunity of seeing the minimum viable set of things to make a new
> arch compile.

This sounds very reasonable.

> This looks to be the same as what we've chosen to do in x86, give or
> take a fencepost.
> 
> I'd recommend using L{3..0} for similarity with the spec (you surely
> don't want an off-by-one difference between code in Xen and the spec). 
> Whatever you end up choosing, put a description at the top of mm.h (or
> wherever appropriate) which briefly introduces the terminology in Xen,
> and cross references any differences with the spec.

Roger that.

Thanks again for the feedback.

-Bobby

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Andrew Cooper]

On 25/01/2020 03:26, Bobby Eshleman wrote:
> On Fri, Jan 24, 2020 at 01:41:38PM +, Andrew Cooper wrote:
>> On 22/01/2020 01:58, Bobby Eshleman wrote:
>>> Currently, this patchset really only sets up virtual memory for Xen and
>>> initializes UART to enable print output.  None of RISC-V's
>>> virtualization support has been implemented yet, although that is the
>>> next road to start going down.  Many functions only contain dummy
>>> implementations.  Many shortcuts have been taken and TODO's have been
>>> left accordingly.  It is very, very rough.  Be forewarned: you are quite
>>> likely to see some ungainly code here (despite my efforts to clean it up
>>> before sending this patchset out).  My intent with this RFC is to align
>>> early and gauge interest, as opposed to presenting a totally complete
>>> patchset.
>> I've taken a very quick look over the series.
>>
>> Normally, we require that all patches be bisectable, and this series is
>> not because of the Makefile changes in patch 3 specifying object files
>> which are introduced in the following 20 patches.  Of course,
>> introducing a brand new architecture is a special case, but the
>> suggested plan of getting a "minimal build" together will help keep the
>> second phase of "making it boot".
> That totally makes sense.

I see I trailed off in the middle of that final sentence, but it seems I
did get my point across.

>> Within that first phase however, there are some obvious tweaks we should
>> make to common code.  For one, the debugger_trap() infrastructure really
>> is x86-specific (and I haven't seen it used in a decade) so should have
>> its ARM stubs dropped so as not to be a burden on other architectures.
>>
>> There are other aspects (the atomic_t infrastructure) where x86 and ARM
>> already have basically identical copies of the header file, and you've
>> taken a 3rd copy.
>>
>> Other areas where you can reduce the minimum build would be to put some
>> defaults into the defconfig, such as disabling grant tables and mem
>> access.  There are almost certainly others which will help, so have a
>> search around menuconfig.
> Taking note of these, I can definitely do that.

To fore-warn you, there probably will be more things like this which
become apparent on further review.

While I don't expect you to do all the cleanup (some will almost
certainly be quicker for others to do), I would like to take the
opportunity to do the obvious bits of cleanup, given the rare
opportunity of seeing the minimum viable set of things to make a new
arch compile.

>> Some ARM-isms you've inherited would be much better if you didn't.  In
>> particular, I *really* hope RISC-V hasn't made the same backwards naming
>> bug in their pagetable levels which results in {second,first,zeroth}_*
>> et al.  In x86, we purposefully chose not to follow Intel's naming, and
>> instead went with L1, L2, L3, and for 64bit L4.
> The RISC-V spec does indeed describe the table level numbers in reverse
> order: L2 points to L1 points to L0 (for 39bit addresses).
>
> For SV48 48bit addresses, it's L3, L2, L1, to L0.
>
> That said, in the spec there is no direct mention of LX for levels, and
> instead these are referred to by the section from the "virtual page
> number" that they are referred by, (ie VPN[3], VPN[2], etc...).
>
> I definitely would not be strongly opposed to this change for a form
> that reads better.

This looks to be the same as what we've chosen to do in x86, give or
take a fencepost.

I'd recommend using L{3..0} for similarity with the spec (you surely
don't want an off-by-one difference between code in Xen and the spec). 
Whatever you end up choosing, put a description at the top of mm.h (or
wherever appropriate) which briefly introduces the terminology in Xen,
and cross references any differences with the spec.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Bobby Eshleman]

On Fri, Jan 24, 2020 at 01:41:38PM +, Andrew Cooper wrote:
> On 22/01/2020 01:58, Bobby Eshleman wrote:
> > Currently, this patchset really only sets up virtual memory for Xen and
> > initializes UART to enable print output.  None of RISC-V's
> > virtualization support has been implemented yet, although that is the
> > next road to start going down.  Many functions only contain dummy
> > implementations.  Many shortcuts have been taken and TODO's have been
> > left accordingly.  It is very, very rough.  Be forewarned: you are quite
> > likely to see some ungainly code here (despite my efforts to clean it up
> > before sending this patchset out).  My intent with this RFC is to align
> > early and gauge interest, as opposed to presenting a totally complete
> > patchset.
> 
> I've taken a very quick look over the series.
> 
> Normally, we require that all patches be bisectable, and this series is
> not because of the Makefile changes in patch 3 specifying object files
> which are introduced in the following 20 patches.  Of course,
> introducing a brand new architecture is a special case, but the
> suggested plan of getting a "minimal build" together will help keep the
> second phase of "making it boot".

That totally makes sense.

> 
> To start with, I'd recommend a head.S with _start: and an infinite loop,
> along with whatever makefile/kconfig infrastructure is required to get a
> build.
> 

Got it, sounds like a plan.

> Within that first phase however, there are some obvious tweaks we should
> make to common code.  For one, the debugger_trap() infrastructure really
> is x86-specific (and I haven't seen it used in a decade) so should have
> its ARM stubs dropped so as not to be a burden on other architectures.
> 
> There are other aspects (the atomic_t infrastructure) where x86 and ARM
> already have basically identical copies of the header file, and you've
> taken a 3rd copy.
> 
> Other areas where you can reduce the minimum build would be to put some
> defaults into the defconfig, such as disabling grant tables and mem
> access.  There are almost certainly others which will help, so have a
> search around menuconfig.

Taking note of these, I can definitely do that.

> 
> Disabling grant tables in particular will work around the fact that the
> ARM snapshot you based your port on was pre-XSA-295, and the cmpxchg
> loop against guest memory in gnttab_clear_flags() is reintroducing a
> previously-fixed security vulnerability.

Duly noted, I'll fix that and look over some of those again and compare them to 
the
newer files, as some are definitely a few version bumps behind.

> 
> Some ARM-isms you've inherited would be much better if you didn't.  In
> particular, I *really* hope RISC-V hasn't made the same backwards naming
> bug in their pagetable levels which results in {second,first,zeroth}_*
> et al.  In x86, we purposefully chose not to follow Intel's naming, and
> instead went with L1, L2, L3, and for 64bit L4.
> 

The RISC-V spec does indeed describe the table level numbers in reverse
order: L2 points to L1 points to L0 (for 39bit addresses).

For SV48 48bit addresses, it's L3, L2, L1, to L0.

That said, in the spec there is no direct mention of LX for levels, and
instead these are referred to by the section from the "virtual page
number" that they are referred by, (ie VPN[3], VPN[2], etc...).

I definitely would not be strongly opposed to this change for a form
that reads better.

> As a couple of general points from our coding style, please avoid
> commented out code, and you are free to omit braces for single statement
> blocks.
> 
> ~Andrew


Thanks again,
-Bobby

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Bobby Eshleman]

On Thu, Jan 23, 2020 at 11:44:02PM +, Lars Kurth wrote:
> If you add your name/email address to 
> https://cryptpad.fr/pad/#/2/pad/edit/D9vGzihPxxAOe6RFPz0sRCf+/ 
>  I will CC 
> you on the next invite
> They are usually the 1st Thursday of each month 
> Past minutes can be found at 
> https://cryptpad.fr/drive/#/2/drive/edit/uZ1UjYxICjse+XlJrXrIwZXN/
> Lars

Done!  Thanks.

-Bobby

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Bobby Eshleman]

On Thu, Jan 23, 2020 at 04:02:56PM +, Andrew Cooper wrote:
> 
> We have https://gitlab.com/xen-project/xen/pipelines which runs a load
> of containerised builds.  How easy is it to set up a containerised
> RISC-V cross-compiler environment?
> 

It's pretty easy.  I personally use a Docker container that has the bins
built from source from both the RISC-V gcc toolchain and QEMU.

> The test step also boots Xen under Qemu software emulation to check that
> we don't have any boot-time screamers.  A reasonable second step might
> be to have start_xen() panic() at the end, and check for that in the
> console log, which would allow for some kind of boot testing before
> getting all the way to "and here is dom0 ready to run".
> 
> All configuration is in .gitlab-ci.yml and automation/.
> 

That sounds ideal and like a perfect place to start.

Thanks!
-Bobby

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Andrew Cooper]

On 22/01/2020 01:58, Bobby Eshleman wrote:
> Currently, this patchset really only sets up virtual memory for Xen and
> initializes UART to enable print output.  None of RISC-V's
> virtualization support has been implemented yet, although that is the
> next road to start going down.  Many functions only contain dummy
> implementations.  Many shortcuts have been taken and TODO's have been
> left accordingly.  It is very, very rough.  Be forewarned: you are quite
> likely to see some ungainly code here (despite my efforts to clean it up
> before sending this patchset out).  My intent with this RFC is to align
> early and gauge interest, as opposed to presenting a totally complete
> patchset.

I've taken a very quick look over the series.

Normally, we require that all patches be bisectable, and this series is
not because of the Makefile changes in patch 3 specifying object files
which are introduced in the following 20 patches.  Of course,
introducing a brand new architecture is a special case, but the
suggested plan of getting a "minimal build" together will help keep the
second phase of "making it boot".

To start with, I'd recommend a head.S with _start: and an infinite loop,
along with whatever makefile/kconfig infrastructure is required to get a
build.

Within that first phase however, there are some obvious tweaks we should
make to common code.  For one, the debugger_trap() infrastructure really
is x86-specific (and I haven't seen it used in a decade) so should have
its ARM stubs dropped so as not to be a burden on other architectures.

There are other aspects (the atomic_t infrastructure) where x86 and ARM
already have basically identical copies of the header file, and you've
taken a 3rd copy.

Other areas where you can reduce the minimum build would be to put some
defaults into the defconfig, such as disabling grant tables and mem
access.  There are almost certainly others which will help, so have a
search around menuconfig.

Disabling grant tables in particular will work around the fact that the
ARM snapshot you based your port on was pre-XSA-295, and the cmpxchg
loop against guest memory in gnttab_clear_flags() is reintroducing a
previously-fixed security vulnerability.

Some ARM-isms you've inherited would be much better if you didn't.  In
particular, I *really* hope RISC-V hasn't made the same backwards naming
bug in their pagetable levels which results in {second,first,zeroth}_*
et al.  In x86, we purposefully chose not to follow Intel's naming, and
instead went with L1, L2, L3, and for 64bit L4.

As a couple of general points from our coding style, please avoid
commented out code, and you are free to omit braces for single statement
blocks.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Lars Kurth]

> On 23 Jan 2020, at 05:31, Bobby Eshleman  wrote:
> 
> On Wed, Jan 22, 2020 at 04:27:39PM +, Lars Kurth wrote:
>> 
>> You should also leverage the developer summit: see 
>> https://events.linuxfoundation.org/xen-summit/program/cfp/ 
>> 
>> CfP closes March 6th. Design sessions can be submitted afterwards
>> 
>> Community calls may also be a good option to deal with specific issues / 
>> questions, e.g. around compile support in the CI, etc.
>> 
>> Lars
>> 
> 
> That's a really good idea.  I'll submit as I do think I can get there if 
> accepted.  Thanks for the tip on
> community calls, I did not realize Xen did those!
> 
> -Bobby

If you add your name/email address to 
https://cryptpad.fr/pad/#/2/pad/edit/D9vGzihPxxAOe6RFPz0sRCf+/ 
 I will CC you 
on the next invite
They are usually the 1st Thursday of each month 
Past minutes can be found at 
https://cryptpad.fr/drive/#/2/drive/edit/uZ1UjYxICjse+XlJrXrIwZXN/
Lars___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Andrew Cooper]

On 23/01/2020 05:19, Bobby Eshleman wrote:
> On Wed, Jan 22, 2020 at 02:57:47PM +, Andrew Cooper wrote:
>> How much time do you have to put towards the port?  Is this something in
>> your free time, or something you are doing as part of work?  Ultimately,
>> we are going to need to increase the level of RISC-V knowledge in the
>> community to maintain things in the future.
>>
> This is something in my free time, and I have about 20 hours a week to
> put into it.

Ok, so not full time, but hopefully enough time to help spread some
knowledge.

>
>> Other than that, very RFC series are entirely fine.  A good first step
>> would be simply to get the build working, and get some kind of
>> cross-compile build in CI, to make sure that we don't clobber the RISC-V
>> build with common or other-arch changes.
>>
> That's something I can look at, if the idea of QEMU in CI is
> not too horrific.

We have https://gitlab.com/xen-project/xen/pipelines which runs a load
of containerised builds.  How easy is it to set up a containerised
RISC-V cross-compiler environment?

The test step also boots Xen under Qemu software emulation to check that
we don't have any boot-time screamers.  A reasonable second step might
be to have start_xen() panic() at the end, and check for that in the
console log, which would allow for some kind of boot testing before
getting all the way to "and here is dom0 ready to run".

All configuration is in .gitlab-ci.yml and automation/.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Alistair Francis]

On Wed, 2020-01-22 at 14:57 +, Andrew Cooper wrote:
> On 22/01/2020 01:58, Bobby Eshleman wrote:
> > Hey everybody,
> > 
> > This is an RFC patchset for the very beginnings of adding RISC-V
> > support
> > to Xen.  This RFC is really just to start a dialogue about
> > supporting
> > RISC-V and align with the Xen project and community before going
> > further.  For that reason, it is very rough and very incomplete. 
> > 
> > My name is Bobby Eshleman, I'm a software engineer at
> > Star Lab / Wind River on the ARM team, mostly having worked in the
> > Linux
> > kernel.  I've also been involved a good amount with Xen on ARM
> > here,
> > mostly dealing with tooling, deployment, and testing.  A lot of
> > this
> > patchset is heavily inspired by the Xen/ARM source code
> > (particularly
> > the early setup up code).
> > 
> > Currently, this patchset really only sets up virtual memory for Xen
> > and
> > initializes UART to enable print output.  None of RISC-V's
> > virtualization support has been implemented yet, although that is
> > the
> > next road to start going down.  Many functions only contain dummy
> > implementations.  Many shortcuts have been taken and TODO's have
> > been
> > left accordingly.  It is very, very rough.  Be forewarned: you are
> > quite
> > likely to see some ungainly code here (despite my efforts to clean
> > it up
> > before sending this patchset out).  My intent with this RFC is to
> > align
> > early and gauge interest, as opposed to presenting a totally
> > complete
> > patchset.
> > 
> > Because the ARM and RISC-V use cases will likely bear resemblance,
> > the
> > RISC-V port should probably respect the design considerations that
> > have
> > been laid out and respected by Xen on ARM for dom0less, safety
> > certification, etc...  My inclination has been to initially target
> > or
> > prioritize dom0less (without excluding dom0full) and use the ARM
> > dom0less implementation as a model to follow.  I'd love feedback on
> > this
> > point and on how the Xen project might envision a RISC-V
> > implementation.
> > 
> > This patchset has _some_ code for future support for 32-bit, but
> > currently my focus is on 64-bit.
> > 
> > Again, this is a very, very rough and totally incomplete
> > patchset.  My
> > goal here is just to gauge community interest, begin discussing
> > what Xen
> > on RISC-V may look like, receive feedback, and see if I'm heading
> > in the
> > right direction.
> > 
> > My big questions are:
> > Does the Xen project have interest in RISC-V?
> 
> There is very large downstream interest in RISC-V.  So a definite
> yes.
> 
> > What can be done to make the RISC-V port as upstreamable as
> > possible?
> > Any major pitfalls?
> > 
> > It would be great to hear all of your feedback.
> 
> Both RISC-V and Power9 are frequently requested things, and both
> suffer
> from the fact that, while we as a community would like them, the
> upstream intersection of "people who know Xen" and "people who know
> enough arch $X to do an initial port" is 0.
> 
> This series clearly demonstrates a change in the status quo, and I
> think
> a lot of people will be happy.
> 
> To get RISC-V to being fully supported, we will ultimately need to
> get
> hardware into the CI system, and an easy way for developers to test
> changes.  Do you have any thoughts on production RISC-V hardware
> (ideally server form factor) for the CI system, and/or dev boards
> which
> might be available fairly cheaply?

AFAIK this is awhile off. The Hypervisor spec is still in a draft state
(expecting to be frozen this year) so right now QEMU is the only
option.

Alistair

> 
> How much time do you have to put towards the port?  Is this something
> in
> your free time, or something you are doing as part of work? 
> Ultimately,
> we are going to need to increase the level of RISC-V knowledge in the
> community to maintain things in the future.
> 
> Other than that, very RFC series are entirely fine.  A good first
> step
> would be simply to get the build working, and get some kind of
> cross-compile build in CI, to make sure that we don't clobber the
> RISC-V
> build with common or other-arch changes.
> 
> I hope this helps.
> 
> ~Andrew
___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Bobby Eshleman]

On Wed, Jan 22, 2020 at 04:27:39PM +, Lars Kurth wrote:
> 
> You should also leverage the developer summit: see 
> https://events.linuxfoundation.org/xen-summit/program/cfp/ 
> 
> CfP closes March 6th. Design sessions can be submitted afterwards
> 
> Community calls may also be a good option to deal with specific issues / 
> questions, e.g. around compile support in the CI, etc.
> 
> Lars
>

That's a really good idea.  I'll submit as I do think I can get there if 
accepted.  Thanks for the tip on
community calls, I did not realize Xen did those!

-Bobby

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Bobby Eshleman]

On Wed, Jan 22, 2020 at 02:57:47PM +, Andrew Cooper wrote:
> How much time do you have to put towards the port?  Is this something in
> your free time, or something you are doing as part of work?  Ultimately,
> we are going to need to increase the level of RISC-V knowledge in the
> community to maintain things in the future.
> 

This is something in my free time, and I have about 20 hours a week to
put into it.

> Other than that, very RFC series are entirely fine.  A good first step
> would be simply to get the build working, and get some kind of
> cross-compile build in CI, to make sure that we don't clobber the RISC-V
> build with common or other-arch changes.
> 

That's something I can look at, if the idea of QEMU in CI is
not too horrific.


> I hope this helps.
> 
> ~Andrew


Definitely helps, thanks!

-Bobby

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Bobby Eshleman]

On Wed, Jan 22, 2020 at 01:05:11PM -0800, Stefano Stabellini wrote:
> On Wed, 22 Jan 2020, Andrew Cooper wrote:
> > > My big questions are:
> > >   Does the Xen project have interest in RISC-V?
> > 
> > There is very large downstream interest in RISC-V.  So a definite yes.
> 
> Definite Yes from me too
> 

Both great to hear!

> 
> > >   What can be done to make the RISC-V port as upstreamable as
> > >   possible?
> > >   Any major pitfalls?
> > >
> > > It would be great to hear all of your feedback.
> > 
> > Both RISC-V and Power9 are frequently requested things, and both suffer
> > from the fact that, while we as a community would like them, the
> > upstream intersection of "people who know Xen" and "people who know
> > enough arch $X to do an initial port" is 0.
> > 
> > This series clearly demonstrates a change in the status quo, and I think
> > a lot of people will be happy.
> > 
> > To get RISC-V to being fully supported, we will ultimately need to get
> > hardware into the CI system, and an easy way for developers to test
> > changes.  Do you have any thoughts on production RISC-V hardware
> > (ideally server form factor) for the CI system, and/or dev boards which
> > might be available fairly cheaply?
> 
> My understanding is that virtualization development for RISC-V is done
> on QEMU right now (which could still be hooked into the CI system if
> somebody wanted to do the work I think.)

That is correct.  I think the RTL and hardware folks are waiting for the
spec to be finalized before committing to the effort, so everyone is
just developing against QEMU for now.

I can certainly look at hooking in QEMU to the CI at some point soon.  That
is the OSSTest repo, correct?

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Stefano Stabellini]

On Wed, 22 Jan 2020, Andrew Cooper wrote:
> > My big questions are:
> > Does the Xen project have interest in RISC-V?
> 
> There is very large downstream interest in RISC-V.  So a definite yes.

Definite Yes from me too


> > What can be done to make the RISC-V port as upstreamable as
> > possible?
> > Any major pitfalls?
> >
> > It would be great to hear all of your feedback.
> 
> Both RISC-V and Power9 are frequently requested things, and both suffer
> from the fact that, while we as a community would like them, the
> upstream intersection of "people who know Xen" and "people who know
> enough arch $X to do an initial port" is 0.
> 
> This series clearly demonstrates a change in the status quo, and I think
> a lot of people will be happy.
> 
> To get RISC-V to being fully supported, we will ultimately need to get
> hardware into the CI system, and an easy way for developers to test
> changes.  Do you have any thoughts on production RISC-V hardware
> (ideally server form factor) for the CI system, and/or dev boards which
> might be available fairly cheaply?

My understanding is that virtualization development for RISC-V is done
on QEMU right now (which could still be hooked into the CI system if
somebody wanted to do the work I think.)___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Lars Kurth]

> On 22 Jan 2020, at 14:57, Andrew Cooper  wrote:
> 
> On 22/01/2020 01:58, Bobby Eshleman wrote:
>> Hey everybody,
>> 
>> This is an RFC patchset for the very beginnings of adding RISC-V support
>> to Xen.  This RFC is really just to start a dialogue about supporting
>> RISC-V and align with the Xen project and community before going
>> further.  For that reason, it is very rough and very incomplete. 
>> 
>> My name is Bobby Eshleman, I'm a software engineer at
>> Star Lab / Wind River on the ARM team, mostly having worked in the Linux
>> kernel.  I've also been involved a good amount with Xen on ARM here,
>> mostly dealing with tooling, deployment, and testing.  A lot of this
>> patchset is heavily inspired by the Xen/ARM source code (particularly
>> the early setup up code).
>> 
>> Currently, this patchset really only sets up virtual memory for Xen and
>> initializes UART to enable print output.  None of RISC-V's
>> virtualization support has been implemented yet, although that is the
>> next road to start going down.  Many functions only contain dummy
>> implementations.  Many shortcuts have been taken and TODO's have been
>> left accordingly.  It is very, very rough.  Be forewarned: you are quite
>> likely to see some ungainly code here (despite my efforts to clean it up
>> before sending this patchset out).  My intent with this RFC is to align
>> early and gauge interest, as opposed to presenting a totally complete
>> patchset.
>> 
>> Because the ARM and RISC-V use cases will likely bear resemblance, the
>> RISC-V port should probably respect the design considerations that have
>> been laid out and respected by Xen on ARM for dom0less, safety
>> certification, etc...  My inclination has been to initially target or
>> prioritize dom0less (without excluding dom0full) and use the ARM
>> dom0less implementation as a model to follow.  I'd love feedback on this
>> point and on how the Xen project might envision a RISC-V implementation.
>> 
>> This patchset has _some_ code for future support for 32-bit, but
>> currently my focus is on 64-bit.
>> 
>> Again, this is a very, very rough and totally incomplete patchset.  My
>> goal here is just to gauge community interest, begin discussing what Xen
>> on RISC-V may look like, receive feedback, and see if I'm heading in the
>> right direction.
>> 
>> My big questions are:
>>  Does the Xen project have interest in RISC-V?
> 
> There is very large downstream interest in RISC-V.  So a definite yes.
> 
>>  What can be done to make the RISC-V port as upstreamable as
>>  possible?
>>  Any major pitfalls?
>> 
>> It would be great to hear all of your feedback.
> 
> Both RISC-V and Power9 are frequently requested things, and both suffer
> from the fact that, while we as a community would like them, the
> upstream intersection of "people who know Xen" and "people who know
> enough arch $X to do an initial port" is 0.
> 
> This series clearly demonstrates a change in the status quo, and I think
> a lot of people will be happy.
> 
> To get RISC-V to being fully supported, we will ultimately need to get
> hardware into the CI system, and an easy way for developers to test
> changes.  Do you have any thoughts on production RISC-V hardware
> (ideally server form factor) for the CI system, and/or dev boards which
> might be available fairly cheaply?
> 
> How much time do you have to put towards the port?  Is this something in
> your free time, or something you are doing as part of work?  Ultimately,
> we are going to need to increase the level of RISC-V knowledge in the
> community to maintain things in the future.
> 
> Other than that, very RFC series are entirely fine.  A good first step
> would be simply to get the build working, and get some kind of
> cross-compile build in CI, to make sure that we don't clobber the RISC-V
> build with common or other-arch changes.
> 
> I hope this helps.

I totally agree with what Andy says. 

You should also leverage the developer summit: see 
https://events.linuxfoundation.org/xen-summit/program/cfp/ 

CfP closes March 6th. Design sessions can be submitted afterwards

Community calls may also be a good option to deal with specific issues / 
questions, e.g. around compile support in the CI, etc.

Lars




___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

Re: [Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Andrew Cooper]

On 22/01/2020 01:58, Bobby Eshleman wrote:
> Hey everybody,
>
> This is an RFC patchset for the very beginnings of adding RISC-V support
> to Xen.  This RFC is really just to start a dialogue about supporting
> RISC-V and align with the Xen project and community before going
> further.  For that reason, it is very rough and very incomplete. 
>
> My name is Bobby Eshleman, I'm a software engineer at
> Star Lab / Wind River on the ARM team, mostly having worked in the Linux
> kernel.  I've also been involved a good amount with Xen on ARM here,
> mostly dealing with tooling, deployment, and testing.  A lot of this
> patchset is heavily inspired by the Xen/ARM source code (particularly
> the early setup up code).
>
> Currently, this patchset really only sets up virtual memory for Xen and
> initializes UART to enable print output.  None of RISC-V's
> virtualization support has been implemented yet, although that is the
> next road to start going down.  Many functions only contain dummy
> implementations.  Many shortcuts have been taken and TODO's have been
> left accordingly.  It is very, very rough.  Be forewarned: you are quite
> likely to see some ungainly code here (despite my efforts to clean it up
> before sending this patchset out).  My intent with this RFC is to align
> early and gauge interest, as opposed to presenting a totally complete
> patchset.
>
> Because the ARM and RISC-V use cases will likely bear resemblance, the
> RISC-V port should probably respect the design considerations that have
> been laid out and respected by Xen on ARM for dom0less, safety
> certification, etc...  My inclination has been to initially target or
> prioritize dom0less (without excluding dom0full) and use the ARM
> dom0less implementation as a model to follow.  I'd love feedback on this
> point and on how the Xen project might envision a RISC-V implementation.
>
> This patchset has _some_ code for future support for 32-bit, but
> currently my focus is on 64-bit.
>
> Again, this is a very, very rough and totally incomplete patchset.  My
> goal here is just to gauge community interest, begin discussing what Xen
> on RISC-V may look like, receive feedback, and see if I'm heading in the
> right direction.
>
> My big questions are:
>   Does the Xen project have interest in RISC-V?

There is very large downstream interest in RISC-V.  So a definite yes.

>   What can be done to make the RISC-V port as upstreamable as
>   possible?
>   Any major pitfalls?
>
> It would be great to hear all of your feedback.

Both RISC-V and Power9 are frequently requested things, and both suffer
from the fact that, while we as a community would like them, the
upstream intersection of "people who know Xen" and "people who know
enough arch $X to do an initial port" is 0.

This series clearly demonstrates a change in the status quo, and I think
a lot of people will be happy.

To get RISC-V to being fully supported, we will ultimately need to get
hardware into the CI system, and an easy way for developers to test
changes.  Do you have any thoughts on production RISC-V hardware
(ideally server form factor) for the CI system, and/or dev boards which
might be available fairly cheaply?

How much time do you have to put towards the port?  Is this something in
your free time, or something you are doing as part of work?  Ultimately,
we are going to need to increase the level of RISC-V knowledge in the
community to maintain things in the future.

Other than that, very RFC series are entirely fine.  A good first step
would be simply to get the build working, and get some kind of
cross-compile build in CI, to make sure that we don't clobber the RISC-V
build with common or other-arch changes.

I hope this helps.

~Andrew

___
Xen-devel mailing list
Xen-devel@lists.xenproject.org
https://lists.xenproject.org/mailman/listinfo/xen-devel

[Xen-devel] [RFC XEN PATCH 00/23] xen: beginning support for RISC-V

[Bobby Eshleman]

Hey everybody,

This is an RFC patchset for the very beginnings of adding RISC-V support
to Xen.  This RFC is really just to start a dialogue about supporting
RISC-V and align with the Xen project and community before going
further.  For that reason, it is very rough and very incomplete. 

My name is Bobby Eshleman, I'm a software engineer at
Star Lab / Wind River on the ARM team, mostly having worked in the Linux
kernel.  I've also been involved a good amount with Xen on ARM here,
mostly dealing with tooling, deployment, and testing.  A lot of this
patchset is heavily inspired by the Xen/ARM source code (particularly
the early setup up code).

Currently, this patchset really only sets up virtual memory for Xen and
initializes UART to enable print output.  None of RISC-V's
virtualization support has been implemented yet, although that is the
next road to start going down.  Many functions only contain dummy
implementations.  Many shortcuts have been taken and TODO's have been
left accordingly.  It is very, very rough.  Be forewarned: you are quite
likely to see some ungainly code here (despite my efforts to clean it up
before sending this patchset out).  My intent with this RFC is to align
early and gauge interest, as opposed to presenting a totally complete
patchset.

Because the ARM and RISC-V use cases will likely bear resemblance, the
RISC-V port should probably respect the design considerations that have
been laid out and respected by Xen on ARM for dom0less, safety
certification, etc...  My inclination has been to initially target or
prioritize dom0less (without excluding dom0full) and use the ARM
dom0less implementation as a model to follow.  I'd love feedback on this
point and on how the Xen project might envision a RISC-V implementation.

This patchset has _some_ code for future support for 32-bit, but
currently my focus is on 64-bit.

Again, this is a very, very rough and totally incomplete patchset.  My
goal here is just to gauge community interest, begin discussing what Xen
on RISC-V may look like, receive feedback, and see if I'm heading in the
right direction.

My big questions are:
Does the Xen project have interest in RISC-V?
What can be done to make the RISC-V port as upstreamable as
possible?
Any major pitfalls?

It would be great to hear all of your feedback.

Alistair Francis (20):
  HACK: OE Build changes
  HACK: Makefile: Don't build Xen tools
  riscv: makefiles and Kconfig
  riscv: Add riscv to tools/libxc header files
  riscv: Add asm-offsets.c
  riscv: Add delay.c
  riscv: Add domain.c
  riscv: Add domctl.c
  riscv: Add guestcopy.c
  riscv: Add time.c
  riscv: Add smp.c
  riscv: Add shutdown.c
  riscv: Add traps.c
  riscv: Add irq.c
  riscv: Add vm_event.c
  riscv: Add p2m.c
  riscv: Add the lib directory
  riscv: Add smpboot.c
  riscv: Add percpu.c
  riscv: Add sysctl.c

Bobby Eshleman (3):
  riscv: header files
  riscv: early setup code
  riscv: Add iommu.c

 Makefile |  13 +-
 config/StdGNU.mk |  12 +-
 config/riscv64.mk|   7 +
 tools/configure  |  32 +-
 tools/firmware/Makefile  |  12 +-
 tools/libxc/include/xenctrl.h|   7 +
 tools/libxc/xc_core.h|   2 +
 tools/libxc/xc_core_riscv.h  |  57 ++
 xen/Makefile |   2 +-
 xen/Rules.mk |   2 +-
 xen/arch/Kconfig |   1 +
 xen/arch/riscv/Kconfig   |  36 +
 xen/arch/riscv/Makefile  |  62 ++
 xen/arch/riscv/Rules.mk  |  55 ++
 xen/arch/riscv/asm-offsets.c |  38 +
 xen/arch/riscv/configs/riscv32_defconfig |   0
 xen/arch/riscv/configs/riscv64_defconfig |   0
 xen/arch/riscv/delay.c   | 114 +++
 xen/arch/riscv/domain.c  | 273 +++
 xen/arch/riscv/domctl.c  |  53 ++
 xen/arch/riscv/guestcopy.c   | 158 
 xen/arch/riscv/head.S| 180 +
 xen/arch/riscv/irq.c | 107 +++
 xen/arch/riscv/lib/Makefile  |   1 +
 xen/arch/riscv/lib/find_next_bit.c   | 284 +++
 xen/arch/riscv/mm.c  | 925 +++
 xen/arch/riscv/p2m.c | 261 +++
 xen/arch/riscv/percpu.c  |  84 ++
 xen/arch/riscv/platforms/Kconfig |  31 +
 xen/arch/riscv/setup.c   | 122 +++
 xen/arch/riscv/shutdown.c|  24 +
 xen/arch/riscv/smp.c |  41 +
 xen/arch/riscv/smpboot.c | 114 +++
 xen/arch/riscv/sysctl.c  |  31 +
 xen/arch/riscv/time.c|  74 ++
 xen/arch/riscv/traps.c   |  56 ++
 xen/arch/riscv/vm_event.c|  42 +
 xen/arch/riscv/xen.lds.S | 262 +++
 xen/drivers/passthrough/Makefile |   1 +