Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-10-12 Thread Manish Jaggi 



On 10/12/2017 5:14 PM, Julien Grall wrote:



On 12/10/17 12:22, Manish Jaggi wrote:

Hi Julien,

Why do you omit parts of mail where I have asked a question , please 
avoid skiping  that removes the context.


I believe I answered it just after because you asked twice the same 
thing. So may I dropped the context but the answer was there...


For your convenience here the replicated answer.

"Why? The generation of IORT is fairly standalone.

And again, this was suggestion to share in the future and an 
expectation for this series. What I care the most is the generation to 
be fully separated from the rest."


I raised a valid point and it was totally ignored and you asked me to 
explain the rationale on a later point.

So if you choose to ignore my first point, how can I put any point.


Well, maybe you should read the e-mail more carefully because your 
point have been addressed. If they are not, then please say it rather 
than accusing the reviewers on spending not enough time on your series...


[...]

Now if you see both the codes are quite similar and there is 
redundancy in libxl and in xen code for preparing ACPI tables for 
dom0 and domU.
The point I am raising is quite clear, if all other tables like MADT, 
XSDT, RSDP, GTDT etc does not share a common generation code with xen 
what is so special about IORT.
Either we move all generation into a common code or keep redundancy 
for IORT.


I hope I have shown the code and made the point quite clear.
Please provide a technical answer rather than a simple "Why".


Why do you still continue arguing on how this is going to interact 
with libxl when your only work now (as I stated in every single 
e-mail) is for Dom0.


If the generation is generic enough, it will require little code to 
interface. After all, you only need:

- informations (e.g DeviceID, MasterID...)
- buffer for writing the generated IORT

So now it is maybe time for you to suggest an interface we can discuss 
on.

Sure. A quick draft is shared on mailing list. [1]

[1] https://marc.info/?l=xen-devel=150784236208192=2


Cheers,




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-10-12 Thread Julien Grall 



On 12/10/17 12:22, Manish Jaggi wrote:

Hi Julien,

Why do you omit parts of mail where I have asked a question , please 
avoid skiping  that removes the context.


I believe I answered it just after because you asked twice the same 
thing. So may I dropped the context but the answer was there...


For your convenience here the replicated answer.

"Why? The generation of IORT is fairly standalone.

And again, this was suggestion to share in the future and an expectation 
for this series. What I care the most is the generation to be fully 
separated from the rest."


I raised a valid point and it was totally ignored and you asked me to 
explain the rationale on a later point.

So if you choose to ignore my first point, how can I put any point.


Well, maybe you should read the e-mail more carefully because your point 
have been addressed. If they are not, then please say it rather than 
accusing the reviewers on spending not enough time on your series...


[...]

Now if you see both the codes are quite similar and there is redundancy 
in libxl and in xen code for preparing ACPI tables for dom0 and domU.
The point I am raising is quite clear, if all other tables like MADT, 
XSDT, RSDP, GTDT etc does not share a common generation code with xen 
what is so special about IORT.
Either we move all generation into a common code or keep redundancy for 
IORT.


I hope I have shown the code and made the point quite clear.
Please provide a technical answer rather than a simple "Why".


Why do you still continue arguing on how this is going to interact with 
libxl when your only work now (as I stated in every single e-mail) is 
for Dom0.


If the generation is generic enough, it will require little code to 
interface. After all, you only need:

- informations (e.g DeviceID, MasterID...)
- buffer for writing the generated IORT

So now it is maybe time for you to suggest an interface we can discuss on.

Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-10-12 Thread Manish Jaggi 

Hi Julien,

Why do you omit parts of mail where I have asked a question , please 
avoid skiping  that removes the context.
I raised a valid point and it was totally ignored and you asked me to 
explain the rationale on a later point.

So if you choose to ignore my first point, how can I put any point.

This is what I have asked


>>The ACPI tables for DomU are generated by the toolstack today. So I 
don't see why we would change that to support IORT.

>>
>>However, you can have a file shared between the toolstack and Xen and 
contain the generation of IORT.

>>
>>For instance, this is what we already does with libelf (see 
common/libelf).


This will amount to  adding a function make_iort in libxl__prepare_acpi,
 which would use the common code that is also use generate dom0 
IORT (domain_build.c).

Correct ?

If we go by this logic, then libxl_prepare_acpi and domain_build.c 
should use a common code for all acpi tables.
- Are you suggesting we change that as well and make it part of common 
code.


The code in domain_build.c and in libxl__prepare_acpi is very similar, 
see the code below.


static int prepare_acpi(struct domain *d, struct kernel_info *kinfo)
{

d->arch.efi_acpi_table = alloc_xenheap_pages(order, 0);
...

rc = acpi_create_fadt(d, tbl_add);
if ( rc != 0 )
return rc;

rc = acpi_create_madt(d, tbl_add);
if ( rc != 0 )
return rc;

rc = acpi_create_stao(d, tbl_add);
if ( rc != 0 )
return rc;

rc = acpi_create_xsdt(d, tbl_add);
if ( rc != 0 )
return rc;

rc = acpi_create_rsdp(d, tbl_add);
if ( rc != 0 )
return rc;

...
}

int libxl__prepare_acpi(libxl__gc *gc, libxl_domain_build_info *info,
struct xc_dom_image *dom)
{
...

rc = libxl__allocate_acpi_tables(gc, info, dom, acpitables);
if (rc)
goto out;

make_acpi_rsdp(gc, dom, acpitables);
make_acpi_xsdt(gc, dom, acpitables);
make_acpi_gtdt(gc, dom, acpitables);
rc = make_acpi_madt(gc, dom, info, acpitables);
if (rc)
goto out;

make_acpi_fadt(gc, dom, acpitables);
make_acpi_dsdt(gc, dom, acpitables);

out:
return rc;
}

Now if you see both the codes are quite similar and there is redundancy 
in libxl and in xen code for preparing ACPI tables for dom0 and domU.
The point I am raising is quite clear, if all other tables like MADT, 
XSDT, RSDP, GTDT etc does not share a common generation code with xen 
what is so special about IORT.
Either we move all generation into a common code or keep redundancy for 
IORT.


I hope I have shown the code and made the point quite clear.
Please provide a technical answer rather than a simple "Why".

Cheers!

Manish

On 10/12/2017 4:34 PM, Julien Grall wrote:

Hello,

On 12/10/17 07:11, Manish Jaggi wrote:

On 10/6/2017 7:54 PM, Julien Grall wrote:
I am not asking to write the DomU support, but at least have a full 
separation between the Parsing and Generation. So we could easily 
adapt and re-use the code when we get the DomU support.


I got your point, but as of today there is no code reuse for most of 
apci_tables. So code result _only_ for IORT but not for acpi is not 
correct approach.


Why? The generation of IORT is fairly standalone.

And again, this was suggestion to share in the future and an 
expectation for this series. What I care the most is the generation to 
be fully separated from the rest.




Also this is the part of PCI passthrough flow so that also might 
change few things.


But from pov of dom0 smmu hiding, it is a different flow and is 
coupled with PCI PT.






I think 1) can be solved using this series as a base. I have 
quite some

comments ready for the patches, shall we follow this route.

2) obviously would change the game completely. We need to sit 
down and
design this properly. Probably this means that Xen parses the 
IORT and
builds internal representations of the mappings, 
Can you please add more detail on the internal representations of 
the mappings.


What exactly do you want? This is likely going to be decided once 
you looked what is the expected interaction between IORT and Xen.
More details on this line "Probably this means that Xen parses the 
IORT and

builds internal representations of the mappings,"


I think you have enough meat in this thread to come up with a 
proposition based on the feedback. Once you send it, we can have a 
discussion and find agreement.


[...]

The IORT for the hardware domain is just a specific case as it is 
based pre-existing information. But because of removing nodes (e.g 
SMMU nodes and probably the PMU nodes), it is basically a full 
re-write.


So I would consider of full separate the logic of generating the 
IORT table from the host IORT table. By that I mean not browsing 
the host IORT when generating the host.



by "the host" you mean dom0 IORT  ?


yes.


Something on the lines of this 
 struct iort_table_struct {
 header h;

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-10-12 Thread Julien Grall 

Hello,

On 12/10/17 07:11, Manish Jaggi wrote:

On 10/6/2017 7:54 PM, Julien Grall wrote:
I am not asking to write the DomU support, but at least have a full 
separation between the Parsing and Generation. So we could easily 
adapt and re-use the code when we get the DomU support.


I got your point, but as of today there is no code reuse for most of 
apci_tables. So code result _only_ for IORT but not for acpi is not 
correct approach.


Why? The generation of IORT is fairly standalone.

And again, this was suggestion to share in the future and an expectation 
for this series. What I care the most is the generation to be fully 
separated from the rest.




Also this is the part of PCI passthrough flow so that also might 
change few things.


But from pov of dom0 smmu hiding, it is a different flow and is 
coupled with PCI PT.






I think 1) can be solved using this series as a base. I have quite 
some

comments ready for the patches, shall we follow this route.

2) obviously would change the game completely. We need to sit down 
and
design this properly. Probably this means that Xen parses the IORT 
and
builds internal representations of the mappings, 
Can you please add more detail on the internal representations of the 
mappings.


What exactly do you want? This is likely going to be decided once you 
looked what is the expected interaction between IORT and Xen.

More details on this line "Probably this means that Xen parses the IORT and
builds internal representations of the mappings,"


I think you have enough meat in this thread to come up with a 
proposition based on the feedback. Once you send it, we can have a 
discussion and find agreement.


[...]

The IORT for the hardware domain is just a specific case as it is 
based pre-existing information. But because of removing nodes (e.g 
SMMU nodes and probably the PMU nodes), it is basically a full 
re-write.


So I would consider of full separate the logic of generating the 
IORT table from the host IORT table. By that I mean not browsing the 
host IORT when generating the host.



by "the host" you mean dom0 IORT  ?


yes.


Something on the lines of this 
     struct iort_table_struct {
     header h;
     list_head pci_rc_list;
     list_head its_groups;
     list_head smmu;
     list_head platForm_devices;
     } host_iort;


Some context is probably missing.


This has two benefits:
    1) The code generation could be re-used for generating the guest 
IORT later on.

See my comment above

    2) See 2) from Andre
    3) We could decide in a finer grain which devices (e.g platform 
device) Dom0 can see.

ok,


So, IHMO, we should take a different approach from this series and 
extending the scope of it. Rather than focusing on only IORT for the 
hardware, I would be better to see IORT as a whole. I.e  how IORT 
will interact with the hypervisor?


For instance, likely you would need to parse the IORT in quite a few 
places in Xen. It would be better to get IORT parsed only once and 
store the information in Xen like data-structures.


I am thinking of reusing much of ACPI tables for that and introducing 
less abstractions.
This require more work than this current scope of this series. But I 
think it would helful for future work such as PCI passthrough support.


Any opinions?


for DomUs it is tied to PCI PT, so both design should evolve together.


Why that? The generation of IORT is pretty standard.
DomU IORT will have a virtual PCI_RC and virtual ITS_GROUP and IIUC at 
the time of generation of domU IORT (with device passthrough) toolstack 
will communicate to xen virtual / physical deviceID.
So Generation logic is standard what to put in IORT and what mappings to 
add, is a design consideration which is based on PCI_PT.
I still don't get why you are trying to argue on the information written 
in the IORT. The generation does not care whether those information are 
taken from the host IORT, virtual, a mix...


It just takes in input the information and output the IORT. That's it.



I have a question here: if a virtual e1000 device is added to domU 
(non-PCI PT case),

should it be added as a DomU platform device or a virtual pci device.


Hu? Is that e1000 devices a PCI or platform device? If you answer that, 
you answer to your question here.


Cheers,

--
Julien Grall

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-10-12 Thread Manish Jaggi 



On 10/6/2017 7:54 PM, Julien Grall wrote:

Hello,

On 04/10/17 06:22, Manish Jaggi wrote:

On 10/4/2017 12:12 AM, Julien Grall wrote:

On 25/09/17 05:22, Manish Jaggi wrote:

On 9/22/2017 7:42 PM, Andre Przywara wrote:

Hi Manish,

On 11/09/17 22:33, mja...@caviumnetworks.com wrote:

From: Manish Jaggi 

The set is divided into two patches. First one calculates the 
size of IORT

while second one writes the IORT table itself.
It would be good if you could give a quick introduction *why* this 
set

is needed here (and introduce IORT to the casual reader).
In general some more high-level documentation on your functions 
would be
good, as it took me quite some time to understand what each 
function does.

ok, will add more documentation.

So my understanding is:
phase 1:
- go over each entry in each RC node
Rather than each entry (which could be a large number) I am taking 
the complete range and checking it with the same logic.
If the ID range is a subset or a super-set of id range in smmu, new 
id range is created.


So if pci_rc node has an id map 
{p_input-base,p_output-base,p_out_ref, p_count} and it an output 
reference to smmu node with id-map
{s_input-base, s_output-base,s_out_ref,  s_count}, based on the the 
the s_count and s_input/p_output the new id-map is created with 
{p_input, s_output, s_out_ref, adjusted_count}


update_id_mapping function does that.

So I am following the same logic. We can chat over IRC / I can give 
a code walk-through ...


-   if that points to an SMMU node, go over each outgoing ITS 
entry and

find overlaps with this RC entry
- for each overlap create a new entry in a list with this RC
pointing to the ITS directly

phase 2, creating the new IORT
- go over each RC node
-   if that points to an ITS, copy through IORT entries
-   if that points to an SMMU, replace with the remapped entries
- go over each ITS node
-   copy through IORT entries

Thats exactly what this patch does.

What are you comments on the current patch approach to hide smmu nodes.
I have answered to your comments, see below.


I am not sure to understand the 2 sentences above. What are they related?


IMHO we can reuse most of the fixup code here.


That's your choice as long as it is properly documented and fits the 
end goal.





So I believe this would do the trick and you end up with an efficient
representation of the IORT without SMMUs - at least for RC nodes.

After some brainstorming with Julien we found two problems:
1) This only covers RC nodes, but not "named components" (platform
devices), which we will need. That should be fixable by removing the
hardcoded IORT node types in the code and treating NC nodes like 
RC nodes.
Yes, so first we can take this as a base, once this is ok, I can 
add support for named components.
2) Eventually we will need *virtual* deviceID support, for DomUs. 
Now we
I am a bit surprised that you answered to the e-mail but didn't 
provide any opinion on 2).

Apologies for that.

could start introducing that already, also doing some virtual mapping
for Dom0. The ITS code would then translate each virtual device ID 
that

Dom0 requests into a hardware device ID.
I agree that this means a lot more work, but we will need it anyway.


I am a bit surprised that you answered to the e-mail but didn't 
provide any opinion on 2).

Apologies for that. Sorry to surprise you twice :)


Damm, I moved the sentence but forgot to drop the original one.



IMHO It was a bit obvious for DomU and I was waiting to hear what 
other would say on this.

as (2) below.
Moreover we need to discuss IORT generation for DomU
- could be done by xl tools
or xen should do it.


The ACPI tables for DomU are generated by the toolstack today. So I 
don't see why we would change that to support IORT.


However, you can have a file shared between the toolstack and Xen and 
contain the generation of IORT.


For instance, this is what we already does with libelf (see 
common/libelf).

This will amount to  adding a function make_iort in libxl__prepare_acpi,
 which would use the common code that is also use generate dom0 
IORT (domain_build.c).

Correct ?

If we go by this logic, then libxl_prepare_acpi and domain_build.c 
should use a common code for all acpi tables.

- Are you suggesting we change that as well and make it part of common code.



I am not asking to write the DomU support, but at least have a full 
separation between the Parsing and Generation. So we could easily 
adapt and re-use the code when we get the DomU support.


I got your point, but as of today there is no code reuse for most of 
apci_tables. So code result _only_ for IORT but not for acpi is not 
correct approach.


Also this is the part of PCI passthrough flow so that also might 
change few things.


But from pov of dom0 smmu hiding, it is a different flow and is 
coupled with PCI PT.






I think 1) can be solved using this series as a base. I have quite 
some

comments ready for the patches,

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-10-06 Thread Julien Grall 

Hello,

On 04/10/17 06:22, Manish Jaggi wrote:

On 10/4/2017 12:12 AM, Julien Grall wrote:

On 25/09/17 05:22, Manish Jaggi wrote:

On 9/22/2017 7:42 PM, Andre Przywara wrote:

Hi Manish,

On 11/09/17 22:33, mja...@caviumnetworks.com wrote:

From: Manish Jaggi 

The set is divided into two patches. First one calculates the size 
of IORT

while second one writes the IORT table itself.

It would be good if you could give a quick introduction *why* this set
is needed here (and introduce IORT to the casual reader).
In general some more high-level documentation on your functions 
would be
good, as it took me quite some time to understand what each function 
does.

ok, will add more documentation.

So my understanding is:
phase 1:
- go over each entry in each RC node
Rather than each entry (which could be a large number) I am taking 
the complete range and checking it with the same logic.
If the ID range is a subset or a super-set of id range in smmu, new 
id range is created.


So if pci_rc node has an id map 
{p_input-base,p_output-base,p_out_ref, p_count} and it an output 
reference to smmu node with id-map
{s_input-base, s_output-base,s_out_ref,  s_count}, based on the the 
the s_count and s_input/p_output the new id-map is created with 
{p_input, s_output, s_out_ref, adjusted_count}


update_id_mapping function does that.

So I am following the same logic. We can chat over IRC / I can give a 
code walk-through ...



-   if that points to an SMMU node, go over each outgoing ITS entry and
find overlaps with this RC entry
- for each overlap create a new entry in a list with this RC
pointing to the ITS directly

phase 2, creating the new IORT
- go over each RC node
-   if that points to an ITS, copy through IORT entries
-   if that points to an SMMU, replace with the remapped entries
- go over each ITS node
-   copy through IORT entries

Thats exactly what this patch does.

What are you comments on the current patch approach to hide smmu nodes.
I have answered to your comments, see below.


I am not sure to understand the 2 sentences above. What are they related?


IMHO we can reuse most of the fixup code here.


That's your choice as long as it is properly documented and fits the end 
goal.





So I believe this would do the trick and you end up with an efficient
representation of the IORT without SMMUs - at least for RC nodes.

After some brainstorming with Julien we found two problems:
1) This only covers RC nodes, but not "named components" (platform
devices), which we will need. That should be fixable by removing the
hardcoded IORT node types in the code and treating NC nodes like RC 
nodes.
Yes, so first we can take this as a base, once this is ok, I can add 
support for named components.
2) Eventually we will need *virtual* deviceID support, for DomUs. 
Now we
I am a bit surprised that you answered to the e-mail but didn't 
provide any opinion on 2).

Apologies for that.

could start introducing that already, also doing some virtual mapping
for Dom0. The ITS code would then translate each virtual device ID that
Dom0 requests into a hardware device ID.
I agree that this means a lot more work, but we will need it anyway.


I am a bit surprised that you answered to the e-mail but didn't 
provide any opinion on 2).

Apologies for that. Sorry to surprise you twice :)


Damm, I moved the sentence but forgot to drop the original one.



IMHO It was a bit obvious for DomU and I was waiting to hear what other 
would say on this.

as (2) below.
Moreover we need to discuss IORT generation for DomU
- could be done by xl tools
or xen should do it.


The ACPI tables for DomU are generated by the toolstack today. So I 
don't see why we would change that to support IORT.


However, you can have a file shared between the toolstack and Xen and 
contain the generation of IORT.


For instance, this is what we already does with libelf (see common/libelf).

I am not asking to write the DomU support, but at least have a full 
separation between the Parsing and Generation. So we could easily adapt 
and re-use the code when we get the DomU support.




Also this is the part of PCI passthrough flow so that also might change 
few things.


But from pov of dom0 smmu hiding, it is a different flow and is coupled 
with PCI PT.






I think 1) can be solved using this series as a base. I have quite some
comments ready for the patches, shall we follow this route.

2) obviously would change the game completely. We need to sit down and
design this properly. Probably this means that Xen parses the IORT and
builds internal representations of the mappings, 
Can you please add more detail on the internal representations of the 
mappings.


What exactly do you want? This is likely going to be decided once you 
looked what is the expected interaction between IORT and Xen.


IIUC the information is already there in ACPI tables, would it not add 
extra overhead of abstractions to maintain.
Enumeration of PCI devices

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-10-03 Thread Manish Jaggi 

Hello Julien,

On 10/4/2017 12:12 AM, Julien Grall wrote:

Hello,

On 25/09/17 05:22, Manish Jaggi wrote:

On 9/22/2017 7:42 PM, Andre Przywara wrote:

Hi Manish,

On 11/09/17 22:33, mja...@caviumnetworks.com wrote:

From: Manish Jaggi 

The set is divided into two patches. First one calculates the size 
of IORT

while second one writes the IORT table itself.

It would be good if you could give a quick introduction *why* this set
is needed here (and introduce IORT to the casual reader).
In general some more high-level documentation on your functions 
would be
good, as it took me quite some time to understand what each function 
does.

ok, will add more documentation.

So my understanding is:
phase 1:
- go over each entry in each RC node
Rather than each entry (which could be a large number) I am taking 
the complete range and checking it with the same logic.
If the ID range is a subset or a super-set of id range in smmu, new 
id range is created.


So if pci_rc node has an id map 
{p_input-base,p_output-base,p_out_ref, p_count} and it an output 
reference to smmu node with id-map
{s_input-base, s_output-base,s_out_ref,  s_count}, based on the the 
the s_count and s_input/p_output the new id-map is created with 
{p_input, s_output, s_out_ref, adjusted_count}


update_id_mapping function does that.

So I am following the same logic. We can chat over IRC / I can give a 
code walk-through ...



-   if that points to an SMMU node, go over each outgoing ITS entry and
find overlaps with this RC entry
- for each overlap create a new entry in a list with this RC
pointing to the ITS directly

phase 2, creating the new IORT
- go over each RC node
-   if that points to an ITS, copy through IORT entries
-   if that points to an SMMU, replace with the remapped entries
- go over each ITS node
-   copy through IORT entries

Thats exactly what this patch does.

What are you comments on the current patch approach to hide smmu nodes.
I have answered to your comments, see below.
IMHO we can reuse most of the fixup code here.


So I believe this would do the trick and you end up with an efficient
representation of the IORT without SMMUs - at least for RC nodes.

After some brainstorming with Julien we found two problems:
1) This only covers RC nodes, but not "named components" (platform
devices), which we will need. That should be fixable by removing the
hardcoded IORT node types in the code and treating NC nodes like RC 
nodes.
Yes, so first we can take this as a base, once this is ok, I can add 
support for named components.
2) Eventually we will need *virtual* deviceID support, for DomUs. 
Now we
I am a bit surprised that you answered to the e-mail but didn't 
provide any opinion on 2).

Apologies for that.

could start introducing that already, also doing some virtual mapping
for Dom0. The ITS code would then translate each virtual device ID that
Dom0 requests into a hardware device ID.
I agree that this means a lot more work, but we will need it anyway.


I am a bit surprised that you answered to the e-mail but didn't 
provide any opinion on 2).

Apologies for that. Sorry to surprise you twice :)

IMHO It was a bit obvious for DomU and I was waiting to hear what other 
would say on this.

as (2) below.
Moreover we need to discuss IORT generation for DomU
- could be done by xl tools
or xen should do it.

Also this is the part of PCI passthrough flow so that also might change 
few things.


But from pov of dom0 smmu hiding, it is a different flow and is coupled 
with PCI PT.






I think 1) can be solved using this series as a base. I have quite some
comments ready for the patches, shall we follow this route.

2) obviously would change the game completely. We need to sit down and
design this properly. Probably this means that Xen parses the IORT and
builds internal representations of the mappings, 
Can you please add more detail on the internal representations of the 
mappings.
IIUC the information is already there in ACPI tables, would it not add 
extra overhead of abstractions to maintain.
Enumeration of PCI devices would generate a pci list which would be 
anyways separate.

which are consulted as
needed when passing through devices. The guest's (that would include
Dom0) IORT would then be generated completely from scratch.

I have a different opinion here, dom0 IORT would is most cases be very 
close to host IORT sans smmu nodes and few platform devices.
And which platform devices to hide would probably depend on the xen 
command line,
For instance for dom0 we would copy ITS information while for domU it 
would have to be generated, so scratch would be more for domU.
We could have a common code for creating IORT structure but it would be 
a bit complex code with lot of abstractions and callbacks, so I suggest 
that keeping code simpler would be better.

I would like to hear your opinion on this. I will try to discuss the
feasibility of 2) with people at Connect. It would be good if we could

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-10-03 Thread Julien Grall 

Hello,

On 25/09/17 05:22, Manish Jaggi wrote:

On 9/22/2017 7:42 PM, Andre Przywara wrote:

Hi Manish,

On 11/09/17 22:33, mja...@caviumnetworks.com wrote:

From: Manish Jaggi 

The set is divided into two patches. First one calculates the size of 
IORT

while second one writes the IORT table itself.

It would be good if you could give a quick introduction *why* this set
is needed here (and introduce IORT to the casual reader).
In general some more high-level documentation on your functions would be
good, as it took me quite some time to understand what each function 
does.

ok, will add more documentation.

So my understanding is:
phase 1:
- go over each entry in each RC node
Rather than each entry (which could be a large number) I am taking the 
complete range and checking it with the same logic.
If the ID range is a subset or a super-set of id range in smmu, new id 
range is created.


So if pci_rc node has an id map {p_input-base,p_output-base,p_out_ref, 
p_count} and it an output reference to smmu node with id-map
{s_input-base, s_output-base,s_out_ref,  s_count}, based on the the the 
s_count and s_input/p_output the new id-map is created with {p_input, 
s_output, s_out_ref, adjusted_count}


update_id_mapping function does that.

So I am following the same logic. We can chat over IRC / I can give a 
code walk-through ...



-   if that points to an SMMU node, go over each outgoing ITS entry and
find overlaps with this RC entry
- for each overlap create a new entry in a list with this RC
pointing to the ITS directly

phase 2, creating the new IORT
- go over each RC node
-   if that points to an ITS, copy through IORT entries
-   if that points to an SMMU, replace with the remapped entries
- go over each ITS node
-   copy through IORT entries

Thats exactly what this patch does.

So I believe this would do the trick and you end up with an efficient
representation of the IORT without SMMUs - at least for RC nodes.

After some brainstorming with Julien we found two problems:
1) This only covers RC nodes, but not "named components" (platform
devices), which we will need. That should be fixable by removing the
hardcoded IORT node types in the code and treating NC nodes like RC 
nodes.
Yes, so first we can take this as a base, once this is ok, I can add 
support for named components.

2) Eventually we will need *virtual* deviceID support, for DomUs. Now we
I am a bit surprised that you answered to the e-mail but didn't provide 
any opinion on 2).

could start introducing that already, also doing some virtual mapping
for Dom0. The ITS code would then translate each virtual device ID that
Dom0 requests into a hardware device ID.
I agree that this means a lot more work, but we will need it anyway.


I am a bit surprised that you answered to the e-mail but didn't provide 
any opinion on 2).




I think 1) can be solved using this series as a base. I have quite some
comments ready for the patches, shall we follow this route.

2) obviously would change the game completely. We need to sit down and
design this properly. Probably this means that Xen parses the IORT and
builds internal representations of the mappings, which are consulted as
needed when passing through devices. The guest's (that would include
Dom0) IORT would then be generated completely from scratch.

I would like to hear your opinion on this. I will try to discuss the
feasibility of 2) with people at Connect. It would be good if we could
decide whether this is the way to go or we should use a solution based
on this series.


Andre, Stefano and I had a chat during connect about how we want to see 
IORT support in Xen. In the near future, IORT will be used for different 
components accross the hypervisor (ITS, SMMU...) and as a way to

communicate the topology to the guests.

The IORT for the hardware domain is just a specific case as it is based 
pre-existing information. But because of removing nodes (e.g SMMU nodes 
and probably the PMU nodes), it is basically a full re-write.


So I would consider of full separate the logic of generating the IORT 
table from the host IORT table. By that I mean not browsing the host 
IORT when generating the host.


This has two benefits:
	1) The code generation could be re-used for generating the guest IORT 
later on.

2) See 2) from Andre
	3) We could decide in a finer grain which devices (e.g platform device) 
Dom0 can see.


So, IHMO, we should take a different approach from this series and 
extending the scope of it. Rather than focusing on only IORT for the 
hardware, I would be better to see IORT as a whole. I.e  how IORT will 
interact with the hypervisor?


For instance, likely you would need to parse the IORT in quite a few 
places in Xen. It would be better to get IORT parsed only once and store 
the information in Xen like data-structures.


This require more work than this current scope of this series. But I 
think it would helful for future work such as PCI

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-09-24 Thread Manish Jaggi 

Hi Andre,

On 9/22/2017 7:42 PM, Andre Przywara wrote:

Hi Manish,

On 11/09/17 22:33, mja...@caviumnetworks.com wrote:

From: Manish Jaggi 

The set is divided into two patches. First one calculates the size of IORT
while second one writes the IORT table itself.

It would be good if you could give a quick introduction *why* this set
is needed here (and introduce IORT to the casual reader).
In general some more high-level documentation on your functions would be
good, as it took me quite some time to understand what each function does.

ok, will add more documentation.

So my understanding is:
phase 1:
- go over each entry in each RC node
Rather than each entry (which could be a large number) I am taking the 
complete range and checking it with the same logic.
If the ID range is a subset or a super-set of id range in smmu, new id 
range is created.


So if pci_rc node has an id map {p_input-base,p_output-base,p_out_ref, 
p_count} and it an output reference to smmu node with id-map
{s_input-base, s_output-base,s_out_ref,  s_count}, based on the the the 
s_count and s_input/p_output the new id-map is created with {p_input, 
s_output, s_out_ref, adjusted_count}


update_id_mapping function does that.

So I am following the same logic. We can chat over IRC / I can give a 
code walk-through ...



-   if that points to an SMMU node, go over each outgoing ITS entry and
find overlaps with this RC entry
- for each overlap create a new entry in a list with this RC
pointing to the ITS directly

phase 2, creating the new IORT
- go over each RC node
-   if that points to an ITS, copy through IORT entries
-   if that points to an SMMU, replace with the remapped entries
- go over each ITS node
-   copy through IORT entries

Thats exactly what this patch does.

So I believe this would do the trick and you end up with an efficient
representation of the IORT without SMMUs - at least for RC nodes.

After some brainstorming with Julien we found two problems:
1) This only covers RC nodes, but not "named components" (platform
devices), which we will need. That should be fixable by removing the
hardcoded IORT node types in the code and treating NC nodes like RC nodes.
Yes, so first we can take this as a base, once this is ok, I can add 
support for named components.

2) Eventually we will need *virtual* deviceID support, for DomUs. Now we
could start introducing that already, also doing some virtual mapping
for Dom0. The ITS code would then translate each virtual device ID that
Dom0 requests into a hardware device ID.
I agree that this means a lot more work, but we will need it anyway.

I think 1) can be solved using this series as a base. I have quite some
comments ready for the patches, shall we follow this route.

2) obviously would change the game completely. We need to sit down and
design this properly. Probably this means that Xen parses the IORT and
builds internal representations of the mappings, which are consulted as
needed when passing through devices. The guest's (that would include
Dom0) IORT would then be generated completely from scratch.

I would like to hear your opinion on this. I will try to discuss the
feasibility of 2) with people at Connect. It would be good if we could
decide whether this is the way to go or we should use a solution based
on this series.

Cheers,
Andre.



patch1: estimates size of hardware domain IORT table by parsing all
the pcirc nodes and their idmaps, and thereby calculating size by
removing smmu nodes.

Hardware domain IORT table will have only ITS and PCIRC nodes, and PCIRC
nodes' idmap will have output refrences to ITS group nodes.

patch 2: The steps are:
a. First ITS group nodes are written and their offsets are saved
along with the respective offsets from the firmware table.
This is required when smmu node is hidden and smmu node still points
to the old output_reference.

b. PCIRC idmap is parsed and a list of idmaps is created which will
have PCIRC idmap -> ITS group nodes.
Each idmap is written by resolving ITS offset from the map saved in
previous step.

Changes wrt v1:
No assumption is made wrt format of IORT / hw support

Manish Jaggi (2):
   ARM: ACPI: IORT: Estimate the size of hardware domain IORT table
   ARM: ACPI: IORT: Write Hardware domain's IORT table

  xen/arch/arm/acpi/Makefile  |   1 +
  xen/arch/arm/acpi/iort.c| 414 
  xen/arch/arm/domain_build.c |  49 +-
  xen/include/asm-arm/acpi.h  |   1 +
  xen/include/asm-arm/iort.h  |  17 ++
  5 files changed, 481 insertions(+), 1 deletion(-)
  create mode 100644 xen/arch/arm/acpi/iort.c
  create mode 100644 xen/include/asm-arm/iort.h




___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel

Re: [Xen-devel] [PATCH v2 0/2] ARM: ACPI: IORT: Hide SMMU from hardware domain's IORT table

2017-09-22 Thread Andre Przywara 
Hi Manish,

On 11/09/17 22:33, mja...@caviumnetworks.com wrote:
> From: Manish Jaggi 
> 
> The set is divided into two patches. First one calculates the size of IORT
> while second one writes the IORT table itself.

It would be good if you could give a quick introduction *why* this set
is needed here (and introduce IORT to the casual reader).
In general some more high-level documentation on your functions would be
good, as it took me quite some time to understand what each function does.

So my understanding is:
phase 1:
- go over each entry in each RC node
-   if that points to an SMMU node, go over each outgoing ITS entry and
find overlaps with this RC entry
- for each overlap create a new entry in a list with this RC
pointing to the ITS directly

phase 2, creating the new IORT
- go over each RC node
-   if that points to an ITS, copy through IORT entries
-   if that points to an SMMU, replace with the remapped entries
- go over each ITS node
-   copy through IORT entries

So I believe this would do the trick and you end up with an efficient
representation of the IORT without SMMUs - at least for RC nodes.

After some brainstorming with Julien we found two problems:
1) This only covers RC nodes, but not "named components" (platform
devices), which we will need. That should be fixable by removing the
hardcoded IORT node types in the code and treating NC nodes like RC nodes.
2) Eventually we will need *virtual* deviceID support, for DomUs. Now we
could start introducing that already, also doing some virtual mapping
for Dom0. The ITS code would then translate each virtual device ID that
Dom0 requests into a hardware device ID.
I agree that this means a lot more work, but we will need it anyway.

I think 1) can be solved using this series as a base. I have quite some
comments ready for the patches, shall we follow this route.

2) obviously would change the game completely. We need to sit down and
design this properly. Probably this means that Xen parses the IORT and
builds internal representations of the mappings, which are consulted as
needed when passing through devices. The guest's (that would include
Dom0) IORT would then be generated completely from scratch.

I would like to hear your opinion on this. I will try to discuss the
feasibility of 2) with people at Connect. It would be good if we could
decide whether this is the way to go or we should use a solution based
on this series.

Cheers,
Andre.


> patch1: estimates size of hardware domain IORT table by parsing all
> the pcirc nodes and their idmaps, and thereby calculating size by
> removing smmu nodes.
> 
> Hardware domain IORT table will have only ITS and PCIRC nodes, and PCIRC
> nodes' idmap will have output refrences to ITS group nodes.
> 
> patch 2: The steps are:
> a. First ITS group nodes are written and their offsets are saved
> along with the respective offsets from the firmware table.
> This is required when smmu node is hidden and smmu node still points
> to the old output_reference.
> 
> b. PCIRC idmap is parsed and a list of idmaps is created which will
> have PCIRC idmap -> ITS group nodes.
> Each idmap is written by resolving ITS offset from the map saved in
> previous step.
> 
> Changes wrt v1:
> No assumption is made wrt format of IORT / hw support
> 
> Manish Jaggi (2):
>   ARM: ACPI: IORT: Estimate the size of hardware domain IORT table
>   ARM: ACPI: IORT: Write Hardware domain's IORT table
> 
>  xen/arch/arm/acpi/Makefile  |   1 +
>  xen/arch/arm/acpi/iort.c| 414 
> 
>  xen/arch/arm/domain_build.c |  49 +-
>  xen/include/asm-arm/acpi.h  |   1 +
>  xen/include/asm-arm/iort.h  |  17 ++
>  5 files changed, 481 insertions(+), 1 deletion(-)
>  create mode 100644 xen/arch/arm/acpi/iort.c
>  create mode 100644 xen/include/asm-arm/iort.h
> 

___
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel