Re: Fwd: umm_map returns unaligned address?

2021-04-23 Thread Philip Guenther
On Fri, Apr 23, 2021 at 4:50 PM Alessandro Pistocchi 
wrote:
...

> What I was flagging is just that sometimes uvm_map returns an address that
> is not
> aligned to PAGE_SIZE ( I printed it out and it has 0x004 in the lower 12
> bits).On the
>
other hand uvm_unmap has an assertion that panics if the address passed to
> it is not
> page aligned. I believe that there could be a bug somewhere.
>

You apparently didn't print out the value directly after return from
uvm_map() but rather later after a bunch of your other code had run.  Yes,
there's a bug, in your game_mode_start_audio_thread(), where you advance
the pointer from uvm_map() by four.


Philip Guenther


Re: Fwd: umm_map returns unaligned address?

2021-04-23 Thread Alessandro Pistocchi
Hi,
apologies again.

I am not familiar with cvs so I have used the diff command between an
original sys folder and mine.

I am attaching the diff file and two files to be put in sys/kern. Sorry but
the diff
command did not include the content of the two files in the diff itself.

Please notice that this is not code that is intended to be put into the
openbsd kernel,
as it would probably not be interesting to the general users.

What I am doing here is to have a syscall that stops the scheduler on 3 of
the 4 cores,
runs my own code on those while the calling process runs on the other core
and restarts
scheduling on the 3 cores when the process terminates.
The shared memory is used for the 3 cores to communicate with the process.
I know that it might sound crazy to you but it does make a lot of sense for
what
I am trying to achieve.

Regarding the mapping, I hope that it works well enough even with
reordering issues and other stuff.
It does for the example I am sending and I don't need much more than that.

What I was flagging is just that sometimes uvm_map returns an address that
is not
aligned to PAGE_SIZE ( I printed it out and it has 0x004 in the lower 12
bits). On the
other hand uvm_unmap has an assertion that panics if the address passed to
it is not
page aligned. I believe that there could be a bug somewhere.

In previous runs I was getting constantly an address which is not aligned
but now
I get an address that is aligned.

Best :-)
Alessandro

On Sat, Apr 24, 2021 at 2:32 AM Theo de Raadt  wrote:

> Alessandro Pistocchi  wrote:
>
> > During the syscall I allocate some memory that I want to share between
> the
> > kernel and the calling process.
>
> When you get the mapping working, it will not work as well as you like.
>
> Compiler re-ordering of writes & reads, caches, write buffers, and other
> details such as non-TSO will creat problems, meaning you'll need very
> careful
> data-structure, and great care with co-dependent fields, otherwise one side
> can fool the other.
>
> That is why the copyin/copyout approach is used as a high-level
> serializing-forcing
> primitive in most operating systems.
>
> But since you don't show your whole diff.. I suspect this is a waste of
> time.
>
struct uvm_object *game_uvm_object;

void game_mode_create_shared_memory_region(struct proc *p) {
	game_uvm_object = uao_create(game_memory_size, 0);
	if(!game_uvm_object) return;

	// TODO(ale): make sure that this memory cannot be swapped out

	uao_reference(game_uvm_object);
	if(uvm_map(kernel_map, (vaddr_t *)_memory, round_page(game_memory_size), game_uvm_object,
		0, 0, UVM_MAPFLAG(PROT_READ | PROT_WRITE, PROT_READ | PROT_WRITE,
		MAP_INHERIT_SHARE, MADV_NORMAL, 0))) {
		uao_detach(game_uvm_object);
		game_uvm_object = 0;
		return;
	}

	uao_reference(game_uvm_object);
	if(uvm_map(>p_vmspace->vm_map, (vaddr_t *)_memory_in_proc_space, round_page(game_memory_size), game_uvm_object,
		0, 0, UVM_MAPFLAG(PROT_READ | PROT_WRITE, PROT_READ | PROT_WRITE,
		MAP_INHERIT_NONE, MADV_NORMAL, 0))) {
		// TODO(ale): the kernel mapping returns an address for game_memory that is 4 bytes more than page aligned. This panics.
		uvm_unmap(kernel_map, trunc_page((vaddr_t)game_memory), round_page((vaddr_t)game_memory + game_memory_size));
		game_memory = 0;
		uao_detach(game_uvm_object);
		uao_detach(game_uvm_object);
		game_uvm_object = 0;
		return;
	}

	game_mode_proc = p;

}

void game_mode_free_shared_memory_region(void) {
	if(game_memory_in_proc_space)uvm_unmap(_mode_proc->p_vmspace->vm_map, trunc_page((vaddr_t)game_memory_in_proc_space),
		round_page((vaddr_t)game_memory_in_proc_space + game_memory_size));
	// TODO(ale): the kernel mapping returns an address for game_memory that is 4 bytes more than page aligned. This panics.
	if(game_memory)uvm_unmap(kernel_map, trunc_page((vaddr_t)game_memory), round_page((vaddr_t)game_memory + game_memory_size));
	if(game_uvm_object){
		uao_detach(game_uvm_object);
		uao_detach(game_uvm_object);
	}
	game_memory_in_proc_space = game_memory = 0;
	game_uvm_object = 0;
}

// static inline void *phisical_from_virtual(struct proc *p, void *va) {
// 	paddr_t pa;
// 	vaddr_t faddr = trunc_page((vaddr_t)va), off = ((vaddr_t)va) - faddr;
// 	if (pmap_extract(vm_map_pmap(>p_vmspace->vm_map), faddr, ) == FALSE) panic("vmapbuf: null page frame");
// 	return (void *)(pa+(paddr_t)off);
// }
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include 
#include  
#include 

#define VBLANK_THREAD 		1
#define AUDIO_THREAD 		2
#define INPUT_THREAD 		3

void *game_memory;
void *game_memory_in_proc_space;
vsize_t game_memory_size;

void sched_start_secondary_cpus(void);
void sched_stop_secondary_cpus(void);

void game_mode_create_shared_memory_region(struct proc *p);
void game_mode_free_shared_memory_region(void);

volatile int game_mode_on;
struct proc *game_mode_proc;
volatile int game_mode_number_of_threads;

int 

Re: Fwd: umm_map returns unaligned address?

2021-04-23 Thread Theo de Raadt
Alessandro Pistocchi  wrote:

> During the syscall I allocate some memory that I want to share between the
> kernel and the calling process.

When you get the mapping working, it will not work as well as you like.

Compiler re-ordering of writes & reads, caches, write buffers, and other
details such as non-TSO will creat problems, meaning you'll need very careful
data-structure, and great care with co-dependent fields, otherwise one side
can fool the other.

That is why the copyin/copyout approach is used as a high-level 
serializing-forcing
primitive in most operating systems.

But since you don't show your whole diff.. I suspect this is a waste of time.



Fwd: umm_map returns unaligned address?

2021-04-23 Thread Alessandro Pistocchi
-- Forwarded message -
From: Alessandro Pistocchi 
Date: Fri, Apr 23, 2021 at 1:55 PM
Subject: umm_map returns unaligned address?
To: 


Hi all,

I am fairly new to openbsd so if this is something obvious that I missed
please be understanding.

I am adding a syscall to openbsd 6.8. I am working on a raspberry pi.

During the syscall I allocate some memory that I want to share between the
kernel
and the calling process.

When it's time to wrap up and unmap the memory, I unmap it both from the
kernel
map and from the process map.

The unmapping from the process map goes fine, the unmapping from the kernel
map
fails by saying that the virtual address in kernel map is not aligned to
the page size
( it's actually 4 bytes off ).

What have I missed? I assumed that umm_map would return a page aligned
virtual
address for the kernel mapping as well.

Here is my code for creating the shared memory chunk:


// memory_size is a multiple of page size
uvm_object = uao_create(memory_size, 0);
if(!uvm_object) return;

// TODO(ale): make sure that this memory cannot be swapped out

uao_reference(uvm_object)
if(uvm_map(kernel_map, (vaddr_t *), round_page(memory_size),
uvm_object,
   0, 0, UVM_MAPFLAG(PROT_READ | PROT_WRITE, PROT_READ | PROT_WRITE,
   MAP_INHERIT_SHARED, MADV_NORMAL, 0))) {
uao_detach(uvm_object);
uvm_object = 0;
return;
}

uao_reference(uvm_object);
if(uvm_map(>p_vmspace->vm_map, _in_proc_space,
round_page(memory_size), uvm_object,
   0, 0, UVM_MAPFLAG(PROT_READ | PROT_WRITE, PROT_READ | PROT_WRITE,
   MAP_INHERIT_NONE, MADV_NORMAL, 0))) {
memory = 0;
uao_detach(uvm_object);
uao_detach(uvm_object);
uvm_object = 0;
return;
}


Thanks,
A