Re: [PATCH 18/27] habanalabs: change user interrupt to threaded IRQ

2023-02-16 Thread Stanislaw Gruszka 
On Thu, Feb 16, 2023 at 03:47:44PM +0200, Oded Gabbay wrote:
> On Thu, Feb 16, 2023 at 12:28 PM Stanislaw Gruszka
>  wrote:
> >
> > Hi
> >
> > On Sun, Feb 12, 2023 at 10:44:45PM +0200, Oded Gabbay wrote:
> >
> > >  irqreturn_t hl_irq_handler_user_interrupt(int irq, void *arg)
> > > +{
> > > + return IRQ_WAKE_THREAD;
> > > +}
> >
> > This is not needed. You can pass NULL to request_threaded_irq() and
> > the irq core will use irq_default_primary_handler() which is exactly
> > the same function :-)
> >
> > Regards
> > Stanislaw
> >
> >
> You are correct but in patch 19/27 (the one after this), this function
> is filled with actual code, so I don't know if it's worth changing
> this patch...

I see, no need to change this patch if the function will be extended.

Regards
Stanislaw

Re: [PATCH 18/27] habanalabs: change user interrupt to threaded IRQ

2023-02-16 Thread Oded Gabbay 
On Thu, Feb 16, 2023 at 12:39 PM Stanislaw Gruszka
 wrote:
>
> On Sun, Feb 12, 2023 at 10:44:45PM +0200, Oded Gabbay wrote:
> > - rc = request_irq(irq, irq_handler, 0, gaudi2_irq_name(i), 
> > >user_interrupt[j]);
> > + rc = request_threaded_irq(irq, irq_handler, 
> > hl_irq_user_interrupt_thread_handler,
> > + IRQF_ONESHOT, gaudi2_irq_name(i), 
> > >user_interrupt[j]);
> > +
>
> Would be nice to change to devm_ and simplify exit paths. Up to you.
>
> Regards
> Stanislaw
>
Using drm helpers is a part of a much larger task of connecting the
habanalabs driver to accel/drm.
We are working on it now, but we will do it in parts, as this task
will take many months.

Oded

Re: [PATCH 18/27] habanalabs: change user interrupt to threaded IRQ

2023-02-16 Thread Oded Gabbay 
On Thu, Feb 16, 2023 at 12:28 PM Stanislaw Gruszka
 wrote:
>
> Hi
>
> On Sun, Feb 12, 2023 at 10:44:45PM +0200, Oded Gabbay wrote:
>
> >  irqreturn_t hl_irq_handler_user_interrupt(int irq, void *arg)
> > +{
> > + return IRQ_WAKE_THREAD;
> > +}
>
> This is not needed. You can pass NULL to request_threaded_irq() and
> the irq core will use irq_default_primary_handler() which is exactly
> the same function :-)
>
> Regards
> Stanislaw
>
>
You are correct but in patch 19/27 (the one after this), this function
is filled with actual code, so I don't know if it's worth changing
this patch...
Oded

Re: [PATCH 18/27] habanalabs: change user interrupt to threaded IRQ

2023-02-16 Thread Stanislaw Gruszka 
On Sun, Feb 12, 2023 at 10:44:45PM +0200, Oded Gabbay wrote:
> - rc = request_irq(irq, irq_handler, 0, gaudi2_irq_name(i), 
> >user_interrupt[j]);
> + rc = request_threaded_irq(irq, irq_handler, 
> hl_irq_user_interrupt_thread_handler,
> + IRQF_ONESHOT, gaudi2_irq_name(i), 
> >user_interrupt[j]);
> +

Would be nice to change to devm_ and simplify exit paths. Up to you. 

Regards
Stanislaw

Re: [PATCH 18/27] habanalabs: change user interrupt to threaded IRQ

2023-02-16 Thread Stanislaw Gruszka 
Hi

On Sun, Feb 12, 2023 at 10:44:45PM +0200, Oded Gabbay wrote:

>  irqreturn_t hl_irq_handler_user_interrupt(int irq, void *arg)
> +{
> + return IRQ_WAKE_THREAD;
> +}

This is not needed. You can pass NULL to request_threaded_irq() and
the irq core will use irq_default_primary_handler() which is exactly
the same function :-)

Regards
Stanislaw

[PATCH 18/27] habanalabs: change user interrupt to threaded IRQ

2023-02-12 Thread Oded Gabbay 
From: Tal Cohen 

We prefer not to handle the user interrupt job inside the interrupt
context. Instead, use threaded IRQ to handle the user interrupts.
This will allow to avoid disabling interrupts when the user process
registers for a new event and to avoid long handling inside an
interrupt.

Signed-off-by: Tal Cohen 
Reviewed-by: Oded Gabbay 
Signed-off-by: Oded Gabbay 
---
 .../habanalabs/common/command_submission.c| 45 +--
 drivers/accel/habanalabs/common/habanalabs.h  |  1 +
 drivers/accel/habanalabs/common/irq.c | 13 ++
 drivers/accel/habanalabs/gaudi2/gaudi2.c  | 29 +++-
 4 files changed, 53 insertions(+), 35 deletions(-)

diff --git a/drivers/accel/habanalabs/common/command_submission.c 
b/drivers/accel/habanalabs/common/command_submission.c
index 24f3d82ee4cb..6c4d9b1aa5de 100644
--- a/drivers/accel/habanalabs/common/command_submission.c
+++ b/drivers/accel/habanalabs/common/command_submission.c
@@ -1082,9 +1082,8 @@ static void
 wake_pending_user_interrupt_threads(struct hl_user_interrupt *interrupt)
 {
struct hl_user_pending_interrupt *pend, *temp;
-   unsigned long flags;
 
-   spin_lock_irqsave(>wait_list_lock, flags);
+   spin_lock(>wait_list_lock);
list_for_each_entry_safe(pend, temp, >wait_list_head, 
wait_list_node) {
if (pend->ts_reg_info.buf) {
list_del(>wait_list_node);
@@ -1095,7 +1094,7 @@ wake_pending_user_interrupt_threads(struct 
hl_user_interrupt *interrupt)
complete_all(>fence.completion);
}
}
-   spin_unlock_irqrestore(>wait_list_lock, flags);
+   spin_unlock(>wait_list_lock);
 }
 
 void hl_release_pending_user_interrupts(struct hl_device *hdev)
@@ -3159,7 +3158,7 @@ static int ts_buff_get_kernel_ts_record(struct 
hl_mmap_mem_buf *buf,
struct hl_user_pending_interrupt *cb_last =
(struct hl_user_pending_interrupt 
*)ts_buff->kernel_buff_address +
(ts_buff->kernel_buff_size / sizeof(struct 
hl_user_pending_interrupt));
-   unsigned long flags, iter_counter = 0;
+   unsigned long iter_counter = 0;
u64 current_cq_counter;
ktime_t timestamp;
 
@@ -3173,7 +3172,7 @@ static int ts_buff_get_kernel_ts_record(struct 
hl_mmap_mem_buf *buf,
timestamp = ktime_get();
 
 start_over:
-   spin_lock_irqsave(wait_list_lock, flags);
+   spin_lock(wait_list_lock);
 
/* Unregister only if we didn't reach the target value
 * since in this case there will be no handling in irq context
@@ -3184,7 +3183,7 @@ static int ts_buff_get_kernel_ts_record(struct 
hl_mmap_mem_buf *buf,
current_cq_counter = *requested_offset_record->cq_kernel_addr;
if (current_cq_counter < 
requested_offset_record->cq_target_value) {
list_del(_offset_record->wait_list_node);
-   spin_unlock_irqrestore(wait_list_lock, flags);
+   spin_unlock(wait_list_lock);
 

hl_mmap_mem_buf_put(requested_offset_record->ts_reg_info.buf);
hl_cb_put(requested_offset_record->ts_reg_info.cq_cb);
@@ -3195,8 +3194,8 @@ static int ts_buff_get_kernel_ts_record(struct 
hl_mmap_mem_buf *buf,
dev_dbg(buf->mmg->dev,
"ts node in middle of irq handling\n");
 
-   /* irq handling in the middle give it time to finish */
-   spin_unlock_irqrestore(wait_list_lock, flags);
+   /* irq thread handling in the middle give it time to 
finish */
+   spin_unlock(wait_list_lock);
usleep_range(100, 1000);
if (++iter_counter == MAX_TS_ITER_NUM) {
dev_err(buf->mmg->dev,
@@ -3217,7 +3216,7 @@ static int ts_buff_get_kernel_ts_record(struct 
hl_mmap_mem_buf *buf,
(u64 *) cq_cb->kernel_address + cq_offset;
requested_offset_record->cq_target_value = target_value;
 
-   spin_unlock_irqrestore(wait_list_lock, flags);
+   spin_unlock(wait_list_lock);
}
 
*pend = requested_offset_record;
@@ -3237,7 +3236,7 @@ static int _hl_interrupt_wait_ioctl(struct hl_device 
*hdev, struct hl_ctx *ctx,
struct hl_user_pending_interrupt *pend;
struct hl_mmap_mem_buf *buf;
struct hl_cb *cq_cb;
-   unsigned long timeout, flags;
+   unsigned long timeout;
long completion_rc;
int rc = 0;
 
@@ -3284,7 +3283,7 @@ static int _hl_interrupt_wait_ioctl(struct hl_device 
*hdev, struct hl_ctx *ctx,
pend->cq_target_value = target_value;
}
 
-   spin_lock_irqsave(>wait_list_lock, flags);
+   spin_lock(>wait_list_lock);
 
/* We check for completion value as interrupt could have been received
 * before we