date:20200130



On 2020-01-30 17:11, Alex Deucher wrote:

On Thu, Jan 30, 2020 at 4:55 PM Felix Kuehling  wrote:

On 2020-01-30 14:01, Bhardwaj, Rajneesh wrote:

Hello Felix,

Thanks for your time to review and for your feedback.

On 1/29/2020 5:52 PM, Felix Kuehling wrote:

HI Rajneesh,

See comments inline ...

And a general question: Why do you need to set the autosuspend_delay
in so many places? Amdgpu only has a single call to this function
during initialization.


We don't. I have called this out in cover letter since i too feel its
not the best way to do what we want to do. I have seen weird issues on
dual GPUs with KFDTest that sometimes results in system hang and gpu
hang etc.

Even if i try with running kfdtest on just one node in a multi gpu
system, the baco exit seems to wake up all gpus and then one goes to
auto suspend again while other performs the desired kfdtest operation.
I have never seen any issue with a single gpu card. So in current
approch, i am making sure that the GPUs are not quickly auto-suspended
while the kfd operations are ongoing but once the kfdtest is finished,
the runtime suspend call with ensure to reset the timeout back to 5
seconds.

So by setting the timeout to 60s, you can fix applications that run for
less than 60s without calling into KFD. What about applications that run
for longer without calling into KFD? This doesn't sound like a solution,
it's just a hack.




I would like to avoid this and appreciate some pointers where i can
put get_sync calls while unbinding. I have tried a number of places
but saw some issues. So any help will be highly appreciated, to
identify the proper logical inverse of kfd_bind_process_to_device.

kfd_bind_process_to_device is called when KFD starts using a device. It
only stops using that device when the process is destroyed. Therefore I
recommended moving the cleanup code into kfd_process_destroy_pdds, which
iterates over all devices (PDDs) during process termination.

Note that kfd_bind_process_to_device can be called many times for the
same device. If you need to keep usage counters balanced, you need extra
checks to make sure you only do runtime-PM stuff the first time it's
called for a particular device in a particular process.

Also, in kfd_process_destroy_pdds you may need to check whether
kfd_bind_process_to_device has been called for a particular PDD before
releasing runtime PM.

runtimepm already has reference counting.  You can use something like
pm_runtime_get_sync() or pm_runtime_get_noresume() depending on
whether you want to make sure the GPU is powered up or not.  That will
increase the usage count on the device for runtime pm.  Then when you
want to decrement and possibly suspend the device, call
pm_runtime_mark_last_busy() and pm_runtime_put_autosuspend().  That is
basically what we do in amdgpu to handle device usage.


My point is, kfd_bind_process_to_device can be called many times. If we 
call pm_runtime_sync_get, we'll need to release an unpredictable number 
of usage counts later. Therefore I suggested ensuring we only do this 
the first time kfd_bind_process_to_device is called for a particular 
process-device.


Similarly, in kfd_process_destroy_pdds we should only release the usage 
count if we actually took it in kfd_bind_process_to_device.




   We call
pm_runtime_get_noresume() in our fence_emit function which is what
gets called when we submit work to the GPU.  That increments the usage
count for runpm.  Then we call pm_runtime_mark_last_busy() and
pm_runtime_put_autosuspend() in our fence_process function.  That gets
called as each job on the GPU is retired.  For kfd, I think you'll
want to call pm_runtime_get_sync() in at the start of your ioctl to
wake the device, then pm_runtime_mark_last_busy() and
pm_runtime_put_autosuspend() at the end of your ioctl to mark to match
the start.  Then in your actual ioctl work, you'll want to call
pm_runtime_get_noresume() everytime a queue gets created and
pm_runtime_mark_last_busy() and pm_runtime_put_autosuspend() each time
a queue is destroyed.  That should keep the device awake as long as
there are any queues around.  Due to stuff like xgmi, it's probably
best to just do this for all GPUs on the system just in case you have
any p2p stuff going on.


OK, that's a completely different approach from what I had in mind. 
Basically tying device usage to both ioctls and the existence of user 
mode queues. P2P adds extra complication. Not just XGMI but also PCIe 
P2P where GPU A wants to access GPU B's memory but GPU B doesn't have 
any activity to keep it awake. To simplify this, we could also associate 
usage counters with allocating memory. That way any GPU that has memory 
that may be remotely accessed would be kept awake.


Always keeping all GPUs awake seems wasteful. I think being able to 
power down unused GPUs would be a very useful feature.


I still believe my idea of taking a usage counter in 
kfd_bind_process_to_device and dropping it in kfd_process_destroy_pdds 
is

RE: [PATCH] drm/amdkfd: Add queue information to sysfs

2020-01-30 Thread Kasiviswanathan, Harish

[AMD Official Use Only - Internal Distribution Only]

One minor comment.

-Original Message-
From: amd-gfx  On Behalf Of Amber Lin
Sent: Thursday, January 30, 2020 12:46 AM
To: amd-gfx@lists.freedesktop.org
Cc: Lin, Amber 
Subject: [PATCH] drm/amdkfd: Add queue information to sysfs

Provide compute queues information in sysfs under /sys/class/kfd/kfd/proc.
The format is /sys/class/kfd/kfd/proc//queues//XX where
XX are size, type, and gpuid three files to represent queue size, queue
type, and the GPU this queue uses.  folder and files underneath
are generated when a queue is created. They are removed when the queue is
destroyed.

Signed-off-by: Amber Lin 
---
 drivers/gpu/drm/amd/amdkfd/kfd_priv.h  |  9 ++
 drivers/gpu/drm/amd/amdkfd/kfd_process.c   | 99 ++
 .../gpu/drm/amd/amdkfd/kfd_process_queue_manager.c |  2 +
 3 files changed, 110 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h 
b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
index c0b0def..cb2d2d7 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_priv.h
@@ -503,6 +503,12 @@ struct queue {
struct kfd_process  *process;
struct kfd_dev  *device;
void *gws;
+
+   /* procfs */
+   struct kobject *kobj_qid;
+   struct attribute attr_size;
+   struct attribute attr_type;
+   struct attribute attr_gpuid;
 };
 
 /*
@@ -730,6 +736,7 @@ struct kfd_process {
 
/* Kobj for our procfs */
struct kobject *kobj;
+   struct kobject *kobj_queues;
struct attribute attr_pasid;
 };
 
@@ -836,6 +843,8 @@ extern struct device *kfd_device;
 /* KFD's procfs */
 void kfd_procfs_init(void);
 void kfd_procfs_shutdown(void);
+int kfd_procfs_add_queue(struct queue *q);
+void kfd_procfs_del_queue(struct queue *q);
 
 /* Topology */
 int kfd_topology_init(void);
diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_process.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
index 25b90f7..0220651 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_process.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_process.c
@@ -132,6 +132,97 @@ void kfd_procfs_shutdown(void)
}
 }
 
+static int kfd_procfs_add_file(const char *name, struct kobject *kobj,
+  struct attribute *attr)
+{
+   int ret;
+
+   attr->name = name;
+   attr->mode = KFD_SYSFS_FILE_MODE;
+   sysfs_attr_init(attr);
+   ret = sysfs_create_file(kobj, attr);
+   if (ret)
+   pr_warn("Creating %s file failed", name);
+   return ret;
+}
+
+static ssize_t kfd_procfs_queue_show(struct kobject *kobj,
+struct attribute *attr, char *buffer)
+{
+   if (!strcmp(attr->name, "size")) {
+   struct queue *q = container_of(attr, struct queue, attr_size);
+   return snprintf(buffer, PAGE_SIZE, "%llu",
+   q->properties.queue_size);
+   } else if (!strcmp(attr->name, "type")) {
+   struct queue *q = container_of(attr, struct queue, attr_type);
+   return snprintf(buffer, PAGE_SIZE, "%d", q->properties.type);
+   } else if (!strcmp(attr->name, "gpuid")) {
+   struct queue *q = container_of(attr, struct queue, attr_gpuid);
+   return snprintf(buffer, PAGE_SIZE, "%u", q->device->id);
+   } else
+   pr_err("Invalid attribute");
+
+   return 0;
+}
+
+static const struct sysfs_ops procfs_queue_ops = {
+   .show = kfd_procfs_queue_show,
+};
+
+static struct kobj_type procfs_queue_type = {
+   .release = kfd_procfs_kobj_release,
+   .sysfs_ops = _queue_ops,
+};
+
+int kfd_procfs_add_queue(struct queue *q)
+{
+   struct kfd_process *proc;
+   int ret;
+
+   if (!q || !q->process)
+   return -EINVAL;
+   proc = q->process;
+
+   /* Create proc//queues/ folder*/
+   if (!proc->kobj_queues)
+   return -EFAULT;
+   if (q->kobj_qid)
+   return -EEXIST;
+   q->kobj_qid = kfd_alloc_struct(q->kobj_qid);
+   if (!q->kobj_qid)
+   return -ENOMEM;
+   ret = kobject_init_and_add(q->kobj_qid, _queue_type,
+   proc->kobj_queues, "%u", q->properties.queue_id);
+   if (ret < 0) {
+   pr_warn("Creating proc//queues/%u failed",
+   q->properties.queue_id);
+   return ret;
+   }
+
+   /* Create proc//queues//XX files */
+   kfd_procfs_add_file("size", q->kobj_qid, >attr_size);
+   kfd_procfs_add_file("type", q->kobj_qid, >attr_type);
+   kfd_procfs_add_file("gpuid", q->kobj_qid, >attr_gpuid);
+
+   return 0;
+}
+
+void kfd_procfs_del_queue(struct queue *q)
+{
+   struct kfd_process *proc;

[HK] : proc is not used

+
+   if (!q || !q->process)
+   return;
+   proc = q->process;
+
+   sysfs_remove_file(q->kobj_qid, >attr_size);
+   sysfs_remove_file(q->kobj_qid,

Re: [PATCH] drm/amdkfd: Fix a bug in SDMA RLC queue counting under HWS mode

2020-01-30 Thread Yong Zhao


True. It is a bug too. I am looking into it.

Yong

On 2020-01-30 5:51 p.m., Felix Kuehling wrote:

On 2020-01-30 17:29, Yong Zhao wrote:

The sdma_queue_count increment should be done before
execute_queues_cpsch(), which calls pm_calc_rlib_size() where
sdma_queue_count is used to calculate whether over_subscription is
triggered.

With the previous code, when a SDMA queue is created,
compute_queue_count in pm_calc_rlib_size() is one more than the
actual compute queue number, because the queue_count has been
incremented while sdma_queue_count has not. This patch fixes that.

Change-Id: I20353e657efd505353d0dd9f7eb2fab5085e7202
Signed-off-by: Yong Zhao 


Reviewed-by: Felix Kuehling 

But I took a look at pm_calc_rlib_size. I don't think subtracting 
dqm->sdma_queue_count from dqm->queue_count is not quite correct, 
because sdma_queue_count counts all SDMA queues, while queue_count 
only counts active queues. So an application that creates inactive 
SDMA queues will also create errors here. We probably need to count 
active compute and active SDMA queues separately in DQM to fix this 
properly.


Regards,
  Felix



---
  drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 10 ++
  1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c

index 2870553a2ce0..80d22bf702e8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1237,16 +1237,18 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,

    list_add(>list, >queues_list);
  qpd->queue_count++;
+
+    if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
+    dqm->sdma_queue_count++;
+    else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
+    dqm->xgmi_sdma_queue_count++;
+
  if (q->properties.is_active) {
  dqm->queue_count++;
  retval = execute_queues_cpsch(dqm,
  KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0);
  }
  -    if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
-    dqm->sdma_queue_count++;
-    else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
-    dqm->xgmi_sdma_queue_count++;
  /*
   * Unconditionally increment this counter, regardless of the 
queue's

   * type or whether the queue is active.

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdkfd: Fix a bug in SDMA RLC queue counting under HWS mode


On 2020-01-30 17:29, Yong Zhao wrote:

The sdma_queue_count increment should be done before
execute_queues_cpsch(), which calls pm_calc_rlib_size() where
sdma_queue_count is used to calculate whether over_subscription is
triggered.

With the previous code, when a SDMA queue is created,
compute_queue_count in pm_calc_rlib_size() is one more than the
actual compute queue number, because the queue_count has been
incremented while sdma_queue_count has not. This patch fixes that.

Change-Id: I20353e657efd505353d0dd9f7eb2fab5085e7202
Signed-off-by: Yong Zhao 


Reviewed-by: Felix Kuehling 

But I took a look at pm_calc_rlib_size. I don't think subtracting 
dqm->sdma_queue_count from dqm->queue_count is not quite correct, 
because sdma_queue_count counts all SDMA queues, while queue_count only 
counts active queues. So an application that creates inactive SDMA 
queues will also create errors here. We probably need to count active 
compute and active SDMA queues separately in DQM to fix this properly.


Regards,
  Felix



---
  drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 10 ++
  1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 2870553a2ce0..80d22bf702e8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1237,16 +1237,18 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
  
  	list_add(>list, >queues_list);

qpd->queue_count++;
+
+   if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
+   dqm->sdma_queue_count++;
+   else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
+   dqm->xgmi_sdma_queue_count++;
+
if (q->properties.is_active) {
dqm->queue_count++;
retval = execute_queues_cpsch(dqm,
KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0);
}
  
-	if (q->properties.type == KFD_QUEUE_TYPE_SDMA)

-   dqm->sdma_queue_count++;
-   else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
-   dqm->xgmi_sdma_queue_count++;
/*
 * Unconditionally increment this counter, regardless of the queue's
 * type or whether the queue is active.

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 5/5] drm/amdgpu: rework synchronization of VM updates v4


On 2020-01-30 7:49, Christian König wrote:

If provided we only sync to the BOs reservation
object and no longer to the root PD.

v2: update comment, cleanup amdgpu_bo_sync_wait_resv
v3: use correct reservation object while clearing
v4: fix typo in amdgpu_bo_sync_wait_resv

Signed-off-by: Christian König 
Tested-by: Tom St Denis 


Reviewed-by: Felix Kuehling 



---
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.c  | 35 ++
  drivers/gpu/drm/amd/amdgpu/amdgpu_object.h  |  3 ++
  drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c|  7 
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c  | 41 -
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h  |  4 +-
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c  | 22 +++
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 15 ++--
  7 files changed, 66 insertions(+), 61 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 46c76e2e1281..6f60a581e3ba 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -1403,30 +1403,51 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct 
dma_fence *fence,
  }
  
  /**

- * amdgpu_sync_wait_resv - Wait for BO reservation fences
+ * amdgpu_bo_sync_wait_resv - Wait for BO reservation fences
   *
- * @bo: buffer object
+ * @adev: amdgpu device pointer
+ * @resv: reservation object to sync to
+ * @sync_mode: synchronization mode
   * @owner: fence owner
   * @intr: Whether the wait is interruptible
   *
+ * Extract the fences from the reservation object and waits for them to finish.
+ *
   * Returns:
   * 0 on success, errno otherwise.
   */
-int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr)
+int amdgpu_bo_sync_wait_resv(struct amdgpu_device *adev, struct dma_resv *resv,
+enum amdgpu_sync_mode sync_mode, void *owner,
+bool intr)
  {
-   struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
struct amdgpu_sync sync;
int r;
  
  	amdgpu_sync_create();

-   amdgpu_sync_resv(adev, , bo->tbo.base.resv,
-AMDGPU_SYNC_NE_OWNER, owner);
+   amdgpu_sync_resv(adev, , resv, sync_mode, owner);
r = amdgpu_sync_wait(, intr);
amdgpu_sync_free();
-
return r;
  }
  
+/**

+ * amdgpu_bo_sync_wait - Wrapper for amdgpu_bo_sync_wait_resv
+ * @bo: buffer object to wait for
+ * @owner: fence owner
+ * @intr: Whether the wait is interruptible
+ *
+ * Wrapper to wait for fences in a BO.
+ * Returns:
+ * 0 on success, errno otherwise.
+ */
+int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr)
+{
+   struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
+
+   return amdgpu_bo_sync_wait_resv(adev, bo->tbo.base.resv,
+   AMDGPU_SYNC_NE_OWNER, owner, intr);
+}
+
  /**
   * amdgpu_bo_gpu_offset - return GPU offset of bo
   * @bo:   amdgpu object for which we query the offset
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
index 2eeafc77c9c1..96d805889e8d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
@@ -283,6 +283,9 @@ void amdgpu_bo_release_notify(struct ttm_buffer_object *bo);
  int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo);
  void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence,
 bool shared);
+int amdgpu_bo_sync_wait_resv(struct amdgpu_device *adev, struct dma_resv *resv,
+enum amdgpu_sync_mode sync_mode, void *owner,
+bool intr);
  int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr);
  u64 amdgpu_bo_gpu_offset(struct amdgpu_bo *bo);
  int amdgpu_bo_validate(struct amdgpu_bo *bo);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index 9f42032676da..b86392253696 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -249,13 +249,6 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct 
amdgpu_sync *sync,
owner != AMDGPU_FENCE_OWNER_UNDEFINED)
continue;
  
-		/* VM updates only sync with moves but not with user

-* command submissions or KFD evictions fences
-*/
-   if (fence_owner != AMDGPU_FENCE_OWNER_UNDEFINED &&
-   owner == AMDGPU_FENCE_OWNER_VM)
-   continue;
-
/* Ignore fences depending on the sync mode */
switch (mode) {
case AMDGPU_SYNC_ALWAYS:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 6038b3c89633..aaae2b5e6eee 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -797,7 +797,7 @@

Re: [PATCH 4/5] drm/amdgpu: simplify and fix amdgpu_sync_resv


On 2020-01-30 7:49, Christian König wrote:

No matter what we always need to sync to moves.

Signed-off-by: Christian König 
Tested-by: Tom St Denis 


Reviewed-by: Felix Kuehling 



---
  drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c | 15 +++
  1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index c124f64e7aae..9f42032676da 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -232,10 +232,19 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct 
amdgpu_sync *sync,
  
  		f = rcu_dereference_protected(flist->shared[i],

  dma_resv_held(resv));
+
+   fence_owner = amdgpu_sync_get_owner(f);
+
+   /* Always sync to moves, no matter what */
+   if (fence_owner == AMDGPU_FENCE_OWNER_UNDEFINED) {
+   r = amdgpu_sync_fence(sync, f, false);
+   if (r)
+   break;
+   }
+
/* We only want to trigger KFD eviction fences on
 * evict or move jobs. Skip KFD fences otherwise.
 */
-   fence_owner = amdgpu_sync_get_owner(f);
if (fence_owner == AMDGPU_FENCE_OWNER_KFD &&
owner != AMDGPU_FENCE_OWNER_UNDEFINED)
continue;
@@ -265,9 +274,7 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct 
amdgpu_sync *sync,
break;
  
  		case AMDGPU_SYNC_EXPLICIT:

-   if (owner != AMDGPU_FENCE_OWNER_UNDEFINED)
-   continue;
-   break;
+   continue;
}
  
  		r = amdgpu_sync_fence(sync, f, false);

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdkfd: Fix a bug in SDMA RLC queue counting under HWS mode

2020-01-30 Thread Yong Zhao

The sdma_queue_count increment should be done before
execute_queues_cpsch(), which calls pm_calc_rlib_size() where
sdma_queue_count is used to calculate whether over_subscription is
triggered.

With the previous code, when a SDMA queue is created,
compute_queue_count in pm_calc_rlib_size() is one more than the
actual compute queue number, because the queue_count has been
incremented while sdma_queue_count has not. This patch fixes that.

Change-Id: I20353e657efd505353d0dd9f7eb2fab5085e7202
Signed-off-by: Yong Zhao 
---
 drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c | 10 ++
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
index 2870553a2ce0..80d22bf702e8 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_device_queue_manager.c
@@ -1237,16 +1237,18 @@ static int create_queue_cpsch(struct 
device_queue_manager *dqm, struct queue *q,
 
list_add(>list, >queues_list);
qpd->queue_count++;
+
+   if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
+   dqm->sdma_queue_count++;
+   else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
+   dqm->xgmi_sdma_queue_count++;
+
if (q->properties.is_active) {
dqm->queue_count++;
retval = execute_queues_cpsch(dqm,
KFD_UNMAP_QUEUES_FILTER_DYNAMIC_QUEUES, 0);
}
 
-   if (q->properties.type == KFD_QUEUE_TYPE_SDMA)
-   dqm->sdma_queue_count++;
-   else if (q->properties.type == KFD_QUEUE_TYPE_SDMA_XGMI)
-   dqm->xgmi_sdma_queue_count++;
/*
 * Unconditionally increment this counter, regardless of the queue's
 * type or whether the queue is active.
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 2/5] drm/amdgpu: return EINVAL instead of ENOENT in the VM code


On 2020-01-30 7:49, Christian König wrote:

That we can't find a PD above the root is expected can only happen if
we try to update a larger range than actually managed by the VM.

Signed-off-by: Christian König 
Tested-by: Tom St Denis 

Reviewed-by: Felix Kuehling 

---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 4ba6a5e5d094..9705c961405b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1475,7 +1475,7 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
 * shift we should go up one level and check it again.
 */
if (!amdgpu_vm_pt_ancestor())
-   return -ENOENT;
+   return -EINVAL;
continue;
}
  

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH 3/5] drm/amdgpu: allow higher level PD invalidations


On 2020-01-30 7:49, Christian König wrote:

Allow partial invalidation on unallocated PDs. This is useful when we
need to silence faults to stop interrupt floods on Vega.

Signed-off-by: Christian König 
Tested-by: Tom St Denis 


I already reviewed this a week ago. With two style nit-picks fixed, this 
patch is


Reviewed-by: Felix Kuehling 



---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 23 ++-
  1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 9705c961405b..6038b3c89633 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1467,9 +1467,8 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
 * smaller than the address shift. Go to the next
 * child entry and try again.
 */
-   if (!amdgpu_vm_pt_descendant(adev, ))
-   return -ENOENT;
-   continue;
+   if (amdgpu_vm_pt_descendant(adev, ))
+   continue;
} else if (frag >= parent_shift) {
/* If the fragment size is even larger than the parent
 * shift we should go up one level and check it again.
@@ -1480,8 +1479,19 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
}
  
  		pt = cursor.entry->base.bo;

-   if (!pt)
-   return -ENOENT;
+   if (!pt) {
+   /* We need all PDs and PTs for mapping something, */
+   if (flags & AMDGPU_PTE_VALID)
+   return -ENOENT;
+
+   /* but unmapping something can happen at a higher
+* level. */


checkpatch.pl complains about multi-line comments with the */ not on its 
own line.




+   if (!amdgpu_vm_pt_ancestor())
+   return -EINVAL;
+
+   pt = cursor.entry->base.bo;
+   shift = parent_shift;
+   }
  
  		/* Looks good so far, calculate parameters for the update */

incr = (uint64_t)AMDGPU_GPU_PAGE_SIZE << shift;
@@ -1495,6 +1505,9 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
uint64_t upd_end = min(entry_end, frag_end);
unsigned nptes = (upd_end - frag_start) >> shift;
  
+			/* This can happen when we set higher level PDs to

+* silent to stop fault floods. */


Same as above.



+   nptes = max(nptes, 1u);
amdgpu_vm_update_flags(params, pt, cursor.level,
   pe_start, dst, nptes, incr,
   flags | AMDGPU_PTE_FRAG(frag));

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [Patch v1 5/5] drm/amdkfd: refactor runtime pm for baco

2020-01-30 Thread Alex Deucher

On Thu, Jan 30, 2020 at 4:55 PM Felix Kuehling  wrote:
>
> On 2020-01-30 14:01, Bhardwaj, Rajneesh wrote:
> > Hello Felix,
> >
> > Thanks for your time to review and for your feedback.
> >
> > On 1/29/2020 5:52 PM, Felix Kuehling wrote:
> >> HI Rajneesh,
> >>
> >> See comments inline ...
> >>
> >> And a general question: Why do you need to set the autosuspend_delay
> >> in so many places? Amdgpu only has a single call to this function
> >> during initialization.
> >
> >
> > We don't. I have called this out in cover letter since i too feel its
> > not the best way to do what we want to do. I have seen weird issues on
> > dual GPUs with KFDTest that sometimes results in system hang and gpu
> > hang etc.
> >
> > Even if i try with running kfdtest on just one node in a multi gpu
> > system, the baco exit seems to wake up all gpus and then one goes to
> > auto suspend again while other performs the desired kfdtest operation.
> > I have never seen any issue with a single gpu card. So in current
> > approch, i am making sure that the GPUs are not quickly auto-suspended
> > while the kfd operations are ongoing but once the kfdtest is finished,
> > the runtime suspend call with ensure to reset the timeout back to 5
> > seconds.
>
> So by setting the timeout to 60s, you can fix applications that run for
> less than 60s without calling into KFD. What about applications that run
> for longer without calling into KFD? This doesn't sound like a solution,
> it's just a hack.
>
>
> >
> >
> > I would like to avoid this and appreciate some pointers where i can
> > put get_sync calls while unbinding. I have tried a number of places
> > but saw some issues. So any help will be highly appreciated, to
> > identify the proper logical inverse of kfd_bind_process_to_device.
>
> kfd_bind_process_to_device is called when KFD starts using a device. It
> only stops using that device when the process is destroyed. Therefore I
> recommended moving the cleanup code into kfd_process_destroy_pdds, which
> iterates over all devices (PDDs) during process termination.
>
> Note that kfd_bind_process_to_device can be called many times for the
> same device. If you need to keep usage counters balanced, you need extra
> checks to make sure you only do runtime-PM stuff the first time it's
> called for a particular device in a particular process.
>
> Also, in kfd_process_destroy_pdds you may need to check whether
> kfd_bind_process_to_device has been called for a particular PDD before
> releasing runtime PM.

runtimepm already has reference counting.  You can use something like
pm_runtime_get_sync() or pm_runtime_get_noresume() depending on
whether you want to make sure the GPU is powered up or not.  That will
increase the usage count on the device for runtime pm.  Then when you
want to decrement and possibly suspend the device, call
pm_runtime_mark_last_busy() and pm_runtime_put_autosuspend().  That is
basically what we do in amdgpu to handle device usage.  We call
pm_runtime_get_noresume() in our fence_emit function which is what
gets called when we submit work to the GPU.  That increments the usage
count for runpm.  Then we call pm_runtime_mark_last_busy() and
pm_runtime_put_autosuspend() in our fence_process function.  That gets
called as each job on the GPU is retired.  For kfd, I think you'll
want to call pm_runtime_get_sync() in at the start of your ioctl to
wake the device, then pm_runtime_mark_last_busy() and
pm_runtime_put_autosuspend() at the end of your ioctl to mark to match
the start.  Then in your actual ioctl work, you'll want to call
pm_runtime_get_noresume() everytime a queue gets created and
pm_runtime_mark_last_busy() and pm_runtime_put_autosuspend() each time
a queue is destroyed.  That should keep the device awake as long as
there are any queues around.  Due to stuff like xgmi, it's probably
best to just do this for all GPUs on the system just in case you have
any p2p stuff going on.

Alex

>
> Regards,
>Felix
>
>
> >
> >
> >>
> >> On 2020-01-27 20:29, Rajneesh Bhardwaj wrote:
> >>> So far the kfd driver implemented same routines for runtime and system
> >>> wide suspend and resume (s2idle or mem). During system wide suspend the
> >>> kfd aquires an atomic lock that prevents any more user processes to
> >>> create queues and interact with kfd driver and amd gpu. This mechanism
> >>> created problem when amdgpu device is runtime suspended with BACO
> >>> enabled. Any application that relies on kfd driver fails to load
> >>> because
> >>> the driver reports a locked kfd device since gpu is runtime suspended.
> >>>
> >>> However, in an ideal case, when gpu is runtime  suspended the kfd
> >>> driver
> >>> should be able to:
> >>>
> >>>   - auto resume amdgpu driver whenever a client requests compute
> >>> service
> >>>   - prevent runtime suspend for amdgpu  while kfd is in use
> >>>
> >>> This change refactors the amdgpu and amdkfd drivers to support BACO and
> >>> runtime runtime power management.
> >>>
> >>> Signed-off-by:

Re: [PATCH 1/5] drm/amdgpu: fix braces in amdgpu_vm_update_ptes



On 2020-01-30 7:49, Christian König wrote:

For the root PD mask can be 0x as well which would
overrun to 0 if we don't cast it before we add one.

You're fixing parentheses, not braces.

Parentheses: ()
Brackets: []
Braces: {}

With the title fixed, this patch is

Reviewed-by: Felix Kuehling 



Signed-off-by: Christian König 
Tested-by: Tom St Denis 
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 5cb182231f5d..4ba6a5e5d094 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1487,7 +1487,7 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
incr = (uint64_t)AMDGPU_GPU_PAGE_SIZE << shift;
mask = amdgpu_vm_entries_mask(adev, cursor.level);
pe_start = ((cursor.pfn >> shift) & mask) * 8;
-   entry_end = (uint64_t)(mask + 1) << shift;
+   entry_end = ((uint64_t)mask + 1) << shift;
entry_end += cursor.pfn & ~(entry_end - 1);
entry_end = min(entry_end, end);
  

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [Patch v1 5/5] drm/amdkfd: refactor runtime pm for baco


On 2020-01-30 14:01, Bhardwaj, Rajneesh wrote:

Hello Felix,

Thanks for your time to review and for your feedback.

On 1/29/2020 5:52 PM, Felix Kuehling wrote:

HI Rajneesh,

See comments inline ...

And a general question: Why do you need to set the autosuspend_delay 
in so many places? Amdgpu only has a single call to this function 
during initialization.



We don't. I have called this out in cover letter since i too feel its 
not the best way to do what we want to do. I have seen weird issues on 
dual GPUs with KFDTest that sometimes results in system hang and gpu 
hang etc.


Even if i try with running kfdtest on just one node in a multi gpu 
system, the baco exit seems to wake up all gpus and then one goes to 
auto suspend again while other performs the desired kfdtest operation. 
I have never seen any issue with a single gpu card. So in current 
approch, i am making sure that the GPUs are not quickly auto-suspended 
while the kfd operations are ongoing but once the kfdtest is finished, 
the runtime suspend call with ensure to reset the timeout back to 5 
seconds.


So by setting the timeout to 60s, you can fix applications that run for 
less than 60s without calling into KFD. What about applications that run 
for longer without calling into KFD? This doesn't sound like a solution, 
it's just a hack.






I would like to avoid this and appreciate some pointers where i can 
put get_sync calls while unbinding. I have tried a number of places 
but saw some issues. So any help will be highly appreciated, to 
identify the proper logical inverse of kfd_bind_process_to_device.


kfd_bind_process_to_device is called when KFD starts using a device. It 
only stops using that device when the process is destroyed. Therefore I 
recommended moving the cleanup code into kfd_process_destroy_pdds, which 
iterates over all devices (PDDs) during process termination.


Note that kfd_bind_process_to_device can be called many times for the 
same device. If you need to keep usage counters balanced, you need extra 
checks to make sure you only do runtime-PM stuff the first time it's 
called for a particular device in a particular process.


Also, in kfd_process_destroy_pdds you may need to check whether 
kfd_bind_process_to_device has been called for a particular PDD before 
releasing runtime PM.


Regards,
  Felix







On 2020-01-27 20:29, Rajneesh Bhardwaj wrote:

So far the kfd driver implemented same routines for runtime and system
wide suspend and resume (s2idle or mem). During system wide suspend the
kfd aquires an atomic lock that prevents any more user processes to
create queues and interact with kfd driver and amd gpu. This mechanism
created problem when amdgpu device is runtime suspended with BACO
enabled. Any application that relies on kfd driver fails to load 
because

the driver reports a locked kfd device since gpu is runtime suspended.

However, in an ideal case, when gpu is runtime  suspended the kfd 
driver

should be able to:

  - auto resume amdgpu driver whenever a client requests compute 
service

  - prevent runtime suspend for amdgpu  while kfd is in use

This change refactors the amdgpu and amdkfd drivers to support BACO and
runtime runtime power management.

Signed-off-by: Rajneesh Bhardwaj 
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 12 -
  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h |  8 +++---
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  4 +--
  drivers/gpu/drm/amd/amdkfd/kfd_device.c    | 31 
+-

  drivers/gpu/drm/amd/amdkfd/kfd_iommu.c |  5 +++-
  drivers/gpu/drm/amd/amdkfd/kfd_process.c   | 19 ++---
  6 files changed, 51 insertions(+), 28 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c

index 8609287620ea..314c4a2a0354 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -178,18 +178,18 @@ void amdgpu_amdkfd_interrupt(struct 
amdgpu_device *adev,

  kgd2kfd_interrupt(adev->kfd.dev, ih_ring_entry);
  }
  -void amdgpu_amdkfd_suspend(struct amdgpu_device *adev)
+void amdgpu_amdkfd_suspend(struct amdgpu_device *adev, bool run_pm)
  {
  if (adev->kfd.dev)
-    kgd2kfd_suspend(adev->kfd.dev);
+    kgd2kfd_suspend(adev->kfd.dev, run_pm);
  }
  -int amdgpu_amdkfd_resume(struct amdgpu_device *adev)
+int amdgpu_amdkfd_resume(struct amdgpu_device *adev, bool run_pm)
  {
  int r = 0;
    if (adev->kfd.dev)
-    r = kgd2kfd_resume(adev->kfd.dev);
+    r = kgd2kfd_resume(adev->kfd.dev, run_pm);
    return r;
  }
@@ -713,11 +713,11 @@ void kgd2kfd_exit(void)
  {
  }
  -void kgd2kfd_suspend(struct kfd_dev *kfd)
+void kgd2kfd_suspend(struct kfd_dev *kfd, bool run_pm)
  {
  }
  -int kgd2kfd_resume(struct kfd_dev *kfd)
+int kgd2kfd_resume(struct kfd_dev *kfd, bool run_pm)
  {
  return 0;
  }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h

Re: [PATCH 3/3] drm/amdgpu/smu_v11_0: Correct behavior of restoring default tables (v2)

2020-01-30 Thread Matt Coffin

It's worth noting here that I don't have a vega20 card to test with, so
it might be prudent to get a Tested-by from someone that has access to one.

It *should* work since it's so similar to the navi10 code, but it is
moderately un-tested.

On 1/29/20 11:17 AM, Alex Deucher wrote:
> From: Matt Coffin 
> 
> Previously, the syfs functionality for restoring the default powerplay
> table was sourcing it's information from the currently-staged powerplay
> table.
> 
> This patch adds a step to cache the first overdrive table that we see on
> boot, so that it can be used later to "restore" the powerplay table
> 
> v2: sqaush my original with Matt's fix
> 
> Bug: https://gitlab.freedesktop.org/drm/amd/issues/1020
> Signed-off-by: Matt Coffin 
> Reviewed-by: Alex Deucher 
> Signed-off-by: Alex Deucher 
> ---
>  .../gpu/drm/amd/powerplay/inc/amdgpu_smu.h|  1 +
>  drivers/gpu/drm/amd/powerplay/navi10_ppt.c|  7 +
>  drivers/gpu/drm/amd/powerplay/smu_v11_0.c |  6 
>  drivers/gpu/drm/amd/powerplay/vega20_ppt.c| 28 ++-
>  4 files changed, 22 insertions(+), 20 deletions(-)
> 
> diff --git a/drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h 
> b/drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h
> index b0591a8dda41..1e33c3e9b98d 100644
> --- a/drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h
> +++ b/drivers/gpu/drm/amd/powerplay/inc/amdgpu_smu.h
> @@ -273,6 +273,7 @@ struct smu_table_context
>   uint8_t thermal_controller_type;
>  
>   void*overdrive_table;
> + void*boot_overdrive_table;
>  };
>  
>  struct smu_dpm_context {
> diff --git a/drivers/gpu/drm/amd/powerplay/navi10_ppt.c 
> b/drivers/gpu/drm/amd/powerplay/navi10_ppt.c
> index d2d45181ae23..26cfccc57331 100644
> --- a/drivers/gpu/drm/amd/powerplay/navi10_ppt.c
> +++ b/drivers/gpu/drm/amd/powerplay/navi10_ppt.c
> @@ -2063,6 +2063,13 @@ static int navi10_od_edit_dpm_table(struct smu_context 
> *smu, enum PP_OD_DPM_TABL
>   return ret;
>   od_table->UclkFmax = input[1];
>   break;
> + case PP_OD_RESTORE_DEFAULT_TABLE:
> + if (!(table_context->overdrive_table && 
> table_context->boot_overdrive_table)) {
> + pr_err("Overdrive table was not initialized!\n");
> + return -EINVAL;
> + }
> + memcpy(table_context->overdrive_table, 
> table_context->boot_overdrive_table, sizeof(OverDriveTable_t));
> + break;
>   case PP_OD_COMMIT_DPM_TABLE:
>   navi10_dump_od_table(od_table);
>   ret = smu_update_table(smu, SMU_TABLE_OVERDRIVE, 0, (void 
> *)od_table, true);
> diff --git a/drivers/gpu/drm/amd/powerplay/smu_v11_0.c 
> b/drivers/gpu/drm/amd/powerplay/smu_v11_0.c
> index 02f8c9cb89d9..0dc49479a7eb 100644
> --- a/drivers/gpu/drm/amd/powerplay/smu_v11_0.c
> +++ b/drivers/gpu/drm/amd/powerplay/smu_v11_0.c
> @@ -1882,6 +1882,12 @@ int smu_v11_0_set_default_od_settings(struct 
> smu_context *smu, bool initialize,
>   pr_err("Failed to export overdrive table!\n");
>   return ret;
>   }
> + if (!table_context->boot_overdrive_table) {
> + table_context->boot_overdrive_table = 
> kmemdup(table_context->overdrive_table, overdrive_table_size, GFP_KERNEL);
> + if (!table_context->boot_overdrive_table) {
> + return -ENOMEM;
> + }
> + }
>   }
>   ret = smu_update_table(smu, SMU_TABLE_OVERDRIVE, 0, 
> table_context->overdrive_table, true);
>   if (ret) {
> diff --git a/drivers/gpu/drm/amd/powerplay/vega20_ppt.c 
> b/drivers/gpu/drm/amd/powerplay/vega20_ppt.c
> index 38febd5ca4da..4ad8d6c14ee5 100644
> --- a/drivers/gpu/drm/amd/powerplay/vega20_ppt.c
> +++ b/drivers/gpu/drm/amd/powerplay/vega20_ppt.c
> @@ -1706,22 +1706,11 @@ static int vega20_set_default_od_settings(struct 
> smu_context *smu,
>   struct smu_table_context *table_context = >smu_table;
>   int ret;
>  
> - if (initialize) {
> - if (table_context->overdrive_table)
> - return -EINVAL;
> -
> - table_context->overdrive_table = 
> kzalloc(sizeof(OverDriveTable_t), GFP_KERNEL);
> -
> - if (!table_context->overdrive_table)
> - return -ENOMEM;
> -
> - ret = smu_update_table(smu, SMU_TABLE_OVERDRIVE, 0,
> -table_context->overdrive_table, false);
> - if (ret) {
> - pr_err("Failed to export over drive table!\n");
> - return ret;
> - }
> + ret = smu_v11_0_set_default_od_settings(smu, initialize, 
> sizeof(OverDriveTable_t));
> + if (ret)
> + return ret;
>  
> + if (initialize) {
>   ret = vega20_set_default_od8_setttings(smu);
>   if (ret)

Re: [PATCH 6/6] drm/amd/display: REFERENCE for srm interface patches

2020-01-30 Thread Harry Wentland

Thanks for providing more documentation and this reference.

The patch set (1-5) is
Reviewed-by: Harry Wentland 

Harry

On 2020-01-22 4:05 p.m., Bhawanpreet Lakha wrote:
> This is just a reference for the patches. not to be merged
> 
> Signed-off-by: Bhawanpreet Lakha 
> ---
>  REFERENCE | 49 +
>  1 file changed, 49 insertions(+)
>  create mode 100644 REFERENCE
> 
> diff --git a/REFERENCE b/REFERENCE
> new file mode 100644
> index ..2e53f9cc82ff
> --- /dev/null
> +++ b/REFERENCE
> @@ -0,0 +1,49 @@
> +SRM interface Reference Usermode scripts for ubuntu. 
> +These are just reference sciprts to facilitate the SRM interface for amdgpu.
> +
> ++-+
> +| Main script, this is called on boot/shutdown/suspend/resume , it calls the 
> sysfs to get/set the SRM |
> +| FILE: /home/amdgpu_hdcp_srm_script.sh  
>  |
> ++-+
> +#!/bin/bash
> +
> +SRMFILE="/home/SRM"
> +sudo cat "$SRMFILE" > /sys/class/drm/card0/device/hdcp_srm
> +sudo cat /sys/class/drm/card0/device/hdcp_srm > "$SRMFILE"
> +
> +
> +
> +
> +
> ++-+
> +| .service file, This is placed into /etc/systemd/system/ so it runs the 
> main script on boot/shutdown |
> +| FILE: /etc/systemd/system/amdgpu_hdc_srm_boot_shutdown.service 
>  |
> ++-+
> +[Unit]
> +Description=HDCP SRM boot and shutdown save/load
> +
> +[Service]
> +Type=simple
> +ExecStart=/home/amdgpu_hdcp_srm_script.sh
> +ExecStop=/home/amdgpu_hdcp_srm_script.sh
> +
> +[Install]
> +WantedBy=multi-user.target
> +
> +
> +
> ++-+
> +| To run the script on boot/start run
>   |
> ++-+
> +sudo systemctl start amdgpu_hdc_srm_boot_shutdown
> +sudo systemctl enable amdgpu_hdc_srm_boot_shutdown
> +
> +
> +
> ++-+
> +| To symlnk the files (adding to these directory will run the script on 
> suspend/resume|
> ++-+
> +sudo ln -s $SCRIPTFILE /lib/systemd/system-sleep/amdgpu_hdcp_srm
> +sudo ln -s $SCRIPTFILE /usr/lib/pm-utils/sleep.d/95amdgpu_hdcp_srm
> +
> +
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

RE: Suspecting corrupted VBIOS after update of AMDGPU on AMD7870

2020-01-30 Thread Liu, Zhan

Hi Jacob,

Thant you for your bug reporting.

I saw you attached xorg.log, which is great. Could you also grab dmesg.log via 
SSH?

Thanks,
Zhan


From: amd-gfx  On Behalf Of Jacob Hrbek
Sent: 2020/January/30, Thursday 12:18 PM
To: amd-gfx@lists.freedesktop.org
Subject: Suspecting corrupted VBIOS after update of AMDGPU on AMD7870

Hello,
I believe that system update that included amdgpu on debian testing (but i am 
on LFS) corrupted my VBIOS on AMD7870 (+- 4 hours after the update the GPU 
using AMDGPU/Radeon drivers resulted in no output).
i'm sending this email to inform about possible bug with my findings on 
https://gist.github.com/Kreyren/3e55e9a754e58956e1690e38b1888de7
 and i would appreciate any help in excluding VBIOS corruption from the 
diagnostics.
Thanks,
- Jacob Hrbek
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [Patch v1 5/5] drm/amdkfd: refactor runtime pm for baco




On 1/28/2020 3:09 PM, Zeng, Oak wrote:

[AMD Official Use Only - Internal Distribution Only]



Regards,
Oak

-Original Message-
From: amd-gfx  On Behalf Of Rajneesh 
Bhardwaj
Sent: Monday, January 27, 2020 8:29 PM
To: amd-gfx@lists.freedesktop.org
Cc: Deucher, Alexander ; Kuehling, Felix 
; Bhardwaj, Rajneesh 
Subject: [Patch v1 5/5] drm/amdkfd: refactor runtime pm for baco

So far the kfd driver implemented same routines for runtime and system wide 
suspend and resume (s2idle or mem). During system wide suspend the kfd aquires 
an atomic lock that prevents any more user processes to create queues and 
interact with kfd driver and amd gpu. This mechanism created problem when 
amdgpu device is runtime suspended with BACO enabled. Any application that 
relies on kfd driver fails to load because the driver reports a locked kfd 
device since gpu is runtime suspended.

However, in an ideal case, when gpu is runtime  suspended the kfd driver should 
be able to:

  - auto resume amdgpu driver whenever a client requests compute service
  - prevent runtime suspend for amdgpu  while kfd is in use

This change refactors the amdgpu and amdkfd drivers to support BACO and runtime 
runtime power management.
[Oak] two runtime above



Thanks, will fix it.




Signed-off-by: Rajneesh Bhardwaj 
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 12 -  
drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h |  8 +++---  
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  4 +--
  drivers/gpu/drm/amd/amdkfd/kfd_device.c| 31 +-
  drivers/gpu/drm/amd/amdkfd/kfd_iommu.c |  5 +++-
  drivers/gpu/drm/amd/amdkfd/kfd_process.c   | 19 ++---
  6 files changed, 51 insertions(+), 28 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
index 8609287620ea..314c4a2a0354 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -178,18 +178,18 @@ void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev,
kgd2kfd_interrupt(adev->kfd.dev, ih_ring_entry);  }
  
-void amdgpu_amdkfd_suspend(struct amdgpu_device *adev)

+void amdgpu_amdkfd_suspend(struct amdgpu_device *adev, bool run_pm)
[Oak] then name run_pm is a little bit confusing. Maybe system_wide_pm or 
none_runtime_pm?
  {
if (adev->kfd.dev)
-   kgd2kfd_suspend(adev->kfd.dev);
+   kgd2kfd_suspend(adev->kfd.dev, run_pm);
  }
  
-int amdgpu_amdkfd_resume(struct amdgpu_device *adev)

+int amdgpu_amdkfd_resume(struct amdgpu_device *adev, bool run_pm)
  {
int r = 0;
  
  	if (adev->kfd.dev)

-   r = kgd2kfd_resume(adev->kfd.dev);
+   r = kgd2kfd_resume(adev->kfd.dev, run_pm);
  
  	return r;

  }
@@ -713,11 +713,11 @@ void kgd2kfd_exit(void)  {  }
  
-void kgd2kfd_suspend(struct kfd_dev *kfd)

+void kgd2kfd_suspend(struct kfd_dev *kfd, bool run_pm)
  {
  }
  
-int kgd2kfd_resume(struct kfd_dev *kfd)

+int kgd2kfd_resume(struct kfd_dev *kfd, bool run_pm)
  {
return 0;
  }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
index 0fa898d30207..3dce4a51e522 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -123,8 +123,8 @@ struct amdkfd_process_info {  int amdgpu_amdkfd_init(void); 
 void amdgpu_amdkfd_fini(void);
  
-void amdgpu_amdkfd_suspend(struct amdgpu_device *adev); -int amdgpu_amdkfd_resume(struct amdgpu_device *adev);

+void amdgpu_amdkfd_suspend(struct amdgpu_device *adev, bool run_pm);
+int amdgpu_amdkfd_resume(struct amdgpu_device *adev, bool run_pm);
  void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev,
const void *ih_ring_entry);
  void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev); @@ -250,8 +250,8 
@@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
 struct drm_device *ddev,
 const struct kgd2kfd_shared_resources *gpu_resources); 
 void kgd2kfd_device_exit(struct kfd_dev *kfd); -void kgd2kfd_suspend(struct 
kfd_dev *kfd); -int kgd2kfd_resume(struct kfd_dev *kfd);
+void kgd2kfd_suspend(struct kfd_dev *kfd, bool run_pm); int
+kgd2kfd_resume(struct kfd_dev *kfd, bool run_pm);
  int kgd2kfd_pre_reset(struct kfd_dev *kfd);  int kgd2kfd_post_reset(struct 
kfd_dev *kfd);  void kgd2kfd_interrupt(struct kfd_dev *kfd, const void 
*ih_ring_entry); diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index d9e5eac182d3..787d49e9f4de 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -3314,7 +3314,7 @@ int amdgpu_device_suspend(struct drm_device *dev, bool 
fbcon)
}
}
  
-	amdgpu_amdkfd_suspend(adev);

+   amdgpu_amdkfd_suspend(adev, fbcon);
  
  	amdgpu_ras_suspend(adev);
  
@@ -3398,7 +3398,7 @@ int amdgpu_device_resume(struct drm_device

Re: [Patch v1 1/5] drm/amdgpu: always enable runtime power management




On 1/28/2020 3:14 PM, Alex Deucher wrote:

[CAUTION: External Email]

On Mon, Jan 27, 2020 at 8:30 PM Rajneesh Bhardwaj
 wrote:

This allows runtime power management to kick in on amdgpu driver when
the underlying hardware supports either BOCO or BACO. This can still be
avoided if boot arg amdgpu.runpm = 0 is supplied.

 BOCO: Bus Off, Chip Off
 BACO: Bus Alive, Chip Off

Signed-off-by: Rajneesh Bhardwaj 

This patch should be the last one in the series, otherwise we'll
enable runpm on BACO capable devices before the KFD code is in place.
Also, it's only supported on VI and newer asics, so we should use this
patch instead:
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Fpatchwork.freedesktop.org%2Fpatch%2F335402%2Fdata=02%7C01%7Crajneesh.bhardwaj%40amd.com%7C01f67fc720d3423ee6b908d7a42eb68d%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637158392852429503sdata=aU07GE56Vfb0JTSDsVDdyCdhxUkjHEVMAHiBaBC4V7g%3Dreserved=0

Alex


Thanks, Will fix in v2.



---
  drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c | 9 -
  1 file changed, 4 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
index 3a0ea9096498..7958d508486e 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c
@@ -169,11 +169,10 @@ int amdgpu_driver_load_kms(struct drm_device *dev, 
unsigned long flags)
 goto out;
 }

-   if (amdgpu_device_supports_boco(dev) &&
-   (amdgpu_runtime_pm != 0)) /* enable runpm by default */
-   adev->runpm = true;
-   else if (amdgpu_device_supports_baco(dev) &&
-(amdgpu_runtime_pm > 0)) /* enable runpm if runpm=1 */
+   /* always enable runtime power management except when amdgpu.runpm=0 */
+   if ((amdgpu_device_supports_boco(dev) ||
+   amdgpu_device_supports_baco(dev))
+   && (amdgpu_runtime_pm != 0))
 adev->runpm = true;

 /* Call ACPI methods: require modeset init
--
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfxdata=02%7C01%7Crajneesh.bhardwaj%40amd.com%7C01f67fc720d3423ee6b908d7a42eb68d%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637158392852429503sdata=%2BXwHkoDeyA9Q%2FwnSyaND6QOc1SxpGuAHkZ4JdaTM3wU%3Dreserved=0

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [Patch v1 3/5] drm/amdkfd: Introduce debugfs option to disable baco

Hi Alex

Thanks for your time and feedback!

On 1/28/2020 3:22 PM, Alex Deucher wrote:

[CAUTION: External Email]

On Mon, Jan 27, 2020 at 8:30 PM Rajneesh Bhardwaj
wrote:

When BACO is enabled by default, sometimes it can cause additional
trouble to debug KFD issues. This debugfs override allows to temporarily
disable BACO for debug purpose without having to reboot the machine.

However, in some cases one suspend-resume cycle might be needed if
the device is already runtime suspended.

e.g

sudo rtcwake -m < mem or freeze > -s 15

by triggering autosuspend event from user space, by doing something
like:

echo 6000 > /sys/bus/pci/devices/\:03\:00.0/power/autosuspend_delay_ms

Usage:

echo 0 > /sys/kernel/debug/kfd/enable_baco and run
cat /sys/kernel/debug/kfd/baco_status to verify whether BACO is
enabled or disabled by kfd driver.

It should be noted that while enabling baco again via kfd override, we
must do the following steps:

1. echo 0 > /sys/kernel/debug/kfd/enable_baco
2. sudo rtcwake -m < mem > -s 15

In this case, we need GPU to be fully reset which is done by BIOS. This
is not possible in case of S2idle i.e. freeze so we must use mem for
sleep.

I think we can drop this patch in favor of just using the standard
runtime pm control. E.g.,
/sys/class/drm/card0/device/power/control

Sure, i was using the /sys/bus/pci way to do it and found it was not
easy. Since this sysfs exists, will drop the patch.

Regards

Rajneesh

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://nam11.safelinks.protection.outlook.com/?url=https%3A%2F%2Flists.freedesktop.org%2Fmailman%2Flistinfo%2Famd-gfxdata=02%7C01%7Crajneesh.bhardwaj%40amd.com%7Cfdaaf630ee6548c6bd9108d7a42fe314%7C3dd8961fe4884e608e11a82d994e183d%7C0%7C0%7C637158397891190611sdata=3jE9jZbbw9IiCu7geMeCCsTC4u4tTdippeWYeSnX3oE%3Dreserved=0

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [Patch v1 4/5] drm/amdkfd: show warning when kfd is locked



On 1/28/2020 5:42 PM, Felix Kuehling wrote:

On 2020-01-27 20:29, Rajneesh Bhardwaj wrote:

During system suspend the kfd driver aquires a lock that prohibits
further kfd actions unless the gpu is resumed. This adds some info which
can be useful while debugging.

Signed-off-by: Rajneesh Bhardwaj 
---
  drivers/gpu/drm/amd/amdkfd/kfd_chardev.c | 2 ++
  1 file changed, 2 insertions(+)

diff --git a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c 
b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c

index 1aebda4bbbe7..081cc5f40d18 100644
--- a/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
+++ b/drivers/gpu/drm/amd/amdkfd/kfd_chardev.c
@@ -127,6 +127,8 @@ static int kfd_open(struct inode *inode, struct 
file *filep)

  return PTR_ERR(process);
    if (kfd_is_locked()) {
+    dev_warn(kfd_device, "kfd is locked!\n"
+    "process %d unreferenced", process->pasid);


If this is for debugging, make it dev_dbg. Printing warnings like this 
usually confuses people reporting completely unrelated problems that 
aren't even kernel problems at all. "Look, I found a warning in the 
kernel log. It must be a kernel problem."


Agree. Will fix in v2.




Regards,
  Felix



  kfd_unref_process(process);
  return -EAGAIN;
  }

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [Patch v1 5/5] drm/amdkfd: refactor runtime pm for baco


Hello Felix,

Thanks for your time to review and for your feedback.

On 1/29/2020 5:52 PM, Felix Kuehling wrote:

HI Rajneesh,

See comments inline ...

And a general question: Why do you need to set the autosuspend_delay 
in so many places? Amdgpu only has a single call to this function 
during initialization.



We don't. I have called this out in cover letter since i too feel its 
not the best way to do what we want to do. I have seen weird issues on 
dual GPUs with KFDTest that sometimes results in system hang and gpu 
hang etc.


Even if i try with running kfdtest on just one node in a multi gpu 
system, the baco exit seems to wake up all gpus and then one goes to 
auto suspend again while other performs the desired kfdtest operation. I 
have never seen any issue with a single gpu card. So in current approch, 
i am making sure that the GPUs are not quickly auto-suspended while the 
kfd operations are ongoing but once the kfdtest is finished, the runtime 
suspend call with ensure to reset the timeout back to 5 seconds.



I would like to avoid this and appreciate some pointers where i can put 
get_sync calls while unbinding. I have tried a number of places but saw 
some issues. So any help will be highly appreciated, to identify the 
proper logical inverse of kfd_bind_process_to_device.





On 2020-01-27 20:29, Rajneesh Bhardwaj wrote:

So far the kfd driver implemented same routines for runtime and system
wide suspend and resume (s2idle or mem). During system wide suspend the
kfd aquires an atomic lock that prevents any more user processes to
create queues and interact with kfd driver and amd gpu. This mechanism
created problem when amdgpu device is runtime suspended with BACO
enabled. Any application that relies on kfd driver fails to load because
the driver reports a locked kfd device since gpu is runtime suspended.

However, in an ideal case, when gpu is runtime  suspended the kfd driver
should be able to:

  - auto resume amdgpu driver whenever a client requests compute service
  - prevent runtime suspend for amdgpu  while kfd is in use

This change refactors the amdgpu and amdkfd drivers to support BACO and
runtime runtime power management.

Signed-off-by: Rajneesh Bhardwaj 
---
  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c | 12 -
  drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h |  8 +++---
  drivers/gpu/drm/amd/amdgpu/amdgpu_device.c |  4 +--
  drivers/gpu/drm/amd/amdkfd/kfd_device.c    | 31 +-
  drivers/gpu/drm/amd/amdkfd/kfd_iommu.c |  5 +++-
  drivers/gpu/drm/amd/amdkfd/kfd_process.c   | 19 ++---
  6 files changed, 51 insertions(+), 28 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c

index 8609287620ea..314c4a2a0354 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.c
@@ -178,18 +178,18 @@ void amdgpu_amdkfd_interrupt(struct 
amdgpu_device *adev,

  kgd2kfd_interrupt(adev->kfd.dev, ih_ring_entry);
  }
  -void amdgpu_amdkfd_suspend(struct amdgpu_device *adev)
+void amdgpu_amdkfd_suspend(struct amdgpu_device *adev, bool run_pm)
  {
  if (adev->kfd.dev)
-    kgd2kfd_suspend(adev->kfd.dev);
+    kgd2kfd_suspend(adev->kfd.dev, run_pm);
  }
  -int amdgpu_amdkfd_resume(struct amdgpu_device *adev)
+int amdgpu_amdkfd_resume(struct amdgpu_device *adev, bool run_pm)
  {
  int r = 0;
    if (adev->kfd.dev)
-    r = kgd2kfd_resume(adev->kfd.dev);
+    r = kgd2kfd_resume(adev->kfd.dev, run_pm);
    return r;
  }
@@ -713,11 +713,11 @@ void kgd2kfd_exit(void)
  {
  }
  -void kgd2kfd_suspend(struct kfd_dev *kfd)
+void kgd2kfd_suspend(struct kfd_dev *kfd, bool run_pm)
  {
  }
  -int kgd2kfd_resume(struct kfd_dev *kfd)
+int kgd2kfd_resume(struct kfd_dev *kfd, bool run_pm)
  {
  return 0;
  }
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h

index 0fa898d30207..3dce4a51e522 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_amdkfd.h
@@ -123,8 +123,8 @@ struct amdkfd_process_info {
  int amdgpu_amdkfd_init(void);
  void amdgpu_amdkfd_fini(void);
  -void amdgpu_amdkfd_suspend(struct amdgpu_device *adev);
-int amdgpu_amdkfd_resume(struct amdgpu_device *adev);
+void amdgpu_amdkfd_suspend(struct amdgpu_device *adev, bool run_pm);
+int amdgpu_amdkfd_resume(struct amdgpu_device *adev, bool run_pm);
  void amdgpu_amdkfd_interrupt(struct amdgpu_device *adev,
  const void *ih_ring_entry);
  void amdgpu_amdkfd_device_probe(struct amdgpu_device *adev);
@@ -250,8 +250,8 @@ bool kgd2kfd_device_init(struct kfd_dev *kfd,
   struct drm_device *ddev,
   const struct kgd2kfd_shared_resources *gpu_resources);
  void kgd2kfd_device_exit(struct kfd_dev *kfd);
-void kgd2kfd_suspend(struct kfd_dev *kfd);
-int kgd2kfd_resume(struct kfd_dev *kfd);
+void kgd2kfd_suspend(struct kfd_dev *kfd, bool run_pm);
+int

Re: [PATCH] drm/amd/display: Only enable cursor on pipes that need it

2020-01-30 Thread Harry Wentland

On 2020-01-30 1:29 p.m., Nicholas Kazlauskas wrote:
> [Why]
> In current code we're essentially drawing the cursor on every pipe
> that contains it. This only works when the planes have the same
> scaling for src to dest rect, otherwise we'll get "double cursor" where
> one cursor is incorrectly filtered and offset from the real position.
> 
> [How]
> Without dedicated cursor planes on DCN we require at least one pipe
> that matches the scaling of the current timing.
> 
> This is an optimization and workaround for the most common case where
> the top-most plane is not scaled but the bottom-most plane is scaled.
> 
> Whenever a pipe has a parent pipe in the blending tree whose recout
> fully contains the current pipe we can disable the pipe.
> 
> This only applies when the pipe is actually visible of course.
> 
> Signed-off-by: Nicholas Kazlauskas 

Reviewed-by: Harry Wentland 

Harry

> ---
>  .../amd/display/dc/dcn10/dcn10_hw_sequencer.c | 30 +++
>  1 file changed, 30 insertions(+)
> 
> diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c 
> b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
> index f2127afb37b2..1008ac8a0f2a 100644
> --- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
> +++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
> @@ -2911,6 +2911,33 @@ void dcn10_update_dchub(struct dce_hwseq *hws, struct 
> dchub_init_data *dh_data)
>   hubbub->funcs->update_dchub(hubbub, dh_data);
>  }
>  
> +static bool dcn10_can_pipe_disable_cursor(struct pipe_ctx *pipe_ctx)
> +{
> + struct pipe_ctx *test_pipe;
> + const struct rect *r1 = _ctx->plane_res.scl_data.recout, *r2;
> + int r1_r = r1->x + r1->width, r1_b = r1->y + r1->height, r2_r, r2_b;
> +
> + /**
> +  * Disable the cursor if there's another pipe above this with a
> +  * plane that contains this pipe's viewport to prevent double cursor
> +  * and incorrect scaling artifacts.
> +  */
> + for (test_pipe = pipe_ctx->top_pipe; test_pipe;
> +  test_pipe = test_pipe->top_pipe) {
> + if (!test_pipe->plane_state->visible)
> + continue;
> +
> + r2 = _pipe->plane_res.scl_data.recout;
> + r2_r = r2->x + r2->width;
> + r2_b = r2->y + r2->height;
> +
> + if (r1->x >= r2->x && r1->y >= r2->y && r1_r <= r2_r && r1_b <= 
> r2_b)
> + return true;
> + }
> +
> + return false;
> +}
> +
>  void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx)
>  {
>   struct dc_cursor_position pos_cpy = pipe_ctx->stream->cursor_position;
> @@ -2956,6 +2983,9 @@ void dcn10_set_cursor_position(struct pipe_ctx 
> *pipe_ctx)
>   == PLN_ADDR_TYPE_VIDEO_PROGRESSIVE)
>   pos_cpy.enable = false;
>  
> + if (pos_cpy.enable && dcn10_can_pipe_disable_cursor(pipe_ctx))
> + pos_cpy.enable = false;
> +
>   // Swap axis and mirror horizontally
>   if (param.rotation == ROTATION_ANGLE_90) {
>   uint32_t temp_x = pos_cpy.x;
> 
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Re: [PATCH] drm/amdgpu: Fix implicit enum conversion in gfx_v9_4_ras_error_inject

2020-01-30 Thread Alex Deucher

On Thu, Jan 30, 2020 at 3:33 AM Nathan Chancellor
 wrote:
>
> Clang warns:
>
> ../drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c:967:35: warning: implicit
> conversion from enumeration type 'enum amdgpu_ras_block' to different
> enumeration type 'enum ta_ras_block' [-Wenum-conversion]
> block_info.block_id = info->head.block;
> ~ ~~~^
> 1 warning generated.
>
> Use the function added in commit 828cfa29093f ("drm/amdgpu: Fix amdgpu
> ras to ta enums conversion") that handles this conversion explicitly.
>
> Fixes: 4c461d89db4f ("drm/amdgpu: add RAS support for the gfx block of 
> Arcturus")
> Link: https://github.com/ClangBuiltLinux/linux/issues/849
> Signed-off-by: Nathan Chancellor 

Applied.  thanks!

Alex

> ---
>  drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c 
> b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c
> index e19d275f3f7d..f099f13d7f1e 100644
> --- a/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c
> +++ b/drivers/gpu/drm/amd/amdgpu/gfx_v9_4.c
> @@ -964,7 +964,7 @@ int gfx_v9_4_ras_error_inject(struct amdgpu_device *adev, 
> void *inject_if)
> if (!amdgpu_ras_is_supported(adev, AMDGPU_RAS_BLOCK__GFX))
> return -EINVAL;
>
> -   block_info.block_id = info->head.block;
> +   block_info.block_id = amdgpu_ras_block_to_ta(info->head.block);
> block_info.sub_block_index = info->head.sub_block_index;
> block_info.inject_error_type = 
> amdgpu_ras_error_to_ta(info->head.type);
> block_info.address = info->address;
> --
> 2.25.0
>
> ___
> dri-devel mailing list
> dri-de...@lists.freedesktop.org
> https://lists.freedesktop.org/mailman/listinfo/dri-devel
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amd/display: Only enable cursor on pipes that need it

2020-01-30 Thread Nicholas Kazlauskas

[Why]
In current code we're essentially drawing the cursor on every pipe
that contains it. This only works when the planes have the same
scaling for src to dest rect, otherwise we'll get "double cursor" where
one cursor is incorrectly filtered and offset from the real position.

[How]
Without dedicated cursor planes on DCN we require at least one pipe
that matches the scaling of the current timing.

This is an optimization and workaround for the most common case where
the top-most plane is not scaled but the bottom-most plane is scaled.

Whenever a pipe has a parent pipe in the blending tree whose recout
fully contains the current pipe we can disable the pipe.

This only applies when the pipe is actually visible of course.

Signed-off-by: Nicholas Kazlauskas 
---
 .../amd/display/dc/dcn10/dcn10_hw_sequencer.c | 30 +++
 1 file changed, 30 insertions(+)

diff --git a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c 
b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
index f2127afb37b2..1008ac8a0f2a 100644
--- a/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
+++ b/drivers/gpu/drm/amd/display/dc/dcn10/dcn10_hw_sequencer.c
@@ -2911,6 +2911,33 @@ void dcn10_update_dchub(struct dce_hwseq *hws, struct 
dchub_init_data *dh_data)
hubbub->funcs->update_dchub(hubbub, dh_data);
 }
 
+static bool dcn10_can_pipe_disable_cursor(struct pipe_ctx *pipe_ctx)
+{
+   struct pipe_ctx *test_pipe;
+   const struct rect *r1 = _ctx->plane_res.scl_data.recout, *r2;
+   int r1_r = r1->x + r1->width, r1_b = r1->y + r1->height, r2_r, r2_b;
+
+   /**
+* Disable the cursor if there's another pipe above this with a
+* plane that contains this pipe's viewport to prevent double cursor
+* and incorrect scaling artifacts.
+*/
+   for (test_pipe = pipe_ctx->top_pipe; test_pipe;
+test_pipe = test_pipe->top_pipe) {
+   if (!test_pipe->plane_state->visible)
+   continue;
+
+   r2 = _pipe->plane_res.scl_data.recout;
+   r2_r = r2->x + r2->width;
+   r2_b = r2->y + r2->height;
+
+   if (r1->x >= r2->x && r1->y >= r2->y && r1_r <= r2_r && r1_b <= 
r2_b)
+   return true;
+   }
+
+   return false;
+}
+
 void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx)
 {
struct dc_cursor_position pos_cpy = pipe_ctx->stream->cursor_position;
@@ -2956,6 +2983,9 @@ void dcn10_set_cursor_position(struct pipe_ctx *pipe_ctx)
== PLN_ADDR_TYPE_VIDEO_PROGRESSIVE)
pos_cpy.enable = false;
 
+   if (pos_cpy.enable && dcn10_can_pipe_disable_cursor(pipe_ctx))
+   pos_cpy.enable = false;
+
// Swap axis and mirror horizontally
if (param.rotation == ROTATION_ANGLE_90) {
uint32_t temp_x = pos_cpy.x;
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH] drm/amdgpu/vcn: leave all dpg mode unpause in idle work

2020-01-30 Thread James Zhu

Leave all dpg mode unpause in idle work to fix VCN2.* timeout issue
during transcoding.

Signed-off-by: James Zhu 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c | 13 ++---
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
index f96464e..ed4b753 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vcn.c
@@ -329,20 +329,11 @@ void amdgpu_vcn_ring_begin_use(struct amdgpu_ring *ring)
 
if (adev->pg_flags & AMD_PG_SUPPORT_VCN_DPG){
struct dpg_pause_state new_state;
-   unsigned int fences = 0;
-   unsigned int i;
-
-   for (i = 0; i < adev->vcn.num_enc_rings; ++i) {
-   fences += 
amdgpu_fence_count_emitted(>vcn.inst[ring->me].ring_enc[i]);
-   }
-   if (fences)
-   new_state.fw_based = VCN_DPG_STATE__PAUSE;
-   else
-   new_state.fw_based = VCN_DPG_STATE__UNPAUSE;
 
if (ring->funcs->type == AMDGPU_RING_TYPE_VCN_ENC)
new_state.fw_based = VCN_DPG_STATE__PAUSE;
-
+   else
+   new_state.fw_based = adev->vcn.pause_state.fw_based;
adev->vcn.pause_dpg_mode(adev, ring->me, _state);
}
 }
-- 
2.7.4

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

Suspecting corrupted VBIOS after update of AMDGPU on AMD7870

2020-01-30 Thread Jacob Hrbek

*Hello,*

I believe that system update that included amdgpu on debian testing (but i
am on LFS) corrupted my VBIOS on AMD7870 (+- 4 hours after the update the
GPU using AMDGPU/Radeon drivers resulted in no output).

i'm sending this email to inform about possible bug with my findings on
https://gist.github.com/Kreyren/3e55e9a754e58956e1690e38b1888de7 and i
would appreciate any help in excluding VBIOS corruption from the
diagnostics.

*Thanks,*
- Jacob Hrbek
___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 3/5] drm/amdgpu: allow higher level PD invalidations

Allow partial invalidation on unallocated PDs. This is useful when we
need to silence faults to stop interrupt floods on Vega.

Signed-off-by: Christian König 
Tested-by: Tom St Denis 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 23 ++-
 1 file changed, 18 insertions(+), 5 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 9705c961405b..6038b3c89633 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1467,9 +1467,8 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
 * smaller than the address shift. Go to the next
 * child entry and try again.
 */
-   if (!amdgpu_vm_pt_descendant(adev, ))
-   return -ENOENT;
-   continue;
+   if (amdgpu_vm_pt_descendant(adev, ))
+   continue;
} else if (frag >= parent_shift) {
/* If the fragment size is even larger than the parent
 * shift we should go up one level and check it again.
@@ -1480,8 +1479,19 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
}
 
pt = cursor.entry->base.bo;
-   if (!pt)
-   return -ENOENT;
+   if (!pt) {
+   /* We need all PDs and PTs for mapping something, */
+   if (flags & AMDGPU_PTE_VALID)
+   return -ENOENT;
+
+   /* but unmapping something can happen at a higher
+* level. */
+   if (!amdgpu_vm_pt_ancestor())
+   return -EINVAL;
+
+   pt = cursor.entry->base.bo;
+   shift = parent_shift;
+   }
 
/* Looks good so far, calculate parameters for the update */
incr = (uint64_t)AMDGPU_GPU_PAGE_SIZE << shift;
@@ -1495,6 +1505,9 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
uint64_t upd_end = min(entry_end, frag_end);
unsigned nptes = (upd_end - frag_start) >> shift;
 
+   /* This can happen when we set higher level PDs to
+* silent to stop fault floods. */
+   nptes = max(nptes, 1u);
amdgpu_vm_update_flags(params, pt, cursor.level,
   pe_start, dst, nptes, incr,
   flags | AMDGPU_PTE_FRAG(frag));
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 2/5] drm/amdgpu: return EINVAL instead of ENOENT in the VM code

That we can't find a PD above the root is expected can only happen if
we try to update a larger range than actually managed by the VM.

Signed-off-by: Christian König 
Tested-by: Tom St Denis 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 4ba6a5e5d094..9705c961405b 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -1475,7 +1475,7 @@ static int amdgpu_vm_update_ptes(struct 
amdgpu_vm_update_params *params,
 * shift we should go up one level and check it again.
 */
if (!amdgpu_vm_pt_ancestor())
-   return -ENOENT;
+   return -EINVAL;
continue;
}
 
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 5/5] drm/amdgpu: rework synchronization of VM updates v4

If provided we only sync to the BOs reservation
object and no longer to the root PD.

v2: update comment, cleanup amdgpu_bo_sync_wait_resv
v3: use correct reservation object while clearing
v4: fix typo in amdgpu_bo_sync_wait_resv

Signed-off-by: Christian König 
Tested-by: Tom St Denis 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.c  | 35 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_object.h  |  3 ++
 drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c|  7 
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c  | 41 -
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm.h  |  4 +-
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm_cpu.c  | 22 +++
 drivers/gpu/drm/amd/amdgpu/amdgpu_vm_sdma.c | 15 ++--
 7 files changed, 66 insertions(+), 61 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
index 46c76e2e1281..6f60a581e3ba 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.c
@@ -1403,30 +1403,51 @@ void amdgpu_bo_fence(struct amdgpu_bo *bo, struct 
dma_fence *fence,
 }
 
 /**
- * amdgpu_sync_wait_resv - Wait for BO reservation fences
+ * amdgpu_bo_sync_wait_resv - Wait for BO reservation fences
  *
- * @bo: buffer object
+ * @adev: amdgpu device pointer
+ * @resv: reservation object to sync to
+ * @sync_mode: synchronization mode
  * @owner: fence owner
  * @intr: Whether the wait is interruptible
  *
+ * Extract the fences from the reservation object and waits for them to finish.
+ *
  * Returns:
  * 0 on success, errno otherwise.
  */
-int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr)
+int amdgpu_bo_sync_wait_resv(struct amdgpu_device *adev, struct dma_resv *resv,
+enum amdgpu_sync_mode sync_mode, void *owner,
+bool intr)
 {
-   struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
struct amdgpu_sync sync;
int r;
 
amdgpu_sync_create();
-   amdgpu_sync_resv(adev, , bo->tbo.base.resv,
-AMDGPU_SYNC_NE_OWNER, owner);
+   amdgpu_sync_resv(adev, , resv, sync_mode, owner);
r = amdgpu_sync_wait(, intr);
amdgpu_sync_free();
-
return r;
 }
 
+/**
+ * amdgpu_bo_sync_wait - Wrapper for amdgpu_bo_sync_wait_resv
+ * @bo: buffer object to wait for
+ * @owner: fence owner
+ * @intr: Whether the wait is interruptible
+ *
+ * Wrapper to wait for fences in a BO.
+ * Returns:
+ * 0 on success, errno otherwise.
+ */
+int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr)
+{
+   struct amdgpu_device *adev = amdgpu_ttm_adev(bo->tbo.bdev);
+
+   return amdgpu_bo_sync_wait_resv(adev, bo->tbo.base.resv,
+   AMDGPU_SYNC_NE_OWNER, owner, intr);
+}
+
 /**
  * amdgpu_bo_gpu_offset - return GPU offset of bo
  * @bo:amdgpu object for which we query the offset
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
index 2eeafc77c9c1..96d805889e8d 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_object.h
@@ -283,6 +283,9 @@ void amdgpu_bo_release_notify(struct ttm_buffer_object *bo);
 int amdgpu_bo_fault_reserve_notify(struct ttm_buffer_object *bo);
 void amdgpu_bo_fence(struct amdgpu_bo *bo, struct dma_fence *fence,
 bool shared);
+int amdgpu_bo_sync_wait_resv(struct amdgpu_device *adev, struct dma_resv *resv,
+enum amdgpu_sync_mode sync_mode, void *owner,
+bool intr);
 int amdgpu_bo_sync_wait(struct amdgpu_bo *bo, void *owner, bool intr);
 u64 amdgpu_bo_gpu_offset(struct amdgpu_bo *bo);
 int amdgpu_bo_validate(struct amdgpu_bo *bo);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index 9f42032676da..b86392253696 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -249,13 +249,6 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct 
amdgpu_sync *sync,
owner != AMDGPU_FENCE_OWNER_UNDEFINED)
continue;
 
-   /* VM updates only sync with moves but not with user
-* command submissions or KFD evictions fences
-*/
-   if (fence_owner != AMDGPU_FENCE_OWNER_UNDEFINED &&
-   owner == AMDGPU_FENCE_OWNER_VM)
-   continue;
-
/* Ignore fences depending on the sync mode */
switch (mode) {
case AMDGPU_SYNC_ALWAYS:
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
index 6038b3c89633..aaae2b5e6eee 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_vm.c
@@ -797,7 +797,7 @@ static int amdgpu_vm_clear_bo(struct amdgpu_device *adev,
params.vm = vm;

[PATCH 4/5] drm/amdgpu: simplify and fix amdgpu_sync_resv

No matter what we always need to sync to moves.

Signed-off-by: Christian König 
Tested-by: Tom St Denis 
---
 drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c | 15 +++
 1 file changed, 11 insertions(+), 4 deletions(-)

diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c 
b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
index c124f64e7aae..9f42032676da 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_sync.c
@@ -232,10 +232,19 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct 
amdgpu_sync *sync,
 
f = rcu_dereference_protected(flist->shared[i],
  dma_resv_held(resv));
+
+   fence_owner = amdgpu_sync_get_owner(f);
+
+   /* Always sync to moves, no matter what */
+   if (fence_owner == AMDGPU_FENCE_OWNER_UNDEFINED) {
+   r = amdgpu_sync_fence(sync, f, false);
+   if (r)
+   break;
+   }
+
/* We only want to trigger KFD eviction fences on
 * evict or move jobs. Skip KFD fences otherwise.
 */
-   fence_owner = amdgpu_sync_get_owner(f);
if (fence_owner == AMDGPU_FENCE_OWNER_KFD &&
owner != AMDGPU_FENCE_OWNER_UNDEFINED)
continue;
@@ -265,9 +274,7 @@ int amdgpu_sync_resv(struct amdgpu_device *adev, struct 
amdgpu_sync *sync,
break;
 
case AMDGPU_SYNC_EXPLICIT:
-   if (owner != AMDGPU_FENCE_OWNER_UNDEFINED)
-   continue;
-   break;
+   continue;
}
 
r = amdgpu_sync_fence(sync, f, false);
-- 
2.17.1

___
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx

[PATCH 1/5] drm/amdgpu: fix braces in amdgpu_vm_update_ptes