Re: [PATCH v6 03/16] drm/amdgpu: Split amdgpu_device_fini into early and late
Hi Andrey,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on drm-intel/for-linux-next]
[also build test WARNING on drm-tip/drm-tip drm-exynos/exynos-drm-next
tegra-drm/drm/tegra/for-next linus/master v5.13-rc1 next-20210510]
[cannot apply to pci/next drm/drm-next]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url:
https://github.com/0day-ci/linux/commits/Andrey-Grodzovsky/RFC-Support-hot-device-unplug-in-amdgpu/20210511-003754
base: git://anongit.freedesktop.org/drm-intel for-linux-next
config: x86_64-randconfig-a012-20210510 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project
492173d42b32cb91d5d0d72d5ed84fcab80d059a)
reproduce (this is a W=1 build):
wget
https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O
~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
#
https://github.com/0day-ci/linux/commit/28901216b0a25add4057d60c10eb305d4a32535e
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review
Andrey-Grodzovsky/RFC-Support-hot-device-unplug-in-amdgpu/20210511-003754
git checkout 28901216b0a25add4057d60c10eb305d4a32535e
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross W=1
ARCH=x86_64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot
All warnings (new ones prefixed by >>):
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:444: warning: Function parameter
or member 'sched_score' not described in 'amdgpu_fence_driver_init_ring'
>> drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c:527: warning: expecting prototype
>> for amdgpu_fence_driver_fini(). Prototype was for
>> amdgpu_fence_driver_fini_hw() instead
--
>> drivers/gpu/drm/amd/amdgpu/amdgpu_device.c:3652: warning: expecting
>> prototype for amdgpu_device_fini(). Prototype was for
>> amdgpu_device_fini_hw() instead
--
>> drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c:376: warning: expecting prototype
>> for amdgpu_irq_fini(). Prototype was for amdgpu_irq_fini_sw() instead
vim +527 drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c
d38ceaf99ed015 Alex Deucher 2015-04-20 517
d38ceaf99ed015 Alex Deucher 2015-04-20 518 /**
d38ceaf99ed015 Alex Deucher 2015-04-20 519 * amdgpu_fence_driver_fini -
tear down the fence driver
d38ceaf99ed015 Alex Deucher 2015-04-20 520 * for all possible rings.
d38ceaf99ed015 Alex Deucher 2015-04-20 521 *
d38ceaf99ed015 Alex Deucher 2015-04-20 522 * @adev: amdgpu device
pointer
d38ceaf99ed015 Alex Deucher 2015-04-20 523 *
d38ceaf99ed015 Alex Deucher 2015-04-20 524 * Tear down the fence driver
for all possible rings (all asics).
d38ceaf99ed015 Alex Deucher 2015-04-20 525 */
28901216b0a25a Andrey Grodzovsky 2021-05-10 526 void
amdgpu_fence_driver_fini_hw(struct amdgpu_device *adev)
d38ceaf99ed015 Alex Deucher 2015-04-20 @527 {
c89377d10a11e5 Christian König 2016-03-13 528unsigned i, j;
c89377d10a11e5 Christian König 2016-03-13 529int r;
d38ceaf99ed015 Alex Deucher 2015-04-20 530
d38ceaf99ed015 Alex Deucher 2015-04-20 531for (i = 0; i <
AMDGPU_MAX_RINGS; i++) {
d38ceaf99ed015 Alex Deucher 2015-04-20 532struct
amdgpu_ring *ring = adev->rings[i];
c2776afe740db5 Christian König 2015-11-03 533
d38ceaf99ed015 Alex Deucher 2015-04-20 534if (!ring ||
!ring->fence_drv.initialized)
d38ceaf99ed015 Alex Deucher 2015-04-20 535
continue;
bb0cd09be45ea4 Emily Deng2021-03-04 536if
(!ring->no_scheduler)
bb0cd09be45ea4 Emily Deng2021-03-04 537
drm_sched_fini(>sched);
d38ceaf99ed015 Alex Deucher 2015-04-20 538r =
amdgpu_fence_wait_empty(ring);
d38ceaf99ed015 Alex Deucher 2015-04-20 539if (r) {
d38ceaf99ed015 Alex Deucher 2015-04-20 540/* no
need to trigger GPU reset as we are unloading */
2f9d4084cac96a Monk Liu 2017-10-16 541
amdgpu_fence_driver_force_completion(ring);
d38ceaf99ed015 Alex Deucher 2015-04-20 542}
55611b507fd645 Jack Xiao 2019-06-05 543if
(ring->fence_drv.irq_src)
c6a4079badc2f0 Chunming Zhou 2015-06-01 544
amdgpu_irq_put(adev, ring->fence_drv.irq_src,
c6a4079badc2f0 Chunming Zhou 2015-06-01 545
ring->fence_drv.irq_type);
bb0cd09be45ea4 Emily Deng2021-03-04 546
8c5e13ec6a2c26 Andrey Grodzovsky 2018-09-21
[PATCH v6 03/16] drm/amdgpu: Split amdgpu_device_fini into early and late
Some of the stuff in amdgpu_device_fini such as HW interrupts
disable and pending fences finilization must be done right away on
pci_remove while most of the stuff which relates to finilizing and
releasing driver data structures can be kept until
drm_driver.release hook is called, i.e. when the last device
reference is dropped.
v4: Change functions prefix early->hw and late->sw
Signed-off-by: Andrey Grodzovsky
Acked-by: Christian König
Reviewed-by: Alex Deucher
---
drivers/gpu/drm/amd/amdgpu/amdgpu.h| 6 -
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 26 +++---
drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c| 7 ++
drivers/gpu/drm/amd/amdgpu/amdgpu_fence.c | 15 -
drivers/gpu/drm/amd/amdgpu/amdgpu_irq.c| 26 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_irq.h| 3 ++-
drivers/gpu/drm/amd/amdgpu/amdgpu_kms.c| 12 +-
drivers/gpu/drm/amd/amdgpu/amdgpu_ras.c| 1 +
drivers/gpu/drm/amd/amdgpu/amdgpu_ring.h | 3 ++-
drivers/gpu/drm/amd/amdgpu/cik_ih.c| 2 +-
drivers/gpu/drm/amd/amdgpu/cz_ih.c | 2 +-
drivers/gpu/drm/amd/amdgpu/iceland_ih.c| 2 +-
drivers/gpu/drm/amd/amdgpu/navi10_ih.c | 2 +-
drivers/gpu/drm/amd/amdgpu/si_ih.c | 2 +-
drivers/gpu/drm/amd/amdgpu/tonga_ih.c | 2 +-
drivers/gpu/drm/amd/amdgpu/vega10_ih.c | 2 +-
drivers/gpu/drm/amd/amdgpu/vega20_ih.c | 2 +-
17 files changed, 79 insertions(+), 36 deletions(-)
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
index 380801b59b07..d830a541ba89 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu.h
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu.h
@@ -1099,7 +1099,9 @@ static inline struct amdgpu_device
*amdgpu_ttm_adev(struct ttm_device *bdev)
int amdgpu_device_init(struct amdgpu_device *adev,
uint32_t flags);
-void amdgpu_device_fini(struct amdgpu_device *adev);
+void amdgpu_device_fini_hw(struct amdgpu_device *adev);
+void amdgpu_device_fini_sw(struct amdgpu_device *adev);
+
int amdgpu_gpu_wait_for_idle(struct amdgpu_device *adev);
void amdgpu_device_vram_access(struct amdgpu_device *adev, loff_t pos,
@@ -1319,6 +1321,8 @@ void amdgpu_driver_lastclose_kms(struct drm_device *dev);
int amdgpu_driver_open_kms(struct drm_device *dev, struct drm_file *file_priv);
void amdgpu_driver_postclose_kms(struct drm_device *dev,
struct drm_file *file_priv);
+void amdgpu_driver_release_kms(struct drm_device *dev);
+
int amdgpu_device_ip_suspend(struct amdgpu_device *adev);
int amdgpu_device_suspend(struct drm_device *dev, bool fbcon);
int amdgpu_device_resume(struct drm_device *dev, bool fbcon);
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
index b4ad1c055c70..3760ce7d8ff8 100644
--- a/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
+++ b/drivers/gpu/drm/amd/amdgpu/amdgpu_device.c
@@ -3648,15 +3648,13 @@ int amdgpu_device_init(struct amdgpu_device *adev,
* Tear down the driver info (all asics).
* Called at driver shutdown.
*/
-void amdgpu_device_fini(struct amdgpu_device *adev)
+void amdgpu_device_fini_hw(struct amdgpu_device *adev)
{
dev_info(adev->dev, "amdgpu: finishing device.\n");
flush_delayed_work(>delayed_init_work);
ttm_bo_lock_delayed_workqueue(>mman.bdev);
adev->shutdown = true;
- kfree(adev->pci_state);
-
/* make sure IB test finished before entering exclusive mode
* to avoid preemption on IB test
* */
@@ -3673,11 +3671,24 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
else
drm_atomic_helper_shutdown(adev_to_drm(adev));
}
- amdgpu_fence_driver_fini(adev);
+ amdgpu_fence_driver_fini_hw(adev);
+
if (adev->pm_sysfs_en)
amdgpu_pm_sysfs_fini(adev);
+ if (adev->ucode_sysfs_en)
+ amdgpu_ucode_sysfs_fini(adev);
+ sysfs_remove_files(>dev->kobj, amdgpu_dev_attributes);
+
+
amdgpu_fbdev_fini(adev);
+
+ amdgpu_irq_fini_hw(adev);
+}
+
+void amdgpu_device_fini_sw(struct amdgpu_device *adev)
+{
amdgpu_device_ip_fini(adev);
+ amdgpu_fence_driver_fini_sw(adev);
release_firmware(adev->firmware.gpu_info_fw);
adev->firmware.gpu_info_fw = NULL;
adev->accel_working = false;
@@ -3703,14 +3714,13 @@ void amdgpu_device_fini(struct amdgpu_device *adev)
adev->rmmio = NULL;
amdgpu_device_doorbell_fini(adev);
- if (adev->ucode_sysfs_en)
- amdgpu_ucode_sysfs_fini(adev);
-
- sysfs_remove_files(>dev->kobj, amdgpu_dev_attributes);
if (IS_ENABLED(CONFIG_PERF_EVENTS))
amdgpu_pmu_fini(adev);
if (adev->mman.discovery_bin)
amdgpu_discovery_fini(adev);
+
+ kfree(adev->pci_state);
+
}
diff --git a/drivers/gpu/drm/amd/amdgpu/amdgpu_drv.c
