ABBA deadlock in Common Clock Framework

2014-07-02 Thread Tomasz Figa 
Hi All,

While testing linux-next (next-20140625) on Exynos4412-based TRATS2
board, from time to time I hit a deadlock between clk_disable_unused()
of Common Clock Framework and parallel clk_prepare() from s3c24xx-i2c
driver.

I believe the following is happening (in processes 1 and 2):
 1: clk_disable_unused()
 1: clk_prepare_lock()
 1: mutex_lock(clk prepare mutex) // lock A
 1: max77686_clk_is_prepared()
 1: regmap_read()
 2: regulator_set_voltage()
 2: regmap_read()
 2: mutex_lock(max77686 regmap mutex) // lock B
 2: i2c_transfer()
 2: s3c24xx_i2c_xfer()
 2: clk_unprepare()
 2: clk_prepare_lock()
 2: mutex_lock(clk prepare mutex) // lock A
 1: mutex_lock(max77686 regmap mutex) // lock B
 1,2: BOOM!

I suppose this is a general problem affecting any MFD device that is
also a clock provider, but I clearly don't have a good idea what to do
about it.

Full boot log below:

 Uncompressing Linux... done, booting the kernel.
 [0.00] Booting Linux on physical CPU 0xa00
 [0.00] Linux version 3.16.0-rc2-next-20140625-7-g993e726d-dirty 
 (t.figa@AMDC1227) (gcc version 4.8.2 (Gentoo 4.8.2 p1.1, pie-0.5.8) ) #905 
 SMP PREEMPT Tue Jul 1 17:48:42 CEST 2014
 [0.00] CPU: ARMv7 Processor [413fc090] revision 0 (ARMv7), cr=10c5387d
 [0.00] CPU: PIPT / VIPT nonaliasing data cache, VIPT aliasing 
 instruction cache
 [0.00] Machine model: Samsung Trats 2 based on Exynos4412
 [0.00] bootconsole [earlycon0] enabled
 [0.00] debug: ignoring loglevel setting.
 [0.00] Memory policy: Data cache writealloc
 [0.00] On node 0 totalpages: 262144
 [0.00] free_area_init_node: node 0, pgdat c063bc00, node_mem_map 
 eeff
 [0.00]   Normal zone: 1520 pages used for memmap
 [0.00]   Normal zone: 0 pages reserved
 [0.00]   Normal zone: 194560 pages, LIFO batch:31
 [0.00]   HighMem zone: 528 pages used for memmap
 [0.00]   HighMem zone: 67584 pages, LIFO batch:15
 [0.00] Running under secure firmware.
 [0.00] PERCPU: Embedded 7 pages/cpu @eefa3000 s7744 r8192 d12736 
 u32768
 [0.00] pcpu-alloc: s7744 r8192 d12736 u32768 alloc=8*4096
 [0.00] pcpu-alloc: [0] 0 [0] 1 [0] 2 [0] 3 
 [0.00] Built 1 zonelists in Zone order, mobility grouping on.  Total 
 pages: 260624
 [0.00] Kernel command line: root=/dev/mmcblk0p5 rootwait 
 console=ttySAC2,115200n8 earlyprintk ignore_loglevel no_console_suspend
 [0.00] PID hash table entries: 4096 (order: 2, 16384 bytes)
 [0.00] Dentry cache hash table entries: 131072 (order: 7, 524288 
 bytes)
 [0.00] Inode-cache hash table entries: 65536 (order: 6, 262144 bytes)
 [0.00] Memory: 1032564K/1048576K available (4329K kernel code, 266K 
 rwdata, 1480K rodata, 275K init, 272K bss, 16012K reserved, 270336K highmem)
 [0.00] Virtual kernel memory layout:
 [0.00] vector  : 0x - 0x1000   (   4 kB)
 [0.00] fixmap  : 0xffc0 - 0xffe0   (2048 kB)
 [0.00] vmalloc : 0xf000 - 0xff00   ( 240 MB)
 [0.00] lowmem  : 0xc000 - 0xef80   ( 760 MB)
 [0.00] pkmap   : 0xbfe0 - 0xc000   (   2 MB)
 [0.00] modules : 0xbf00 - 0xbfe0   (  14 MB)
 [0.00]   .text : 0xc0008000 - 0xc05b47c0   (5810 kB)
 [0.00]   .init : 0xc05b5000 - 0xc05f9e40   ( 276 kB)
 [0.00]   .data : 0xc05fa000 - 0xc063c800   ( 266 kB)
 [0.00].bss : 0xc063c80c - 0xc0680a00   ( 273 kB)
 [0.00] SLUB: HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
 [0.00] Preemptible hierarchical RCU implementation.
 [0.00]  RCU restricting CPUs from NR_CPUS=8 to nr_cpu_ids=4.
 [0.00] RCU: Adjusting geometry for rcu_fanout_leaf=16, nr_cpu_ids=4
 [0.00] NR_IRQS:16 nr_irqs:16 16
 [0.00] L2C: failed to init: -19
 [0.00] Exynos4x12 clocks: sclk_apll = 8, sclk_mpll = 8
 [0.00]  sclk_epll = 9600, sclk_vpll = 10800, arm_clk = 
 8
 [0.12] sched_clock: 64 bits at 24MHz, resolution 41ns, wraps every 
 2863311519744ns
 [0.008211] Console: colour dummy device 80x30
 [0.012619] Calibrating delay loop... 1590.88 BogoMIPS (lpj=3977216)
 [0.037727] pid_max: default: 32768 minimum: 301
 [0.042578] Mount-cache hash table entries: 2048 (order: 1, 8192 bytes)
 [0.049146] Mountpoint-cache hash table entries: 2048 (order: 1, 8192 
 bytes)
 [0.056985] CPU: Testing write buffer coherency: ok
 [0.062107] missing device node for CPU 0
 [0.066114] missing device node for CPU 1
 [0.070169] missing device node for CPU 2
 [0.074212] missing device node for CPU 3
 [0.078298] CPU0: thread -1, cpu 0, socket 10, mpidr 8a00
 [0.084903] Setting up static identity map for 0x4041c888 - 0x4041c8e0
 [0.120728] CPU1: Booted secondary processor
 [0.140009] CPU1: thread -1, cpu 1, socket

Re: ABBA deadlock in Common Clock Framework

2014-07-02 Thread Russell King - ARM Linux 
On Wed, Jul 02, 2014 at 12:59:04PM +0200, Tomasz Figa wrote:
 Hi All,
 
 While testing linux-next (next-20140625) on Exynos4412-based TRATS2
 board, from time to time I hit a deadlock between clk_disable_unused()
 of Common Clock Framework and parallel clk_prepare() from s3c24xx-i2c
 driver.

This is pretty sad.  The Linux kernel has quite a range of truely excellent
debugging tools which can be built in, but it seems many developers don't
enable them.  The important one here is lockdep (which I notice isn't on
in your kernel.)

Lockdep is a static lock checker - it tracks the dependencies and contexts
between various locks and can report whether deadlock is possible without
having to run into the deadlock.  It is /highly/ recommended that all
developers should run their changes through a kernel with this feature
on before submitting them upstream - see Documentation/SubmitChecklist
point 15.

It can really catch these things before the patch is even submitted...
The recommendation is that if you're doing kernel development, always
have lockdep enabled.  If you want to do performance checking, then
obviously it has an impact on that, so turn it off to do that, but
remember to turn it back on before you do further development.

-- 
FTTC broadband for 0.8mile line: now at 9.7Mbps down 460kbps up... slowly
improving, and getting towards what was expected from it.
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ABBA deadlock in Common Clock Framework

2014-07-02 Thread Peter De Schrijver 
On Wed, Jul 02, 2014 at 12:59:04PM +0200, Tomasz Figa wrote:
 Hi All,
 
 While testing linux-next (next-20140625) on Exynos4412-based TRATS2
 board, from time to time I hit a deadlock between clk_disable_unused()
 of Common Clock Framework and parallel clk_prepare() from s3c24xx-i2c
 driver.
 
 I believe the following is happening (in processes 1 and 2):
  1: clk_disable_unused()
  1: clk_prepare_lock()
  1: mutex_lock(clk prepare mutex) // lock A
  1: max77686_clk_is_prepared()
  1: regmap_read()
  2: regulator_set_voltage()
  2: regmap_read()
  2: mutex_lock(max77686 regmap mutex) // lock B
  2: i2c_transfer()
  2: s3c24xx_i2c_xfer()
  2: clk_unprepare()
  2: clk_prepare_lock()
  2: mutex_lock(clk prepare mutex) // lock A
  1: mutex_lock(max77686 regmap mutex) // lock B
  1,2: BOOM!
 
 I suppose this is a general problem affecting any MFD device that is
 also a clock provider, but I clearly don't have a good idea what to do
 about it.
 

Or if you use notifiers which use i2c... See also
http://comments.gmane.org/gmane.linux.kernel/1553699. One workaround is to
always leave the clock of the i2c controller in a prepared state.

Cheers,

Peter.
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: ABBA deadlock in Common Clock Framework

2014-07-02 Thread Sylwester Nawrocki 
On 02/07/14 13:49, Peter De Schrijver wrote:
 Or if you use notifiers which use i2c... See also
 http://comments.gmane.org/gmane.linux.kernel/1553699. One workaround is to
 always leave the clock of the i2c controller in a prepared state.

Keeping the clock always prepared might not be that bad, given
prepare/unprepare ops are empty on Exynos and I'd say chances
this ever changes are very low. Now we have just an overhead
of calling to the clock core before and after each single I2C
transfer.

--
Regards,
Sylwester
--
To unsubscribe from this list: send the line unsubscribe linux-samsung-soc in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html