Doing computation with Float16 is slow -- most CPUs only have Float32 operations implemented in hardware, as well as conversion operations between Float16 and Float32. The only efficient way is to keep the values as Float32 for as long as possible, and only convert to Float16 when storing back to memory.
With a macro such as `@fastmath`, Julia could automatically convert pure Float16 operations to Float32 operations. Otherwise, if Julia offered Float16 operations, it would be required to round in between each operation if the operations are performed as Float32. Or maybe there is language in the standard that allows higher precision operations? I believe this is the case for 64-bit and 80-bit operations. If so, Julia could offer efficient Float16 operations, with probably very surprising semantics: If one manually introduces a temporary variable of type Float16, the result would change... -erik On Tue, Dec 23, 2014 at 9:56 AM, Stefan Karpinski <[email protected]> wrote: > Doing computation with Float16s is not really reasonable - the IEEE standard > describes this as a format that is only for storage. > > > On Dec 23, 2014, at 9:23 AM, Tobias Knopp <[email protected]> > wrote: > > I suppose that the fft limitation is due fftw supporting only float32 and > float64. > > I am not sure if simd supports float16. If not you should not expect any > speed gains. > > Cheers > > Tobi > > Am Dienstag, 23. Dezember 2014 11:56:46 UTC+1 schrieb Mark B: >> >> I was wondering how Julia supports half precision operations? It seems it >> does (more or less) but I'm not sure if there's a lot of type conversion >> going on behind the scenes with associated overhead. Would it be more >> efficient to store and crunch on Float32s? >> >> julia> rand(Float16,2,2) * rand(Float16,2,2) >> 2x2 Array{Float16,2}: >> 0.58301 1.0508 >> 0.48145 0.73438 >> >> julia> sparse(rand(Float16,2,2)) >> 2x2 sparse matrix with 4 Float16 entries: >> [1, 1] = 0.448 >> [2, 1] = 0.15771 >> [1, 2] = 0.79932 >> [2, 2] = 0.50928 >> >> julia> fft(rand(Float16,2,2)) >> 2x2 Array{Complex{Float64},2}: >> 1.76245+0.0im -0.0603027+0.0im >> -0.129639+0.0im -0.390869+0.0im >> >> Oops for the last one - is fft always double precision? >> > -- Erik Schnetter <[email protected]> http://www.perimeterinstitute.ca/personal/eschnetter/
