Re: [PATCH v2] string_helpers: fix precision loss for some inputs

2015-11-04 Thread Rasmus Villemoes
On Wed, Nov 04 2015, James Bottomley wrote: > On Wed, 2015-11-04 at 00:26 +0100, Rasmus Villemoes wrote: >> On Tue, Nov 03 2015, James Bottomley >> wrote: >> >> Please spell it U32_MAX >> > >> > Why? there's no

Re: [PATCH v2] string_helpers: fix precision loss for some inputs

2015-11-03 Thread James Bottomley
On Tue, 2015-11-03 at 23:13 +0100, Rasmus Villemoes wrote: > On Tue, Nov 03 2015, James Bottomley > wrote: > > > From: James Bottomley > > > > It was noticed that we lose precision in the final calculation for some > > inputs. The

Re: [PATCH v2] string_helpers: fix precision loss for some inputs

2015-11-03 Thread Rasmus Villemoes
On Tue, Nov 03 2015, James Bottomley wrote: > From: James Bottomley > > It was noticed that we lose precision in the final calculation for some > inputs. The most egregious example is size=3000 blk_size=1900 in units of 10 > should

Re: [PATCH v2] string_helpers: fix precision loss for some inputs

2015-11-03 Thread James Bottomley
From: James Bottomley It was noticed that we lose precision in the final calculation for some inputs. The most egregious example is size=3000 blk_size=1900 in units of 10 should yield 5.70 MB but in fact yields 3.00 MB (oops). This is because the current algorithm doesn't

Re: [PATCH v2] string_helpers: fix precision loss for some inputs

2015-11-03 Thread Rasmus Villemoes
On Tue, Nov 03 2015, James Bottomley wrote: > On Tue, 2015-11-03 at 23:13 +0100, Rasmus Villemoes wrote: >> On Tue, Nov 03 2015, James Bottomley >> wrote: >> >> > From: James Bottomley >> > >>

Re: [PATCH v2] string_helpers: fix precision loss for some inputs

2015-11-03 Thread James Bottomley
On Wed, 2015-11-04 at 00:26 +0100, Rasmus Villemoes wrote: > On Tue, Nov 03 2015, James Bottomley > wrote: > > > On Tue, 2015-11-03 at 23:13 +0100, Rasmus Villemoes wrote: > >> On Tue, Nov 03 2015, James Bottomley > >>

Re: [PATCH v2] string_helpers: fix precision loss for some inputs

2015-11-03 Thread James Bottomley
On Tue, 2015-11-03 at 13:37 -0800, Bart Van Assche wrote: > On 11/03/2015 01:21 PM, James Bottomley wrote: > > + while (blk_size >= UINT_MAX) > > i++; > > (reduced CC-list) Let's keep at least the lists in cc. > Hello James, > > Is the above loop an infinite loop if blk_size >=