On Sat, May 3, 2008 at 5:33 AM, Hamish <[EMAIL PROTECTED]> wrote: > Markus Neteler wrote: > > It seems that the precision gets somewhere lost in > > db_convert_column_value_to_string() > > > > i.e. lib/db/dbmi_base/columnfmt.c > > > > but I don't find where is is rounded. > > > lib/db/dbmi_base/valuefmt.c db_convert_value_to_string() does: > case DB_C_TYPE_DOUBLE: > sprintf (buf, "%lf",db_get_value_double(value)); > G_trim_decimal(buf); > > the %lf is rounding to %.6f? > > maybe > - "%lf" > + "%.14f" > > then let G_trim_decimal() remove any extra .00000s. > > but maybe such noisy output is not always wanted. :-/
Let's do the fix because a library should keep the precision. We can reduce noise in the modules then. > (e.g. pretty output for d.what.vect tcl form) > ? Yes, but trim that at module level. > FWIW v.out.ascii defaults to dp=8. So we should consequently use the same length everywhere (adjustable) and full precision at library level. > for the v.colors script, SQL "select * WHERE dbl_col = <FP number>" is > always going to be problematic. > > the C way: > if( abs(test - value) < GRASS_EPSILON ) On Sat, May 3, 2008 at 7:41 AM, Hamish <[EMAIL PROTECTED]> wrote: > In the case of v.colors, the SQL set loop could be rewritten to avoid > this problem by matching to the cat number, not trying to logically match > the FP value. This would be good. Markus _______________________________________________ grass-dev mailing list [email protected] http://lists.osgeo.org/mailman/listinfo/grass-dev
