Re: Generic File Access Function to read program output

2024-02-07 Thread Carsten Klein

Am 07.02.2024 um 15:54 schrieb Joe Conway:


Maybe write your own in plpython or plperlu?



Yeah... why didn't I think of if? PL/Python would be a first class option.

Nevertheless, I still believe such a function in PostgreSQL's core would 
be a good addition. Maybe someone feels like implementing one some day... :)


Carsten




Generic File Access Function to read program output

2024-02-07 Thread Carsten Klein

Hi there,

on PostgreSQL 14, I'm using function pg_read_file to read a JSON file on 
the server. After that, the JSON file gets casted to jsonb and with 
function jsonb_array_elements I'm iterating over the "records", which I 
transform into a PostgreSQL ROWTYPE with jsonb_populate_record...


Since the source files are actually XML files, these are turned into 
JSON files with Node JS and the fast-xml-parser module (processing JSON 
is much faster and more comfortable than processing XML in PostgreSQL).


The command line of this conversion process is like this:

# node /opt/my_node_apps/xml_to_json.js 

In order to do this without temporary JSON files (which need to be 
deleted at some time), it would be great to have a new Generic File 
Access Function


pg_read_program_output(command)


Although one could argue, that it's not a Generic *File* Access 
Function, that function would be a worthwhile addition and could use the 
same semantics and rules as with the


COPY table_name FROM PROGRAM 'command'

statement. Also the implementation (running a command with the shell and 
capture it's STDOUT) is nearly the same.


In contrast to the other Generic File Access Functions, it will be 
almost impossible to restrict access to programs or commands within the 
database cluster directory (could be a complex shell command). Aside 
from that this makes no sense since, typically, there are no executable 
programs in those directories.


Even worse, it's likely also not possible to restrict the source of the 
content read (the STDOUT) to be any of these directories, since the 
program could just dump anything to its STDOUT.


AFAIT, that's not really an issue but only makes this new Generic File 
Access Function special, in that these restrictions and the meaning of 
role pg_read_server_files just do not apply for it.


Do you know if there is already such a function, maybe provided by an 
extension I do not yet know?



Cheers
Carsten






File-Access functions by default not executable by predefined role "pg_read_server_files"

2023-05-30 Thread Carsten Klein

Hi there,

in PG 14, you've removed explicit 'superuser()' checks in file-access 
functions (like pg_ls_dir, pg_read_file, etc.) and moved to an ACL based 
approach to restrict access to these functions. In turn, you've also 
removed EXECUTE permission from role "public", leaving these functions 
accessible by superusers only.


See also: 
https://github.com/postgres/postgres/commit/e79350fef2917522571add750e3e21af293b50fe


The docs state (up to PG 16), that, in order to access files *outside* 
the cluster directory and the "log_directory", a user must be a 
superuser OR must be granted the role "pg_read_server_files".


https://www.postgresql.org/docs/16/functions-admin.html#FUNCTIONS-ADMIN-GENFILE

Only files within the database cluster directory and the log_directory can be 
accessed, unless the user is a superuser or is granted the role 
pg_read_server_files.



Actually, it seems that it does not matter what file I'm trying to 
access as a non-superuser. There's no distinction between cluster or log 
directory and any other directory. Only the function's ACL seems to 
govern who can execute the function.


After explicitly granting EXECUTE permission for such a function to a 
non-superuser, it can access any file the server backend has read-access to.


Maybe the documentation is partially outdated (mixing old superuser() 
and new ACL-based behavior)? In any case, it could/should be more clear 
in this respect.


Also, as you can see in the above commit, while removing EXECUTE perms 
from role "public", you grant EXECUTE perms for some of them to role 
"pg_monitor". You still do this in PG 16 for even more functions (now in 
new file system_functions.sql -> 
https://github.com/postgres/postgres/blob/master/src/backend/catalog/system_functions.sql).


Why don't you grant EXECUTE perms to predefined role 
"pg_read_server_files"? That would make this role not as useless as it 
currently seems to be (for my mind, at least).


Is there something that I'm overlooking?

Regards, Carsten




Re:

2023-02-21 Thread Carsten Klein

Hi Alberto,

AFAIK, it's described here: (not using postgresql

https://www.pgadmin.org/download/pgadmin-4-apt/

Follow the instructions in the gray box below the package list.

Carsten



Am 21.02.2023 um 13:44 schrieb Alberto García Fumero:

Good morning to all.

I'd need some advice.

I'm trying to install PgAdmin4  from its  repository, on Debian 11,
by the instructions in the concerning page in www.postgresql.org. The
installation fails, as the packages cannot be verified. What shoud I
do? I tried using  deb [trusted=yes], but understandably (https) that
doesn't work.

Fumero







Re: Connect to specific cluster on command line

2022-05-25 Thread Carsten Klein

On Wed 2022-05-25 at 17:05 David G. Johnston wrote:

IIRC they write wrapper scripts they put into the version-agnostic bin 
directory that deal with the version/cluster-name scheme they’ve setup 
before calling the core commands located in the version-specific install 
directory.


You are completely right. For example, /usr/bin/pg_dump is a Perl script 
which evaluates the --cluster option and then calls the real PG tool.


@all Many thanks for all helpful suggestions :)

Regards, Carsten




Re: Connect to specific cluster on command line

2022-05-25 Thread Carsten Klein





pg_lsclusters is not part of core Postgres, and neither is this
--cluster option you mention.  I'm vaguely aware that some packager
(Debian I think) has added an overlay of that sort; but you'd need
to consult the package-level documentation not the community docs
in order to find out more.



Sorry, my fault, it's Debian/Ubuntu. Actually they must have added the 
--cluster option to the PG programs through source code patches...


So, forget about the packager. With core PostgreSQL tools it is possible 
to have more than one cluster. How do you specify what cluster to 
connect to with psql or pg_dump? Do I have to specify the cluster's 
corresponding Unix domain socket directory via the --host option?


Regards, Carsten




Connect to specific cluster on command line

2022-05-25 Thread Carsten Klein

Hi there,

how can I connect to a specific cluster on the command line, e. g. with 
psql, pg_dump or pg_dumpall?


pg_lsclusters returns a list of all clusters available:

Ver Cluster Port [...]
14  main5432 ...
14  test5433  ...

I want to connect to or dump database xyz in the test cluster. Is it 
sufficient to specify the cluster's port only?


Perl script pg_backupcluster calls psql and pg_dump with a --cluster option:

pg_dump --cluster 14/test ...

However, this option is not documented (at least I didn't find anything) 
and also I didn't find anything in the sources on GitHub.


Actually, I only have the 14/main instance and cannot really test 
accessing the `test` cluster. However, psql and pg_dump actually work 
with --cluster 14/main and report an error when I specify 14/test so, 
the option --cluster seems to work. Is it intentionally not documented?


So, whats the recommended way to connect to a specific cluster? Is it 
just the port?


Regards, Carsten





Re: Strange results when casting string to double

2022-02-20 Thread Carsten Klein




On 19.02.2022 20:34 Tom Lane wrote:


Per grep, there is no call of fesetround() in the Postgres source
tree.  I'm not sure offhand whether libc exposes any other APIs
that could change the rounding mode, but I am quite sure that we
wouldn't be intentionally changing it anywhere.

The OS would surely allow each process to have its own setting of the
rounding mode, so I doubt you can see it from outside.

Another point to keep in mind is that no matter how invasive that
import script might be, it's still hard to explain how it'd affect
the rounding mode in other backend processes.  You have to postulate
either that the rounding mode has been changed in the postmaster
process (and then inherited by session backends via fork()), or that
some code running at the time of child process creation changes the
mode, or that they replaced numeric_float8 with something else.

I think the only way that the postmaster's rounding mode could change
after postmaster start is the cosmic-ray hypothesis; while we do have
features that'd allow loading extra code into the postmaster, I'm
pretty sure they only take effect at postmaster start.  So even if
that import script tried to do that, it wouldn't have succeeded yet.

Of the other two hypotheses, "substitute numeric_float8" seems like
the most likely, especially given the other stuff you mentioned the
script doing.  Have you checked the relevant pg_cast entry to see
if it's been changed?  It'd also be interesting to see if the odd
rounding behavior happens in all databases of the cluster or just
one.


The script has finished!

After a restart of the database, everything works as expected again. 
Rounding as well as text/numeric to double precision works the same on 
all of my servers.


Prior to restarting, I've implemented my own Python based versions of both

int fegetround(void);

int fesetround(int rounding_mode integer);

Have a look a these:

CREATE OR REPLACE FUNCTION fegetround()
  RETURNS integer AS
$BODY$
if 'fn.fegetround' in SD:
return SD['fn.fegetround']()

from ctypes import cdll
from ctypes.util import find_library
libm = cdll.LoadLibrary(find_library('m'))
def fegetround():
return libm.fegetround()

SD['fn.fegetround'] = fegetround
return SD['fn.fegetround']()
 $BODY$
  LANGUAGE plpython3u VOLATILE
  COST 100;

CREATE OR REPLACE FUNCTION fesetround(rounding_mode integer)
  RETURNS integer AS
$BODY$
if 'fn.fesetround' in SD:
return SD['fn.fesetround'](rounding_mode)

from ctypes import cdll
from ctypes.util import find_library
libm = cdll.LoadLibrary(find_library('m'))
def fesetround(rounding_mode):
return libm.fesetround(rounding_mode)

SD['fn.fesetround'] = fesetround
return SD['fn.fesetround'](rounding_mode)
 $BODY$
  LANGUAGE plpython3u VOLATILE STRICT
  COST 100;

With those, I was able to proof, that actually the "wrong" rounding mode

FE_DOWNWARD (0x400)

was in effect for every new process/connection with all the described 
effects on casting from string or numeric to double precision:


SELECT 1.56::double precision
-> 1.55

Setting rounding mode to

FE_TONEAREST (0x0),

instantly lead back to the expected casting behavior:

SELECT 1.56::double precision
-> 1.56

Setting rounding mode after restarting the database is still possible, 
however, new sessions start off with the "correct" rounding mode 
FE_TONEAREST (0x0). So, the only thing that's really changed after the 
restart was, that the postmaster now has the "correct" rounding mode, 
which it promotes down when forking off child processes.


We'll likely never know, why ever the postmaster got tainted with that 
FE_DOWNWARD (0x400) rounding mode.


As Tom Lane said, no matter how aggressive the script could be, it can, 
if at all, only change its current session's rounding mode. So, maybe it 
actually was a random bit flip or a side effect caused by a quite rare 
error condition in postmaster.


Nearly the same is true for any core functions or casts hijacked by the 
script - these are only in effect for the database the script was ever 
connecting to. In my case, the script only used one database. However, 
the issue was present with any database.


Two official math functions to get and set the session's rounding mode 
provided by PostgreSQL could be a good add-on for any of the next 
versions of the database. Thinking about it again... maybe that's just 
too dangerous :-p


Finally, many thanks to all that supported me and came up with that many 
helpful ideas! :-)


Regards, Carsten




Re: Strange results when casting string to double

2022-02-19 Thread Carsten Klein





On 19.02.2022 15:35, Tomas Pospisek wrote:

That would be visible via `lsof`. `libc. The file `...libc...so` that 
`postgres` is keeping open would have the text `DEL` (as in deleted) in 
the `FD` column of `lsof`'s output.


As opposed to a newly started program which would have `REG` (regular 
file) there.

*t


Actually, PostgreSLQ uses the same libc than any recently started 
program. lsof reports REG and my /lib/x86_64-linux-gnu/libc-2.19.so has 
file date Mar 27, 2019, so it's much older than the point in time when 
PostgreSQL was started the last time (aprx. 480 days ago).


After all, I guess the idea of a wrong rounding setting (set to round 
down, raised by Peter J. Holzer) seems most plausible to me. In 
particular, since the "right" and "wrong" values caused by wrong 
rounding shown by Tom Lane are exactly the values I am seeing.


Also, there is a quite aggressive import script running for some days, 
which even tried to replace some of the core functions, like 
array_length (it added a pure PL/pgSQL version in public schema). Seems 
like they wanted to "Polyfill" some required functions. Maybe that 
script is responsible for changing the process' rounding mode? The 
customer started that script without thinking too much about it... :(


@Tom Lane: you say, PostgreSQL does not / cannot change rounding mode at 
any time? There is no function to do so? So, the script (it's more like 
a module) must provide a Shared Object module with a C function in order 
to change the FPU's rounding mode?


Because several people recommended using a debugger in order to see 
whats going on here: actually I have no expertise with debugging on 
Linux without an IDE. So I did not yet think of using a debugger so far.


I will try a restart of the DB ASAP. However, in order to prove the 
rounding mode thesis: someone knows whether fesetround(FE_DOWNWARD) just 
sets a flag in the C runtime environment or does this call actually set 
the rounding mode in the FPU's control word? Is there any chance to get 
that current rounding mode with or even without a debugger? (Maybe some 
file in /proc filesystem?)


I'm very unhappy with the customer starting that import script. In order 
to find the `smoking gun`, I will now have a closer look at the 
"product" that import is based on and what they have done to the database...


Regards, Carsten




Re: Strange results when casting string to double

2022-02-18 Thread Carsten Klein





On 18.02.2022 16:32, Tom Lane wrote:

Yeah, you said that upthread, which makes the whole thing pretty
baffling.  One possible explanation is that your small program got linked
against a different version of libc than what the Postgres backend is
using ("ldd" would help you check that, but given the age of the Postgres
installation, this seems plausible).  Beyond that it's hard to think
of any explanation other than hardware fault or corrupted executable.


Tom,

both PostgreSQL and my C program are linked to the same libc.so.6. Same 
path, same MD5 sum. Since libc is a Shared Object (so), both processes 
should really run the identical code. Am I missing something? I've 
written and compiled the small C program on the same old Ubuntu OS.


So, you're not aware of any ways this behavior could be achieved from 
within PostgreSQL? Something like a custom cast (actually, there is 
none) or something that could intercept string to double conversion? 
That would be something to look at closer. The question is: how would 
you implement such an evil database wide text to double conversion (just 
to kid users) if you had to?


PostgreSQL is up for more than 480 days on that server. I'm thinking of 
giving a restart of the database a try. However, there's a long running 
import taking place, so this will not happen before mid or end of next week.


Regards, Carsten




Re: Strange results when casting string to double

2022-02-18 Thread Carsten Klein





On 18.02.2022 13:28, Peter Eisentraut wrote:

float8in() really just calls the operating system's strtod() function. I 
would test that one directly with a small C program.


It's also possible that different compiler options lead to different 
optimizations.


That's what I did. Here's my small C program: (nicht lachen *g*)

#include 
#include 

int main(int argc, char* argv[]) {

/* default string to convert */
char buf[10] = "1.56\0";

/* choose and print string to convert */
char* sval = argc > 1 ? argv[1] : buf;
printf("string value: %s\n", sval);

/* convert and print */
char* ptr;
double dval = strtod(sval, );
printf("double value: %.20f\n", dval);

return 0;
}

It works correctly on all these servers. Here's its output:

string value: 1.56
double value: 1.56005329

I didn't test different compiler options. However, PostgreSQL was always 
installed from official Ubuntu 14.04 repositories (getting the binaries, 
not the source packages), so all binaries should have been compiled with 
the same options.


Carsten




Re: Strange results when casting string to double

2022-02-17 Thread Carsten Klein




AOn Thu, Feb 17, 2022 at 10:27 AM Peter J. Holzer wrote


I don't think these explain the difference. I'd check whether the
postgresql binaries and all the the shared libraries are the same.

Or - since this only happens on the test system and not on the production
system - I'd just clone the production system again to create a new test
system and see if the problem happens there, too.



File postgresql and it's loaded libraries are identical. Same set of 
libraries loaded on all servers; all loaded library have identical MD5 sums.


It's the customer's VMware, so I can't just make another clone. However, 
I'm quite sure that it will work on a newly cloned testing system (since 
on production system everything is OK).


Carsten




Re: Strange results when casting string to double

2022-02-17 Thread Carsten Klein

On Thu, Feb 17, 2022 at 09:41 AM Thomas Kellerer wrote:

Carsten Klein schrieb am 16.02.2022 um 14:27:

Ah, man versteht sich :)

I'm using several (now unsupported) PostgreSQL 9.3.24 servers on
different (ancient) Ubuntu 14.04 LTS machines. On only one of those
servers, I get strange/wrong results when converting a string into a
double value:

SELECT 1.56::double precision;

--> 1.55   (wrong!)

Although I do not find any differences in configuration, on all other servers 
the result looks like this (correct?):

SELECT 1.56::double precision;

--> 1.56   (correct!)



I wonder if extra_float_digits is different between those  systems
Maybe initialized by differently configured SQL clients.


As I've seen in the sources in file /src/backend/utils/adt/float.c, 
extra_float_digits is used when converting double precision values back 
to text only. The binary (BIT) representation if the double value's 
fraction (mantissa) tells me, that the actual double precision value is 
already wrong.


Also, extra_float_digits is constantly zero in my tests. I've only been 
using pgAdmin III for that and always ensured with


SELECT current_setting('extra_float_digits');

that it's zero.

Carsten





Re: Strange results when casting string to double

2022-02-17 Thread Carsten Klein





On Thu, Feb 17, 2022 at 00:07 Gavan Schneider wrote:


Harking back to my long distant formative years I was taught to never ever rely 
on equality tests when it came to floating point values. Nothing has changed in 
this regard.

If binary exact is part of the requirement then the answer is INTEGER or 
NUMERIC never FLOAT, REAL, DOUBLE or anything similar.

So, assuming the trigger function is the only source of this spurious grief one 
could apply the lesson taught in the 60s from the dawn of computing:
EQUALITY = absolute_value(op1 - op2)  < epsilon — where op1 & op2 are 
DOUBLE, and epsilon is smaller than you care for

Given the intrinsic (standards compliant) uncertainty when converting from 
absolute (e.g., string representation) to floating point there will never be 
value resolving why there are differences.

I suggest using the comparison that is appropriate to the representation of 
those values or fix the design by using the proper representation.


Just some(!) background:

I know that there are other ways to compare _any_ floating point values. 
However, doing that for a whole ROW in a fast manner is not trivial (e. 
g. it tends to get quite slow). With the hstore extension and (so 
called) binary equality I've found a very fast way which is also fully 
generic. hstore uses text representation, so comparisons depend on how 
values are converted to and from text.



But all that is not the point. Double precision conversion algorithms 
are well defined and deterministic, so it should yield the same results 
when called with the same arguments (every time and on every machine). 
That is


1.56::double precision == 1.56005329070518201E0

1.56::double precision != 1.55983124610025698E0

With reduced precision (as in PG), that gives you:

1.56::double precision == 1.56

1.56::double precision != 1.55

However, one of my ProstgreSQL servers returns the latter (wrong) value. 
You can test this with C library function 'strtod' or with any of the 
online converters around:


https://www.binaryconvert.com/result_double.html

Click 'New Conversion' and enter 1.56 into the 'Decimal' field. Then 
'Convert to binary' or hit enter.


So, the primary problem of that PostgreSQL server is, that it converts 
text to double in a wrong way. Apart from any triggers, "binary 
equality" and whatever else I'm doing in this project, this has dramatic 
effects on the database, as it's messing up the values that I'm storing:


Imagine I do:

INSERT INTO foo (my_col) VALUES ('Hello World');

But the database has happily stored a different string:

SELECT my_col FROM foo;
my_col
-
 Hello Worlc
(1 row)

Finding that string again may be done with "fuzzy search" or regular 
expressions, but the much better approach is the database not to let 
mess up the string while storing it.


Double precision values are limited in precision (in binary) and there 
are numbers, that cannot be stored exactly (as it's true for many 
decimal numbers, like 1/3 ~ 0.3 as well). Nevertheless, with a given 
maximum of precision, the same values should have the same (well 
defined) binary value so that conversions between text and double should 
not change the value at any time on any machine.


Carsten




Re: Strange results when casting string to double

2022-02-16 Thread Carsten Klein




On Wed, Feb 16, 2022 at 08:11 PM David G. Johnston wrote:

You said they are more or less the same.  Problems like these tend to 
hide in the "less" portion of the inequality.


On of the virtualized servers was created as a clone of the other one 
(using VMware to clone the VM). So, basically, these are very equal. Of 
course, they diverged over time.


Focusing on PostgreSQL, here are the differences of postgresql.conf, 
comparing testing system and production system:


< work_mem = 8MB# min 64kB
---
> work_mem = 4MB# min 64kB
417c417
< #log_statement = 'all'# none, ddl, mod, all
---
> #log_statement = 'none'   # none, ddl, mod, all

Both PostgreSQL server have the same roles and users, that same 
extensions installed and no preloaded libraries.


Given that this isn't working as expected it doesn't make for a great 
testing system.  Install and initdb 14.2 on this machine and let's see 
what PostgreSQL produces.


The testing system runs since 2015. I don't know whether the problem was 
present from the beginning. But I don't think so, as we also have 
"correct" double values in that database. Now, since "binary equality" 
of the double precision values is a new requirement, we started to 
notice, that (at least not) newly added (UPDATEd) values, e. g. 1.56 are 
not binary equal to already present values:


Table abc, column xyz: currently (before) 1.56

UPDATE abc SET xyz = 1.56;

Table abc, column xyz: after 1.55999

We have a trigger, that monitors such updates and it reports a changed 
value, which is not correct. The problem is, that the assignment


   SET xyz = 1.56

actually means

   SET xyz = 1.55999

since getting a double value from the string 1.56 yields 1.55999.

Yes, moving to the latest PostgreSQL version might fix that error. 
However, this is a customer's testing system. Actually, it is intended 
to be reinstalled with Ubuntu 22.04 LTS which brings PostgreSQL 14. But 
prior to that, we need to complete a project on the testing system that 
requires that "binary equality" of double values.


What is the precise version of libc that is installed for one.  Exact 
ESX releases too.


Both VM servers run on

ESXi 6   (correct behavior)
ESXi 6.5 (misbehaving)

All machines use libc version 2.19 (libc-2.19.so).

This isn't really all that interesting a report for the project if it 
only exists in one ancient system that cannot be experimented with.  
Maybe it's a faulty register on that machine's CPU.  There is more 
double-checking and comparing that can be done here but it seems 
unlikely to be productive.  It is more plausible that the snowflake 
machine in question just has issues and needs to be retired.  Installing 
a newer version of PostgreSQL on it before junking it is about the right 
amount of experimental effort.


I just wanted to ask whether someone knows something about this or has 
ever heard about such a behavior. You say, the snowflake machine has 
issues... I don't believe in hardware issues, since it runs in VMware 
and likely on many different CPUs. Isn't it more than unlikely that such 
a constantly occurring error is caused by one faulty CPU (among that 
many CPUs an ESX server typically has)?


And, keep in mind that strtod function works as expected from a simply C 
testing program. I guess that the parsed double's value gets modified 
somewhere in PostgreSQL after strtod was called. However, I do not yet 
see where and why. I was hoping that someone of you could help.


Carsten




Re: Strange results when casting string to double

2022-02-16 Thread Carsten Klein




On Wed, Feb 16, 2022 at 05:46 PM Adrian Klaver wrote

On 2/16/22 05:27, Carsten Klein wrote:

Hi there,

I'm using several (now unsupported) PostgreSQL 9.3.24 servers on 
different (ancient) Ubuntu 14.04 LTS machines. On only one of those 
servers, I get strange/wrong results when converting a string into a 
double value:




Has anyone an idea of what's going on here? I know, this version is 
far from still being supported, however, there's not much real changes 
in file float.c between these versions (in other words, this may 
happen with recent versions as well?). The database instances on all 
servers are configured quite the same (more or less). 




What is the more or less for the problem server?


What? Didn't get that...



Are the hardware(CPU) architectures the same for all the servers?


The problem server is a virtual machine (VMware). I've tested this on 
two other servers and do receive probably correct results. One of these 
is also a virtual machine (same VMware, running on different hardware), 
the other one is a physical box with an AMD Athlon(tm) II X4 640 64-bit 
processor.


Customer site:

Production System:  VMware (ESX Host A)   works as expected

Testing System: VMware (ESX Host B)   DOES NOT WORK as expected


Our site:

Development System: AMD Athlon(tm) II works as expected


However, since the strtod function works correctly from my tiny C 
program on all these machines, I don't believe that this problem has to 
do with hardware or architecture. I guess, PostgreSQL dynamically links 
to these C library functions, right?




If not how does the problem server differ?


I certainly have no idea. Actually, I don't see any more relevant 
differences.




Was Postgres installed from the same source/same way on all the server?


PostgreSQL was installed from Ubuntu's official repositories (14.04 LTS) 
on all servers.


--
Carsten Klein
c(dot)klein(@)datagis(dot)com




Strange results when casting string to double

2022-02-16 Thread Carsten Klein

Hi there,

I'm using several (now unsupported) PostgreSQL 9.3.24 servers on 
different (ancient) Ubuntu 14.04 LTS machines. On only one of those 
servers, I get strange/wrong results when converting a string into a 
double value:


SELECT 1.56::double precision;

--> 1.55   (wrong!)

Although I do not find any differences in configuration, on all other 
servers the result looks like this (correct?):


SELECT 1.56::double precision;

--> 1.56   (correct!)

AFAIK, this conversion is done by internal function float8in, which, 
when called directly, yields the same results:


SELECT float8in('1.56');

--> 1.55   (wrong!)   on one server, and
--> 1.56   (correct!) on all other servers.

Option extra_float_digits is zero (0) while doing all these tests. Also, 
the problem seems to occur while converting text to double precision and 
not when displaying the obtained double precision value. Why? The binary 
representation of the double precision value is also different.


I've created a small to_bit function in Python to get the double 
precision value's binary representation:



CREATE OR REPLACE FUNCTION to_bit(value double precision)
  RETURNS bit AS
$BODY$
if 'fn.to_bit_d64' in SD:
return SD['fn.to_bit_d64'](value)

import struct
def to_bit_d64(value):
return ''.join('{:0>8b}'.format(c) for c in struct.pack('!d', 
value))


SD['fn.to_bit_d64'] = to_bit_d64
return SD['fn.to_bit_d64'](value)
 $BODY$
  LANGUAGE plpython3u IMMUTABLE STRICT
  COST 100;


The fraction (mantissa) of both values is different by 1:

valuefraction
1.55 10000101111010000101111010000101
1.56 10000101111010000101111010000110

The fraction of the probably wrong value is one less than the fraction 
of the correct value.


Formatting both values with 20 digits right of the decimal separator 
(like printf("%.20f" ...) yields:


1.55983124 (wrong!)
1.56005329 (correct!)

Since even calling function float8in directly returns a probably wrong 
result on one of the servers makes me believe, that there's no custom 
cast in place being responsible for the wrong results.


Function float8in basically relies on C library function

double strtod(const char *str, char **endptr)

which I tested with a tiny C programm (calling strtod only and printing 
the result with printf("%.20f", val);). The result is


1.56005329 (correct!)

on every server. So, seems like the C library function works as expected 
on all servers.


Although I'm not a C expert, I don't find anything suspicious that 
function float8in does with the value returned from strtod.


In version 9.3.24, file /src/backend/utils/adt/float.c looks a bit 
different from the file in master branch. However, basically both 
versions do much the same things. The old 9.3.24 version does some more 
special error checks (#ifdef HAVE_BUGGY_IRIX_STRTOD, #ifdef 
HAVE_BUGGY_SOLARIS_STRTOD and CHECKFLOATVAL), but these either throw 
errors or set the converted value to return to a special value (if 
indicated).


Has anyone an idea of what's going on here? I know, this version is far 
from still being supported, however, there's not much real changes in 
file float.c between these versions (in other words, this may happen 
with recent versions as well?). The database instances on all servers 
are configured quite the same (more or less). All run with the same 
extensions installed; none is using any preloaded libraries (which may 
replace C library function strtod?).


--
Carsten Klein
c(dot)klein(@)datagis(dot)com