Bugs item #2177734, was opened at 2008-10-18 21:58
Message generated for change (Comment added) made by stmane
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2177734&group_id=56967
Please note that this message will contain a full copy of the comment thread,
including the initial issue submission, for this request,
not just the latest update.
Category: Core
Group: Clients CVS Head
Status: Open
Resolution: None
Priority: 5
Private: No
Submitted By: Stefan de Konink (skinkie)
Assigned to: Nobody/Anonymous (nobody)
Summary: Output breaks on foreign characters
Initial Comment:
I presume NLS must be systemwide enabled in order get working 'foreign' output
in mclient. What I now see is that probably the pipe to stdout breaks on a
foreign characters:
| 22771762 | power | line | 22771762 | 22 | 290062167 | 290062167 |
50.91285649999 | 6.130952899999 | Team_Alpha | 2008-08-22 |
: : : : : : : :
9997 : 9996 : : 06:04:08.000000 :
: : : : : : : :
: : : +00:00 :
| 22772346 | power | line | 22772346 | 0 | 244473999 | 244473999 |
50.79486810000 | 6.973957699999 |12315 tuples
sql>select * from way_tags, way_nds, nodes_legacy where k='power' and v='line'
and way_tags.way = way_nds.way and way_nds.to_node = nodes_legacy.id;
12315 tuples
----------------------------------------------------------------------
>Comment By: Stefan Manegold (stmane)
Date: 2008-10-21 17:39
Message:
@skinkie: FYI, here is how I read your initial report --- judge yourself
whether is it sufficient information to reproduce and understand your
problem:
"
I presume NLS must be systemwide enabled in order get working 'foreign'
output in mclient.
"
Ok, so you do have NLS enabled (or not?) on your system --- hm, which
(operating) system are you actually using, here?
In fact, what do you refer to with NLS, here? --- See, e.g.,
http://en.wikipedia.org/wiki/NLS for a list of alternatives; even in
"Computing" there are (at least) three (cf.,
http://en.wikipedia.org/wiki/NLS#Computing) --- let's assume "Native
Language Support" --- so, what is your native language? --- ah, I happen to
know (I guess...): Dutch --- so, you're using a Dutch systems. Good.
Hence, all non-dutch characters are 'foreign" to you? --- hm, 'ë' should
work fine, then ...
"
What I now see is that probably the pipe to stdout breaks on a foreign
characters:
"
"Now?" --- but at least if works fine when redirecting into a file?
"
| 22771762 | power | line | 22771762 | 22 | 290062167 | 290062167 |
50.91285649999 | 6.130952899999 | Team_Alpha | 2008-08-22 |
: : : : : : : :
9997 : 9996 : : 06:04:08.000000 :
: : : : : : : :
: : : +00:00 :
| 22772346 | power | line | 22772346 | 0 | 244473999 | 244473999 |
50.79486810000 | 6.973957699999 |12315 tuples
"
Some output; good.
But what is wrong with that output?
What is / would have been correct?
What did you expect?
Who/what produced that output?
Ok, yet another guessing exercise: Let's assume this is a result of some
SQL query, run on some (unknown) data, most probably using some
not-specified version of MonetDB/SQL, run via mclient in some terminal or
command shell with Dutch NLS (see above).
The queries produces 12315 tuples with 33 attributes each; like the query
itself, the first 12313 tuples have not been added to this bug report, the
last-ut-one tuple apprears to be complete; the last tuple has been cut off
after 9 attribute --- still no indication what kind of 'foreign' character
might have been expected and/or cause any problem ...
Since there is no other information, we must assume that the number of
result tuples (12315) is correct.
"
sql>select * from way_tags, way_nds, nodes_legacy where k='power' and
v='line' and way_tags.way = way_nds.way and way_nds.to_node =
nodes_legacy.id;
12315 tuples
"
Ah, now we have a query --- but no indication what so ever, why it appears
here...
Is it related to the problem? If so how?
It is identical with / similar to / different from the query that created
the above output?
At least is produces the same number of tuples: 12315 --- guess this is
correct, again --- isn't it??
For brevity of the report, the user now chose to omit all 12315 tuples;
hence, they were most probably what he expected. Good.
So --- What was the actual problem, again?
How was the problem triggered?
How can we reproduce it?
... Oh, the first followup of the user tells us more:
"
[...] there is enough information to reproduce the bug on something like
UTF8 characters.
"
Yes? What bug? What information?
"
If even the authors of mclient are not surprised what the output of the
second query is <<12315 tuples>> without setting any '>'-option, better to
not improve my bug reporting skills.
"
Hm, since you refuse to give the slightest information what kind of data
you run your query on, or at least which result you would have expected,
how shall be able to guess whether 12315 tuples is correct or not?
And what is a "'>'-option"?
...
Well, I guess, that should be enough to get the picture --- even without
http://www.chiark.greenend.org.uk/~sgtatham/bugs.html, common sense might
suggest that "showing output" without any indication how that output was
produced and/or what is wrong with that output, and what would have been
correct is just as much information as showing no output at all.
----------------------------------------------------------------------
Comment By: Stefan de Konink (skinkie)
Date: 2008-10-21 16:57
Message:
@sjoerd; personally i would connect non-nls as posix locale but that is me.
So still the bug exist, I read the document after you pointed it to me. In
this document is also mentioned 'show to the programmer the output of your
program' that is the thing I did with the initial posting. I also mentioned
what I *thought* would be the problem. But I'll try to be more verbose.
@mr-meltdown; do you agree with me that in this case GDB was not able to
output too? while it actually does?
----------------------------------------------------------------------
Comment By: Sjoerd Mullender (sjoerd)
Date: 2008-10-21 16:20
Message:
mclient -lsql -Eiso-8859-1
We had to ask lots of questions for you to finally provide some useful
information. If you had internalized the document I pointed you to, that
would not have been necessary. If you feel insulted, tough. But please do
read that document. We have better things to do than trying to be
clairvoyant.
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-10-21 15:58
Message:
utf-8 chars on a posix/C locale is guaranteed to be problematic, IMO.
----------------------------------------------------------------------
Comment By: Stefan de Konink (skinkie)
Date: 2008-10-21 14:54
Message:
locale
------
LANG=
LC_CTYPE="POSIX"
LC_NUMERIC="POSIX"
LC_TIME="POSIX"
LC_COLLATE="POSIX"
LC_MONETARY="POSIX"
LC_MESSAGES="POSIX"
LC_PAPER="POSIX"
LC_NAME="POSIX"
LC_ADDRESS="POSIX"
LC_TELEPHONE="POSIX"
LC_MEASUREMENT="POSIX"
LC_IDENTIFICATION="POSIX"
LC_ALL=
perl -e ''
--------
<< returns nothing >>>
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-10-21 09:09
Message:
I know, but for completeness, I added mine, too ;-)
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-10-21 09:01
Message:
sorry, I meant
@skinkie: what does `locale` say, and what does `perl -e ''` say?
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-10-21 08:56
Message:
$ locale
LANG=en_US.UTF-8
LC_CTYPE="en_US.UTF-8"
LC_NUMERIC="en_US.UTF-8"
LC_TIME="en_US.UTF-8"
LC_COLLATE="en_US.UTF-8"
LC_MONETARY="en_US.UTF-8"
LC_MESSAGES="en_US.UTF-8"
LC_PAPER=nl_NL.UTF-8
LC_NAME="en_US.UTF-8"
LC_ADDRESS="en_US.UTF-8"
LC_TELEPHONE="en_US.UTF-8"
LC_MEASUREMENT="en_US.UTF-8"
LC_IDENTIFICATION="en_US.UTF-8"
LC_ALL=
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-10-21 08:44
Message:
what does `locale` say on your system?
----------------------------------------------------------------------
Comment By: Stefan de Konink (skinkie)
Date: 2008-10-20 23:26
Message:
I presume you have a Linux distro that has NLS enabled, like I did before I
thought I could strip it to save more space for my VM/LiveCD.
SQLrow (len=0x8713698, numeric=0x87136c8, rest=0x87136b0, fields=5,
trim=1) at ../../../src/mapiclient/MapiClient.mx:4
408 for (i = 0; i < fields; i++) {
(gdb)
409 if ((t = rest[i]) != NULL && utf8strlen(t)
> (size_t) len[i]) {
(gdb)
utf8strlen (s=0x8713610 "FSürth") at
../../../src/mapiclient/MapiClient.mx:370
Considering that; I presume something in stream_printf goes wrong where
toConsole is changed. If you want to debug it, of course a quest account is
possible within my vm.
----------------------------------------------------------------------
Comment By: Stefan Manegold (stmane)
Date: 2008-10-20 20:20
Message:
Works for me:
sql>create table MyTab (MyAtt string);
0 tuples
sql>insert into MyTab values ('FSürth');
Rows affected 1
sql>select * from MyTab;
+---------+
| myatt |
+=========+
| FSürth |
+---------+
1 tuple
sql>
----------------------------------------------------------------------
Comment By: Stefan de Konink (skinkie)
Date: 2008-10-20 12:38
Message:
(I'm using the new and improved bugtracker maybe that helps)
The issue is not the white space, the issue is the sudden loss of output
to stdout. If you take a peak on the 'line' with 22771762, you notice a
string Team_Alpha. Now look forward to '22772346' you notice 12315 tuples.
For the reason, that I can positively mark as 'in some way related to my
recompiling to -nls' (in gentoo terms),
http://openstreetmap.org/api/0.5/node/244473999 where the user-value is set
to 'FSürth'.
Now how can I know that I didn't screw up Mserver5, or Mapi in that
perspective? Because my alternative lookup mechanism still works, and is
working on this same dataset.
http://thuis.konink.de/api/0.5/node/244473999
----------------------------------------------------------------------
Comment By: Fabian (mr-meltdown)
Date: 2008-10-20 12:28
Message:
Unfortunately SF eats whitespace, so I'm not able to see the problem
proper, but can it be that you're bitten by mclient breaking up long values
in an attempt not to get wider than your terminal size?
----------------------------------------------------------------------
Comment By: Stefan de Konink (skinkie)
Date: 2008-10-20 12:12
Message:
I take this as offensive; there is enough information to reproduce the bug
on something like UTF8 characters. If even the authors of mclient are not
surprised what the output of the second query is <<12315 tuples>> without
setting any '>'-option, better to not improve my bug reporting skills.
----------------------------------------------------------------------
Comment By: Sjoerd Mullender (sjoerd)
Date: 2008-10-20 10:33
Message:
Please provide details.
Read and internalize
<http://www.chiark.greenend.org.uk/~sgtatham/bugs.html>.
----------------------------------------------------------------------
You can respond by visiting:
https://sourceforge.net/tracker/?func=detail&atid=482468&aid=2177734&group_id=56967
-------------------------------------------------------------------------
This SF.Net email is sponsored by the Moblin Your Move Developer's challenge
Build the coolest Linux based applications with Moblin SDK & win great prizes
Grand prize is a trip for two to an Open Source event anywhere in the world
http://moblin-contest.org/redirect.php?banner_id=100&url=/
_______________________________________________
Monetdb-bugs mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/monetdb-bugs
