Re: [AOLSERVER] chinese characters and oracle driver
Make sure that any vars set in your shell environment that relate to this are also set in your nsd wrapper script. I wish I knew for sure if that is enough for them to be effective, but I don't. I vaguely recall that the C API the driver uses is called Pro-C on the Oracle side - you might want to see if you can find some docs on that for your Oracle version and see if they can shed any light on this. Maybe you have to do something in the driver to make sure your language settings are acted upon. janine On Jul 18, 2011, at 12:03 PM, Brad Chick wrote: Well, I reached an impasse. I grabbed the latest source from CVS, added support for the 2 datatypes (NCHAR, NVARCHAR2), and recompiled. Then, I turned on debugging for the driver. The logs look great. The driver constructs the queries appropriately: e.g. [18/Jul/2011:14:17:30][10744.1099659584][-default:1-] Notice: nsoracle.c:3169:Ns_OracleExec: entry (dbh 0xd8392b0, sql insert into test_zhs (foo, bar) values (N'男孩儿 儿男孩 Yo', N'男孩儿 儿男孩 Yo')) But the data somehow gets mangled in oracle. And that same insert statement works perfectly well in sqlplus: insert into test_zhs (foo, bar) values (N'男孩儿 儿男孩 Yo', N'男孩儿 儿男孩 Yo'); (and i've tried it a million ways (e.g. without the leading 'N'')). When viewed from sqplus, the sqplus added data looks fine, while the stuff that comes in through the driver is mangled. Dunno. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] chinese characters and oracle driver
I've had to deal with Chinese Characters and Postgres. I don't recall all the details anymore but I do recall what Peter is saying, that Tcl was a culprit more than the database. I would use sqlplus to check what myform.tcl is inserting into the database. That will at least tell you whether Tcl is munging it on the way in or the way out (could be both) and give you a place to start tweaking. janine On Jul 15, 2011, at 1:33 PM, Peter Sadlon wrote: Just a quick shot in the dark here. I have ran into encoding issues in the past because by default TCL will assume everything is Latin-1 and it is not always straight forward when a conflict will happen. You can set a variable like $first_names to be a UTF-8 string and it can be written back out as a UTF-8 string until you do something to to. For example, do you have a wrapper that will clean a database value to prevent sqlinjection like ns_dbquotevalue? If you do then that string's data gets converted to Latin 1. Try something like this: set first_names [encoding convertto utf-8 $first_names] Insert it into your database, something like: insert into test_zhs (foo) values ([ns_dbquotevalue $first_name]); Check from the command line if the data is in there correctly. If it is you will probably have some issues getting it out for the same reason. Look at the TCL's encoding: http://tcl.activestate.com/! man/tcl8.5/TclCmd/encoding.htm Like I said, just a shot in the dark at what the issue may be. _Peter Date: Fri, 15 Jul 2011 16:01:19 -0400 From: b...@chickcentral.com Subject: [AOLSERVER] chinese characters and oracle driver To: AOLSERVER@LISTSERV.AOL.COM I am having trouble getting Chinese characters in/out of oracle with AOLserver. Here is my stack: Oracle 11g TCL 8.5 AOLserver 4.51 Oracle Driver version 2.7 The existing database has a database character set of WE8ISO8859P1. But we are using NCHAR and NVARCHAR2 datatypes to store Unicode characters. The NLS_NCHAR_CHARACTERSET is properly set to AL16UTF16. I am setting the following in the environment as an nsd wrapper: export NLS_LANG=_.UTF8 I can login to sqlplus and insert and select chinese characters: insert into test_zhs (foo) values (N'男孩儿男孩儿� (B'); SQL select * from test_zhs; FOO 男孩儿男孩儿 I also am sure that aolserver/tcl are treating the characters appropriately. For example, this form takes whatever characters are inputed into the form, tries to insert them, and spits them back out. When chinese characters are inputed, that is how the server returns them. http://jp.xacte.com:8181/test/db/myform.tcl On the other hand, no matter what I try, I can't get aolserver to get them into oracle properly. I suspect it's the oracle driver, but people have suggested that it is possible to put unicode characters into oracle using that driver. Any help would be way helpful. Thanks -- AOLserver - http://www.aolserver.! com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Permissions for ns_adp_include
I'm setting up a shared server which will have multiple clients logging in, so I'm trying to restrict permissions as much as possible. nsd runs as the user and group that owns the files, so I thought that I could completely turn off permissions for other. However, when I did that ns_adp_include started throwing permission errors. Is there some reason why it requires read permission for other even though the files are owned by the user that nsd is running as? thanks, janine --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Permissions for ns_adp_include
I just ran a bunch of chmod tests... other *has* to have read and execute on the directories. It's fine for them to have no permissions on the files. Everything is owned by the site user so this seems rather wrong to me, but I suppose I will have to live with it. janine On Feb 21, 2011, at 9:52 AM, Scott Goodwin wrote: Check the full path of one of your adp includes that fails with the permissions error and ensure that the owner has read permissions on the file, and that owner has execute perms on all directories in its path. Ownership of a file gives ability to change perms on it, but you can still change the perms so that the file is unreadable by the owning uid. /s. On Feb 21, 2011, at 12:29 PM, Janine Ohmer wrote: I'm setting up a shared server which will have multiple clients logging in, so I'm trying to restrict permissions as much as possible. nsd runs as the user and group that owns the files, so I thought that I could completely turn off permissions for other. However, when I did that ns_adp_include started throwing permission errors. Is there some reason why it requires read permission for other even though the files are owned by the user that nsd is running as? thanks, janine --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] AOLserver, nginx, and port numbers
I'm moving some sites onto an Amazon EC2 instance, which has only one IP address, so I'm using nginx to make this work. The websites (both Apache and AOLserver) are running on various internal ports. The site I'm having trouble with is running on http://localhost:8082 internally, and on http://shared.furfly.net to everyone coming in through nginx. My problem is the value returned by [ns_conn location], which is http://shared.furfly.net:8082. This is partially correct; I told the site it's hostname is shared.furfly.net and it seems to be ok with that. But I can't get away with telling it that it's running on port 80, and [ns_conn location] seems to be returning whatever's in the config file. What I want, of course, is just http://shared.furfly.net. Is there some way to set this up so it returns the desired value, even though from AOLserver's point of view that's not exactly correct? thanks, janine --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] AOLserver, nginx, and port numbers
Thanks, that did the trick! It works in the nsopenssl section also. janine On Feb 2, 2011, at 7:08 PM, Jeff Rogers wrote: It doesn't seem to be documented, but you can set a 'location' parameter for nssock that will override this: ns_section ns/server/server1/module/nssock ns_param port 8082 ns_param location http://shared.furfly.net Cheers, -J Janine Ohmer wrote: My problem is the value returned by [ns_conn location], which is http://shared.furfly.net:8082. This is partially correct; I told the site it's hostname is shared.furfly.net and it seems to be ok with that. But I can't get away with telling it that it's running on port 80, and [ns_conn location] seems to be returning whatever's in the config file. What I want, of course, is just http://shared.furfly.net. Is there some way to set this up so it returns the desired value, even though from AOLserver's point of view that's not exactly correct? -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
I will try this tonight and see what happens (can't restart the site during the
day).
Gustaf, you may well be correct that it's bad data, but this wasn't happening
on the old system (or at least that's what they tell me) so I'm hoping to fix
it with configuration.
Brian, I used pg_dump/restore. I have been using the custom dump format for
a while, which unfortunately doesn't let me look at the data the way a text
dump does, but I would have thought that a binary dump might be less prone to
data corruption than a text one. Perhaps not, or maybe this was unavoidable.
I don't know of any PG tools to fix the character set; I haven't run across
anything like that in Google yet. But Torben sent me an email suggesting I try
the Linux utility recode, which seems to be found here:
http://recode.progiciels-bpi.ca/index.html. If nothing else works, I can try
running this on a text dump and see if it fixes the bad data.
thanks,
janine
On Dec 2, 2010, at 2:08 AM, Bernhard van Woerden wrote:
Just found the same as Russell, it started life as a single byte but needs
some conversion to match it with the correct unicode character which can then
be stored in utf-8.
What happens if you ask AOLserver to output iso-8859-1 does it convert 0xc2
0x92 to 0x92 ?
ns_param OutputCharset iso-8859-1
On 2 December 2010 07:39, Janine Ohmer jan...@furfly.net wrote:
On Dec 1, 2010, at 11:16 AM, Bernhard van Woerden wrote:
Can you check the byte sequence of the string in the db that's causing a
problem.
Sort of... the column in question is of type text, and the only function I
can find that will convert from text to bytea is decode, not encode. So
here's what I did:
select decode(answer_3, 'escape') from public_places where
public_place_id=1012;
The interesting part of the result is Park\302\222s, which is supposed to
be Park's.
I then ran
select encode('\\302\\222':bytea, 'hex');
and got c292.
Google tells me that this is the same as U+0092, which is defined simply as
control, or private use 2.
This doesn't seem very helpful, but it's possible that my initial assumptions
on how to do this were wrong and I'm really just trying to look up garbage
data. :)
Bernhard, does this give you any clues?
thanks,
janine
select encode(column_name::bytea,'hex') from ...
or use get_byte to isolate
On 1 December 2010 18:44, Janine Ohmer jan...@furfly.net wrote:
Hi Brian,
The old system isn't accessible anymore (something went very wrong and even
though it's RAID and all, fsck ended up removing a bunch of files and making
the system unbootable). But from looking at the files in etc from the
backups I can deduce that LANG was set to the same thing as it is on the new
system, en_US.UTF-8.
I had tried setting these:
ns_paramHackContentType 1
ns_paramURLCharset utf-8
ns_paramOutputCharset utf-8
ns_paramHttpOpenCharset utf-8
Which did not work. I will try adding DefaultCharset just in case that is
the key, but I'm not holding my breath...
There is an environment variable I can set, PGCLIENTENCODING, which will let
me specify what encoding to use for the client, so I can add that to
nsd-postgres. But I'm not sure what to set it to - everything's already in
UTF8, so there shouldn't *be* any encoding issues (famous last words, I know
:).
I can't restart the site during the day (very picky client) so will not be
able to try anything until late afternoon.
thanks,
janine
On Dec 1, 2010, at 3:32 AM, Fenton, Brian wrote:
Hi Janine
this may not be the answer but it can't do any harm. You should consider
adding these to the AOLserver tcl file:
ns_param HackContentType1
ns_param DefaultCharsetutf-8
ns_param HttpOpenCharsetutf-8
ns_param OutputCharset utf-8
ns_param URLCharsetutf-8
Also, rule out any OS differences by checking your locale, LANG etc. Does
Postgres have an equivalent to Oracle's NLS_LANG? Hopefully you won't have
to dive into codepages and all that stuff!
best wishes
Brian Fenton
From: AOLserver Discussion [aolser...@listserv.aol.com] On Behalf Of
Janine Ohmer [jan...@furfly.net]
Sent: 01 December 2010 05:19
To: AOLSERVER@LISTSERV.AOL.COM
Subject: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
Me again... still working out the last details on those sites I had to
move.
We're having some issues with characters like apostrophes and dashes
either disappearing (Safari) or showing up as garbage characters (Firefox).
I'm using the same version of Postgres and the same codebase. The
databases on both systems use the UNICODE encoding. The main thing that's
different is the AOLserver version. There isn't anything about charset in
either the old or the new config file
Re: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
Changing the charset seems to have fixed it, after a spot check of some pages I
know had problems. If the client doesn't find more, we'll be in good shape.
Thanks, everyone!
janine
On Dec 2, 2010, at 2:08 AM, Bernhard van Woerden wrote:
Just found the same as Russell, it started life as a single byte but needs
some conversion to match it with the correct unicode character which can then
be stored in utf-8.
What happens if you ask AOLserver to output iso-8859-1 does it convert 0xc2
0x92 to 0x92 ?
ns_param OutputCharset iso-8859-1
On 2 December 2010 07:39, Janine Ohmer jan...@furfly.net wrote:
On Dec 1, 2010, at 11:16 AM, Bernhard van Woerden wrote:
Can you check the byte sequence of the string in the db that's causing a
problem.
Sort of... the column in question is of type text, and the only function I
can find that will convert from text to bytea is decode, not encode. So
here's what I did:
select decode(answer_3, 'escape') from public_places where
public_place_id=1012;
The interesting part of the result is Park\302\222s, which is supposed to
be Park's.
I then ran
select encode('\\302\\222':bytea, 'hex');
and got c292.
Google tells me that this is the same as U+0092, which is defined simply as
control, or private use 2.
This doesn't seem very helpful, but it's possible that my initial assumptions
on how to do this were wrong and I'm really just trying to look up garbage
data. :)
Bernhard, does this give you any clues?
thanks,
janine
select encode(column_name::bytea,'hex') from ...
or use get_byte to isolate
On 1 December 2010 18:44, Janine Ohmer jan...@furfly.net wrote:
Hi Brian,
The old system isn't accessible anymore (something went very wrong and even
though it's RAID and all, fsck ended up removing a bunch of files and making
the system unbootable). But from looking at the files in etc from the
backups I can deduce that LANG was set to the same thing as it is on the new
system, en_US.UTF-8.
I had tried setting these:
ns_paramHackContentType 1
ns_paramURLCharset utf-8
ns_paramOutputCharset utf-8
ns_paramHttpOpenCharset utf-8
Which did not work. I will try adding DefaultCharset just in case that is
the key, but I'm not holding my breath...
There is an environment variable I can set, PGCLIENTENCODING, which will let
me specify what encoding to use for the client, so I can add that to
nsd-postgres. But I'm not sure what to set it to - everything's already in
UTF8, so there shouldn't *be* any encoding issues (famous last words, I know
:).
I can't restart the site during the day (very picky client) so will not be
able to try anything until late afternoon.
thanks,
janine
On Dec 1, 2010, at 3:32 AM, Fenton, Brian wrote:
Hi Janine
this may not be the answer but it can't do any harm. You should consider
adding these to the AOLserver tcl file:
ns_param HackContentType1
ns_param DefaultCharsetutf-8
ns_param HttpOpenCharsetutf-8
ns_param OutputCharset utf-8
ns_param URLCharsetutf-8
Also, rule out any OS differences by checking your locale, LANG etc. Does
Postgres have an equivalent to Oracle's NLS_LANG? Hopefully you won't have
to dive into codepages and all that stuff!
best wishes
Brian Fenton
From: AOLserver Discussion [aolser...@listserv.aol.com] On Behalf Of
Janine Ohmer [jan...@furfly.net]
Sent: 01 December 2010 05:19
To: AOLSERVER@LISTSERV.AOL.COM
Subject: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
Me again... still working out the last details on those sites I had to
move.
We're having some issues with characters like apostrophes and dashes
either disappearing (Safari) or showing up as garbage characters (Firefox).
I'm using the same version of Postgres and the same codebase. The
databases on both systems use the UNICODE encoding. The main thing that's
different is the AOLserver version. There isn't anything about charset in
either the old or the new config file.
Has anyone done this conversion (in recent memory, that is :) and knows
what the problem might be?
thanks,
janine
---
Janine Ohmer (formerly Sisk)
President/CEO of furfly, LLC
503-693-6407
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
---
Janine Ohmer (formerly Sisk)
President/CEO
Re: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
Hi Brian, The old system isn't accessible anymore (something went very wrong and even though it's RAID and all, fsck ended up removing a bunch of files and making the system unbootable). But from looking at the files in etc from the backups I can deduce that LANG was set to the same thing as it is on the new system, en_US.UTF-8. I had tried setting these: ns_paramHackContentType 1 ns_paramURLCharset utf-8 ns_paramOutputCharset utf-8 ns_paramHttpOpenCharset utf-8 Which did not work. I will try adding DefaultCharset just in case that is the key, but I'm not holding my breath... There is an environment variable I can set, PGCLIENTENCODING, which will let me specify what encoding to use for the client, so I can add that to nsd-postgres. But I'm not sure what to set it to - everything's already in UTF8, so there shouldn't *be* any encoding issues (famous last words, I know :). I can't restart the site during the day (very picky client) so will not be able to try anything until late afternoon. thanks, janine On Dec 1, 2010, at 3:32 AM, Fenton, Brian wrote: Hi Janine this may not be the answer but it can't do any harm. You should consider adding these to the AOLserver tcl file: ns_param HackContentType1 ns_param DefaultCharsetutf-8 ns_param HttpOpenCharsetutf-8 ns_param OutputCharset utf-8 ns_param URLCharsetutf-8 Also, rule out any OS differences by checking your locale, LANG etc. Does Postgres have an equivalent to Oracle's NLS_LANG? Hopefully you won't have to dive into codepages and all that stuff! best wishes Brian Fenton From: AOLserver Discussion [aolser...@listserv.aol.com] On Behalf Of Janine Ohmer [jan...@furfly.net] Sent: 01 December 2010 05:19 To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10? Me again... still working out the last details on those sites I had to move. We're having some issues with characters like apostrophes and dashes either disappearing (Safari) or showing up as garbage characters (Firefox). I'm using the same version of Postgres and the same codebase. The databases on both systems use the UNICODE encoding. The main thing that's different is the AOLserver version. There isn't anything about charset in either the old or the new config file. Has anyone done this conversion (in recent memory, that is :) and knows what the problem might be? thanks, janine --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
On Dec 1, 2010, at 11:16 AM, Bernhard van Woerden wrote:
Can you check the byte sequence of the string in the db that's causing a
problem.
Sort of... the column in question is of type text, and the only function I can
find that will convert from text to bytea is decode, not encode. So here's
what I did:
select decode(answer_3, 'escape') from public_places where public_place_id=1012;
The interesting part of the result is Park\302\222s, which is supposed to be
Park's.
I then ran
select encode('\\302\\222':bytea, 'hex');
and got c292.
Google tells me that this is the same as U+0092, which is defined simply as
control, or private use 2.
This doesn't seem very helpful, but it's possible that my initial assumptions
on how to do this were wrong and I'm really just trying to look up garbage
data. :)
Bernhard, does this give you any clues?
thanks,
janine
select encode(column_name::bytea,'hex') from ...
or use get_byte to isolate
On 1 December 2010 18:44, Janine Ohmer jan...@furfly.net wrote:
Hi Brian,
The old system isn't accessible anymore (something went very wrong and even
though it's RAID and all, fsck ended up removing a bunch of files and making
the system unbootable). But from looking at the files in etc from the
backups I can deduce that LANG was set to the same thing as it is on the new
system, en_US.UTF-8.
I had tried setting these:
ns_paramHackContentType 1
ns_paramURLCharset utf-8
ns_paramOutputCharset utf-8
ns_paramHttpOpenCharset utf-8
Which did not work. I will try adding DefaultCharset just in case that is
the key, but I'm not holding my breath...
There is an environment variable I can set, PGCLIENTENCODING, which will let
me specify what encoding to use for the client, so I can add that to
nsd-postgres. But I'm not sure what to set it to - everything's already in
UTF8, so there shouldn't *be* any encoding issues (famous last words, I know
:).
I can't restart the site during the day (very picky client) so will not be
able to try anything until late afternoon.
thanks,
janine
On Dec 1, 2010, at 3:32 AM, Fenton, Brian wrote:
Hi Janine
this may not be the answer but it can't do any harm. You should consider
adding these to the AOLserver tcl file:
ns_param HackContentType1
ns_param DefaultCharsetutf-8
ns_param HttpOpenCharsetutf-8
ns_param OutputCharset utf-8
ns_param URLCharsetutf-8
Also, rule out any OS differences by checking your locale, LANG etc. Does
Postgres have an equivalent to Oracle's NLS_LANG? Hopefully you won't have
to dive into codepages and all that stuff!
best wishes
Brian Fenton
From: AOLserver Discussion [aolser...@listserv.aol.com] On Behalf Of Janine
Ohmer [jan...@furfly.net]
Sent: 01 December 2010 05:19
To: AOLSERVER@LISTSERV.AOL.COM
Subject: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
Me again... still working out the last details on those sites I had to move.
We're having some issues with characters like apostrophes and dashes either
disappearing (Safari) or showing up as garbage characters (Firefox).
I'm using the same version of Postgres and the same codebase. The
databases on both systems use the UNICODE encoding. The main thing that's
different is the AOLserver version. There isn't anything about charset in
either the old or the new config file.
Has anyone done this conversion (in recent memory, that is :) and knows
what the problem might be?
thanks,
janine
---
Janine Ohmer (formerly Sisk)
President/CEO of furfly, LLC
503-693-6407
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the
Subject: field of your email blank.
---
Janine Ohmer (formerly Sisk)
President/CEO of furfly, LLC
503-693-6407
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
--
AOLserver - http://www.aolserver.com/
To Remove yourself from this list, simply send an email to
lists...@listserv.aol.com with the
body of SIGNOFF AOLSERVER in the email message. You can leave the Subject:
field of your email blank.
---
Janine Ohmer (formerly Sisk)
President/CEO of furfly, LLC
503-693-6407
--
AOLserver - http://www.aolserver.com
Re: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
I tried adding DefaultCharset to the config file, to no effect. I also tried setting PGCLIENTENCODING to UTF-8 in nsd-postgres, and that didn't change anything either. Just for grins I tried changing it to WIN, since the assumption is that these are Windows smart quotes, but that just made the garbage character disappear altogether. janine On Dec 1, 2010, at 10:44 AM, Janine Ohmer wrote: Hi Brian, The old system isn't accessible anymore (something went very wrong and even though it's RAID and all, fsck ended up removing a bunch of files and making the system unbootable). But from looking at the files in etc from the backups I can deduce that LANG was set to the same thing as it is on the new system, en_US.UTF-8. I had tried setting these: ns_paramHackContentType 1 ns_paramURLCharset utf-8 ns_paramOutputCharset utf-8 ns_paramHttpOpenCharset utf-8 Which did not work. I will try adding DefaultCharset just in case that is the key, but I'm not holding my breath... There is an environment variable I can set, PGCLIENTENCODING, which will let me specify what encoding to use for the client, so I can add that to nsd-postgres. But I'm not sure what to set it to - everything's already in UTF8, so there shouldn't *be* any encoding issues (famous last words, I know :). I can't restart the site during the day (very picky client) so will not be able to try anything until late afternoon. thanks, janine On Dec 1, 2010, at 3:32 AM, Fenton, Brian wrote: Hi Janine this may not be the answer but it can't do any harm. You should consider adding these to the AOLserver tcl file: ns_param HackContentType1 ns_param DefaultCharsetutf-8 ns_param HttpOpenCharsetutf-8 ns_param OutputCharset utf-8 ns_param URLCharsetutf-8 Also, rule out any OS differences by checking your locale, LANG etc. Does Postgres have an equivalent to Oracle's NLS_LANG? Hopefully you won't have to dive into codepages and all that stuff! best wishes Brian Fenton From: AOLserver Discussion [aolser...@listserv.aol.com] On Behalf Of Janine Ohmer [jan...@furfly.net] Sent: 01 December 2010 05:19 To: AOLSERVER@LISTSERV.AOL.COM Subject: [AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10? Me again... still working out the last details on those sites I had to move. We're having some issues with characters like apostrophes and dashes either disappearing (Safari) or showing up as garbage characters (Firefox). I'm using the same version of Postgres and the same codebase. The databases on both systems use the UNICODE encoding. The main thing that's different is the AOLserver version. There isn't anything about charset in either the old or the new config file. Has anyone done this conversion (in recent memory, that is :) and knows what the problem might be? thanks, janine --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Charset differences between 3.3+ad13 and 4.0.10?
Me again... still working out the last details on those sites I had to move. We're having some issues with characters like apostrophes and dashes either disappearing (Safari) or showing up as garbage characters (Firefox). I'm using the same version of Postgres and the same codebase. The databases on both systems use the UNICODE encoding. The main thing that's different is the AOLserver version. There isn't anything about charset in either the old or the new config file. Has anyone done this conversion (in recent memory, that is :) and knows what the problem might be? thanks, janine --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Site hanging
Thanks Gustaf! That very well could be the problem. I fixed it by increasing the number of database connections, which might just have the effect of preventing me from getting to this point. janine On Nov 7, 2010, at 11:52 PM, Gustaf Neumann wrote: Dear Janine The symptoms sound similar to bug #1615787 https://sourceforge.net/tracker/?func=detailaid=1615787group_id=3152atid=103152 this bug is fixed in the 4.0 branch (not sure, if you are using a version before this fix) and in aolserver 4.5.1 best regards -gustaf neumann 2007-10-19 Gustaf Neumann neum...@wu-wien.ac.at * nsd/driver.c: * nsd/queue.c: Better fix for BUG #1615787 in aolserver 4.5. Although the original fix apparently helped for www.openacs.org, it appears to simply change the timing, but did not fix the problem itself. The problem was that under limited resource configurations (e.g. maxthreads 5, maxconnections 3) and heavy traffic, incoming requests were queued but not processed in new connection threads. The situation was especially bad, when the number of queued requests was larger than maxconnections, since after processing n requests the server was idling with a high number of queued requests. 2006-12-15 Dossy Shiobara do...@panoptic.com * nsd/driver.c: [BUG #1615787] Fix deadlock bug when driver thread stalls under specific conditions. Thanks to Jeff Rogers and Gustaf Neumann. On 06.11.10 17:23, Janine Ohmer wrote: Another day, another problem. ;) This time it's a somewhat more recent (but still old) OpenACS site, that has always run under AOLserver 4.0.10. Overnight it got hit pretty hard by the Yahoo crawler, and it has stopped responding. Netstat shows 117 connections, 78 in CLOSE_WAIT state and the rest in SYN_RECV. This site would hang up very occasionally on my old system; I would just restart it, since it didn't happen often enough to worry about. But it has only been running for about 16 hours, and that is way too soon to be doing this. The new system is more powerful than the old one, and runs a much newer 64 bit Linux, vs a very old 32 bit one. It's entirely possible that my configuration (connections) is not ideal, but it hasn't changed so I would expect it to run at least as well on the new system as it did on the old. For now I will just restart it but I'm sure this will happen again. Any suggestions on how to troubleshoot? I have never had much luck over the years figuring this out; I usually just end up installing keepalive to restart, but that is really not ideal. thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Has anyone built nsd3.3+ad13 on 64-bit Linux?
Thanks, Gustaf. I finally abandoned this and got it working (as far as I can tell, anyway) under 4.0.10. janine On Nov 6, 2010, at 3:02 AM, Gustaf Neumann wrote: If you want to compile the vintage version, i would will have to compile either with -m32 or to backport the 64bit fixes of the last 10 years, which are most likely necessary for aolserver33 as well. Search e.g. through http://www.ohloh.net/p/aolserver/commits?query=64-bitcommit=Update http://www.ohloh.net/p/aolserver/commits?query=64-bitcommit=Update and similar search patterns best regards -gustaf neumann On 05.11.10 23:03, Janine Ohmer wrote: Followup to this... The original build error was: (cd /usr/local/src/aol33+ad13/aolserver/tcl7.6/unix; make CFLAGS='-I../include -D_REENTRANT=1 -DNDEBUG=1 -g -fPIC -Wall -Wno-unused -mcpu=x86_64 -DHAVE_CMMSG=1 -DUSE_FIONREAD=1 -DHAVE_COND_EINTR=1' libtcl7.6.a) make[2]: Entering directory `/usr/local/src/aol33+ad13/aolserver/tcl7.6/unix' gcc -c -I../include -D_REENTRANT=1 -DNDEBUG=1 -g -fPIC -Wall -Wno-unused -mcpu=x86_64 -DHAVE_CMMSG=1 -DUSE_FIONREAD=1 -DHAVE_COND_EINTR=1 -I./../generic -I. -DHAVE_UNISTD_H=1 -DHAVE_SYS_TIME_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_TM_GMTOFF=1 -DHAVE_TIMEZONE_VAR=1 -DSTDC_HEADERS=1 -DNEED_MATHERR=1 -DHAVE_SYS_IOCTL_H=1 -DTCL_SHLIB_EXT=\.so\ ./../generic/panic.c `-mcpu=' is deprecated. Use `-mtune=' or '-march=' instead. ./../generic/panic.c:1: error: bad value (x86_64) for -mtune= switch I fixed this with: #CFLAGS+=$(GCCOPT) -mcpu=$(shell uname -m) CFLAGS+=$(GCCOPT) -m64 -mtune=generic in aolserver/include/Makefile.global. Now everything builds, but it starts to run and then seg faults. *sigh* I also tried CFLAGS+=$(GCCOPT) -O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic (everything from the new Makefile.global that the old one doesn't have) But it still seg faults. janine On Nov 5, 2010, at 1:58 PM, Janine Ohmer wrote: I just posted the following to at openacs.org, and wanted to see if anyone who lurks here but not there might have any ideas: Due to unforeseen circumstances (server death :) I'm moving some very old sites to a new system running 64 bit Redhat (old system was 32 bit). A couple of these sites were running under nsd3.3+ad13. To my great surprise most of nsd worked without having to be rebuilt, but the postgres driver did not work and I was unsuccessful at building a new one. Not wanting to invest a lot of time on an ancient webserver, I went ahead with the move and set the 3.2.5 sites up under AOLserver 4.0.10, also quite old but the client didn't want me to upgrade anything more than was absolutely necessary. Unfortunately, they don't work. There are no errors in the error log on startup but the page load dies with: invalid command name template::adp_parse There don't seem to be many people reporting this error. I've seen this thread: http://openacs.org/forums/message-view?message_id=26808 But have double-checked the config file (several times!) and Home is set correctly. However, this makes me think that the problem is probably due to changes to AOLserver's module/tcl scripts. So does anyone know how to either a) run 3.2.5 under AOLserver 4 or b) get nsd3.3+ad13 built and working on 64 bit Linux? Thanks in advance! --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email tolists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Site hanging
Another day, another problem. ;) This time it's a somewhat more recent (but still old) OpenACS site, that has always run under AOLserver 4.0.10. Overnight it got hit pretty hard by the Yahoo crawler, and it has stopped responding. Netstat shows 117 connections, 78 in CLOSE_WAIT state and the rest in SYN_RECV. This site would hang up very occasionally on my old system; I would just restart it, since it didn't happen often enough to worry about. But it has only been running for about 16 hours, and that is way too soon to be doing this. The new system is more powerful than the old one, and runs a much newer 64 bit Linux, vs a very old 32 bit one. It's entirely possible that my configuration (connections) is not ideal, but it hasn't changed so I would expect it to run at least as well on the new system as it did on the old. For now I will just restart it but I'm sure this will happen again. Any suggestions on how to troubleshoot? I have never had much luck over the years figuring this out; I usually just end up installing keepalive to restart, but that is really not ideal. thanks, janine -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Unable to bind to port 80
I'm installing a group of old OpenACS sites on a managed server at Rackspace and have run into a problem I just can't seem to figure out. The two sites that need to run on port 80 are unable to bind to it - I get permission denied errors. I've checked all the usual things - the sites are being started by daemontools, so the prebind is done by the root user, and I do have the -b ip:80 in both run scripts. Judging from Google, those are the two most common errors. I'm using nsd 4.0.10; I know that's ancient history but the client didn't want me to upgrade anything and it was working fine on the old system. This is the first time I've used it on a 64 bit system (Redhat), for what that's worth. I did build it from scratch on the new system, of course. When I first started working on this Apache had a Listen 80 statement in the config file and I thought that was the culprit, but it has now been changed to specify its own (different) IP address and restarted several times. So that's not it. I don't see anything wrong in my setup and I also don't see anything suspicious in netstat that looks like the port is in use. I've asked the Rackspace folks to look at the system and confirm for me that nothing else has grabbed the port, but I'm sure not seeing it. Are there other gotchas I can check for, or known problems I'm unaware of? thanks, janine --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
[AOLSERVER] Has anyone built nsd3.3+ad13 on 64-bit Linux?
I just posted the following to at openacs.org, and wanted to see if anyone who lurks here but not there might have any ideas: Due to unforeseen circumstances (server death :) I'm moving some very old sites to a new system running 64 bit Redhat (old system was 32 bit). A couple of these sites were running under nsd3.3+ad13. To my great surprise most of nsd worked without having to be rebuilt, but the postgres driver did not work and I was unsuccessful at building a new one. Not wanting to invest a lot of time on an ancient webserver, I went ahead with the move and set the 3.2.5 sites up under AOLserver 4.0.10, also quite old but the client didn't want me to upgrade anything more than was absolutely necessary. Unfortunately, they don't work. There are no errors in the error log on startup but the page load dies with: invalid command name template::adp_parse There don't seem to be many people reporting this error. I've seen this thread: http://openacs.org/forums/message-view?message_id=26808 But have double-checked the config file (several times!) and Home is set correctly. However, this makes me think that the problem is probably due to changes to AOLserver's module/tcl scripts. So does anyone know how to either a) run 3.2.5 under AOLserver 4 or b) get nsd3.3+ad13 built and working on 64 bit Linux? Thanks in advance! --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
Re: [AOLSERVER] Has anyone built nsd3.3+ad13 on 64-bit Linux?
Followup to this... The original build error was: (cd /usr/local/src/aol33+ad13/aolserver/tcl7.6/unix; make CFLAGS='-I../include -D_REENTRANT=1 -DNDEBUG=1 -g -fPIC -Wall -Wno-unused -mcpu=x86_64 -DHAVE_CMMSG=1 -DUSE_FIONREAD=1 -DHAVE_COND_EINTR=1' libtcl7.6.a) make[2]: Entering directory `/usr/local/src/aol33+ad13/aolserver/tcl7.6/unix' gcc -c -I../include -D_REENTRANT=1 -DNDEBUG=1 -g -fPIC -Wall -Wno-unused -mcpu=x86_64 -DHAVE_CMMSG=1 -DUSE_FIONREAD=1 -DHAVE_COND_EINTR=1 -I./../generic -I. -DHAVE_UNISTD_H=1 -DHAVE_SYS_TIME_H=1 -DTIME_WITH_SYS_TIME=1 -DHAVE_TM_ZONE=1 -DHAVE_TM_GMTOFF=1 -DHAVE_TIMEZONE_VAR=1 -DSTDC_HEADERS=1 -DNEED_MATHERR=1 -DHAVE_SYS_IOCTL_H=1 -DTCL_SHLIB_EXT=\.so\ ./../generic/panic.c `-mcpu=' is deprecated. Use `-mtune=' or '-march=' instead. ./../generic/panic.c:1: error: bad value (x86_64) for -mtune= switch I fixed this with: #CFLAGS+=$(GCCOPT) -mcpu=$(shell uname -m) CFLAGS+=$(GCCOPT) -m64 -mtune=generic in aolserver/include/Makefile.global. Now everything builds, but it starts to run and then seg faults. *sigh* I also tried CFLAGS+=$(GCCOPT) -O2 -pipe -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic (everything from the new Makefile.global that the old one doesn't have) But it still seg faults. janine On Nov 5, 2010, at 1:58 PM, Janine Ohmer wrote: I just posted the following to at openacs.org, and wanted to see if anyone who lurks here but not there might have any ideas: Due to unforeseen circumstances (server death :) I'm moving some very old sites to a new system running 64 bit Redhat (old system was 32 bit). A couple of these sites were running under nsd3.3+ad13. To my great surprise most of nsd worked without having to be rebuilt, but the postgres driver did not work and I was unsuccessful at building a new one. Not wanting to invest a lot of time on an ancient webserver, I went ahead with the move and set the 3.2.5 sites up under AOLserver 4.0.10, also quite old but the client didn't want me to upgrade anything more than was absolutely necessary. Unfortunately, they don't work. There are no errors in the error log on startup but the page load dies with: invalid command name template::adp_parse There don't seem to be many people reporting this error. I've seen this thread: http://openacs.org/forums/message-view?message_id=26808 But have double-checked the config file (several times!) and Home is set correctly. However, this makes me think that the problem is probably due to changes to AOLserver's module/tcl scripts. So does anyone know how to either a) run 3.2.5 under AOLserver 4 or b) get nsd3.3+ad13 built and working on 64 bit Linux? Thanks in advance! --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank. --- Janine Ohmer (formerly Sisk) President/CEO of furfly, LLC 503-693-6407 -- AOLserver - http://www.aolserver.com/ To Remove yourself from this list, simply send an email to lists...@listserv.aol.com with the body of SIGNOFF AOLSERVER in the email message. You can leave the Subject: field of your email blank.
