Hi William and Orion, I have solved this problem by explicitly setting LANG in my script, as suggested by both of you. Previously, I thought SGE would use the shell environment settings automatically, but obviously it's not the case.
By the way, if anyone sees "export: command not found" in their STDERR, try using -S /path/to/shell for qsub when submitting jobs. William, I have only a very small scale cluster so accessing to each slave node directly is not much a pain to me. Many thanks, Guchun On 7 October 2012 14:27, William Hay <[email protected]> wrote: > On 6 October 2012 13:00, Guchun Zhang <[email protected]> wrote: > > Hi Orion, > > > > Many thanks for your reply. > > > > But, I am not quite sure that is the real cause of the problem I have. > > I have already checked LANG on all the slave nodes, which is all set to > > How? Submiting a job that does echo $LANG is the only way I'd trust. > Environment variables such as LANG are not necessarily the same for > all processes on a node. You need to find out what it is set to for > processes started by Grid Engine. > > Most likely your batch jobs are not run in a login shell so don't get > LANG set and default to the 'C' locale. I'd try explicitly > setting(and exporting) LANG in your job script to fix it. Possibly > check the values of the various LC_* variables as well. > > > UTF8. I also tried some simple jobs which only contain basic shell > > commands, like cat and cp, and the output is still in UTF8. Moreover, I > > cat and cp don't (normally) worry about character encoding at all they > just move bytes around. These are interpreted when you view them > later. > > > ssh-ed to the slave nodes and ran the processing code in the terminal. > > The output files are also still in UTF8. > > > > So I am really puzzled. > > > > Regards, > > > > Guchun > > > > Sent from my Windows Phone > > From: Orion Poplawski > > Sent: 05/10/2012 16:50 > > To: Guchun Zhang; [email protected] > > Subject: Re: [gridengine users] Unwanted character encoding change > > On 10/05/2012 09:45 AM, Guchun Zhang wrote: > >> Hi, > >> > >> I am processing some UTF-8 coded files. The file processing code runs > fine > >> locally on the slave nodes. However, if I submit a job to run the file > >> processing, the output files will be automatically coded in US-ASCII, > which, > >> well, is unwanted. > >> > >> My question is what could be the problem? Any lead is appreciated. > > > > > > My guess is that the LANG environment variable is getting set > differently. > > > > > > -- > > Orion Poplawski > > Technical Manager 303-415-9701 x222 > > NWRA, Boulder Office FAX: 303-415-9702 > > 3380 Mitchell Lane [email protected] > > Boulder, CO 80301 http://www.nwra.com > > _______________________________________________ > > users mailing list > > [email protected] > > https://gridengine.org/mailman/listinfo/users > > > > >
_______________________________________________ users mailing list [email protected] https://gridengine.org/mailman/listinfo/users
