On Thu, 26 Nov 2009 00:47:00 +0000
Hugo Monteiro <hugo.monte...@fct.unl.pt> wrote:
> Steve et al,
>
Hallo Hugo,
> I noticed that the templates are using html special chars for
> diacritics. Do you think that we could make the build process go through
> the templates and automagically translate those characters into their
> correspondent html special chars? .. That might speed up the translation
> process and increase submissions. Not only that, it would make template
> maintenance a bit easier.
>
> Any thoughts?
>
ach Hugo. If time would not be such a big issue then I could code whatever is
needed to make things go faster, more easy, etc... but there is so much things
to do and so less time to make all this happen. If any one want's to do that
character transformation during build time then I have nothing to say against
it. But going now to sit down and code that is something I will not do. When we
forked DSPAM (or better to say: when we took over) a lot of the users where
claiming to redo the WebUI in something else then Perl. So far nothing has
emerged in that direction. We still are using Perl and we hack that thing to
death with patches, knowing that we want to get rid of it.
If you need a quick and dirty way of replacing those special characters then
you could use GNU sed to do the job:
----------------------
sed
"s:\d160:\ :g;s:\d161:\¡:g;s:\d162:\¢:g;s:\d163:\£:g;s:\d164:\¤:g;s:\d165:\¥:g;s:\d166:\¦:g;s:\d167:\§:g;s:\d168:\¨:g;s:\d169:\©:g;s:\d170:\ª:g;s:\d171:\«:g;s:\d172:\¬:g;s:\d173:\­:g;s:\d174:\®:g;s:\d175:\¯:g;s:\d176:\°:g;s:\d177:\±:g;s:\d178:\²:g;s:\d179:\³:g;s:\d180:\´:g;s:\d181:\µ:g;s:\d182:\¶:g;s:\d183:\·:g;s:\d184:\¸:g;s:\d185:\¹:g;s:\d186:\º:g;s:\d187:\»:g;s:\d188:\¼:g;s:\d189:\½:g;s:\d190:\¾:g;s:\d191:\¿:g;s:\d192:\À:g;s:\d193:\Á:g;s:\d194:\Â:g;s:\d195:\Ã:g;s:\d196:\Ä:g;s:\d197:\Å:g;s:\d198:\Æ:g;s:\d199:\Ç:g;s:\d200:\È:g;s:\d201:\É:g;s:\d202:\Ê:g;s:\d203:\Ë:g;s:\d204:\Ì:g;s:\d205:\Í:g;s:\d206:\Î:g;s:\d207:\Ï:g;s:\d208:\Ð:g;s:\d209:\Ñ:g;s:\d210:\Ò:g;s:\d211:\Ó:g;s:\d212:\Ô:g;s:\d213:\Õ:g;s:\d214:\Ö:g;s:\d215:\×:g;s:\d216:\Ø:g;s:\d217:\Ù:g;s:\d218:\Ú:g;s:\d219:\Û:g;s:\d220:\Ü:g;s:\d221:\Ý:g;s:\d222:\Þ:g;s:\d223:\ß:g;s:\d224:\à:g;s:\d225:\á:g;s:\d226:\â:g;s:\d227:\ã:g;s:\d228:\ä:g;s:\d229:\å:g;s:\d230:\æ:g;s:\d231:\ç:g;s:\d232:\è:g;s:\d233:\é:g;s:\d234:\ê:g;s:\d235:\ë:g;s:\d236:\ì:g;s:\d237:\í:g;s:\d238:\î:g;s:\d239:\ï:g;s:\d240:\ð:g;s:\d241:\ñ:g;s:\d242:\ò:g;s:\d243:\ó:g;s:\d244:\ô:g;s:\d245:\õ:g;s:\d246:\ö:g;s:\d247:\÷:g;s:\d248:\ø:g;s:\d249:\ù:g;s:\d250:\ú:g;s:\d251:\û:g;s:\d252:\ü:g;s:\d253:\ý:g;s:\d254:\þ:g;s:\d255:\ÿ:g;"
----------------------
I have made that one liner by quickly crafting a regexp out of the code I used
in decode.c. I have not taken any character afte ASCII 255 and I have avoided
to use characters that could potentialy be found in normal HTML tags (stuff
like <, >, &, etc).
The sed one liner does not take care where the caracter is found. So it could
potentialy destroy valid HTML. I just have no time to make a rock solid
translation module. That would be to time intensive.
If you want to quickly check what characters are replaced then do fire up your
shell and execute this here:
----------------------
awk 'BEGIN {for(i=32;i<256;i++) printf "%3d %c\n",i,i}'|sed
"s:\d160:\0\t\ :g;s:\d161:\0\t\¡:g;s:\d162:\0\t\¢:g;s:\d163:\0\t\£:g;s:\d164:\0\t\¤:g;s:\d165:\0\t\¥:g;s:\d166:\0\t\¦:g;s:\d167:\0\t\§:g;s:\d168:\0\t\¨:g;s:\d169:\0\t\©:g;s:\d170:\0\t\ª:g;s:\d171:\0\t\«:g;s:\d172:\0\t\¬:g;s:\d173:\0\t\­:g;s:\d174:\0\t\®:g;s:\d175:\0\t\¯:g;s:\d176:\0\t\°:g;s:\d177:\0\t\±:g;s:\d178:\0\t\²:g;s:\d179:\0\t\³:g;s:\d180:\0\t\´:g;s:\d181:\0\t\µ:g;s:\d182:\0\t\¶:g;s:\d183:\0\t\·:g;s:\d184:\0\t\¸:g;s:\d185:\0\t\¹:g;s:\d186:\0\t\º:g;s:\d187:\0\t\»:g;s:\d188:\0\t\¼:g;s:\d189:\0\t\½:g;s:\d190:\0\t\¾:g;s:\d191:\0\t\¿:g;s:\d192:\0\t\À:g;s:\d193:\0\t\Á:g;s:\d194:\0\t\Â:g;s:\d195:\0\t\Ã:g;s:\d196:\0\t\Ä:g;s:\d197:\0\t\Å:g;s:\d198:\0\t\Æ:g;s:\d199:\0\t\Ç:g;s:\d200:\0\t\È:g;s:\d201:\0\t\É:g;s:\d202:\0\t\Ê:g;s:\d203:\0\t\Ë:g;s:\d204:\0\t\Ì:g;s:\d205:\0\t\Í:g;s:\d206:\0\t\Î:g;s:\d207:\0\t\Ï:g;s:\d208:\0\t\Ð:g;s:\d209:\0\t\Ñ:g;s:\d210:\0\t\Ò:g;s:\d211:\0\t\Ó:g;s:\d212:\0\t\Ô:g;s:\d213:\0\t\Õ:g;s:\d214:\0\t\Ö:g;s:\d215:\0\t\×:g;s:\d216:\0\t\Ø:g;s:\d217:\0\t\Ù:g;s:\d218:\0\t\Ú:g;s:\d219:\0\t\Û:g;s:\d220:\0\t\Ü:g;s:\d221:\0\t\Ý:g;s:\d222:\0\t\Þ:g;s:\d223:\0\t\ß:g;s:\d224:\0\t\à:g;s:\d225:\0\t\á:g;s:\d226:\0\t\â:g;s:\d227:\0\t\ã:g;s:\d228:\0\t\ä:g;s:\d229:\0\t\å:g;s:\d230:\0\t\æ:g;s:\d231:\0\t\ç:g;s:\d232:\0\t\è:g;s:\d233:\0\t\é:g;s:\d234:\0\t\ê:g;s:\d235:\0\t\ë:g;s:\d236:\0\t\ì:g;s:\d237:\0\t\í:g;s:\d238:\0\t\î:g;s:\d239:\0\t\ï:g;s:\d240:\0\t\ð:g;s:\d241:\0\t\ñ:g;s:\d242:\0\t\ò:g;s:\d243:\0\t\ó:g;s:\d244:\0\t\ô:g;s:\d245:\0\t\õ:g;s:\d246:\0\t\ö:g;s:\d247:\0\t\÷:g;s:\d248:\0\t\ø:g;s:\d249:\0\t\ù:g;s:\d250:\0\t\ú:g;s:\d251:\0\t\û:g;s:\d252:\0\t\ü:g;s:\d253:\0\t\ý:g;s:\d254:\0\t\þ:g;s:\d255:\0\t\ÿ:g;"
----------------------
The replaced characters are in the third column. I think anything after ASCII
160 is normaly not found in a place that can not be replaced with the HTML
character encoding. But I don't know 100% if this is valid? I just assumed that
it's that way.
Just to make you happy I checked the currently available templates to see if
there is potentialy any HTML or strings.pl file that should be changed. With
the following result:
----------------------
theia dspam # find ./webui/cgi-bin/templates/ -type f -name "*.html" -or -name
"strings.pl"|while read foo;do sed
"s:\d160:\ :g;s:\d161:\¡:g;s:\d162:\¢:g;s:\d163:\£:g;s:\d164:\¤:g;s:\d165:\¥:g;s:\d166:\¦:g;s:\d167:\§:g;s:\d168:\¨:g;s:\d169:\©:g;s:\d170:\ª:g;s:\d171:\«:g;s:\d172:\¬:g;s:\d173:\­:g;s:\d174:\®:g;s:\d175:\¯:g;s:\d176:\°:g;s:\d177:\±:g;s:\d178:\²:g;s:\d179:\³:g;s:\d180:\´:g;s:\d181:\µ:g;s:\d182:\¶:g;s:\d183:\·:g;s:\d184:\¸:g;s:\d185:\¹:g;s:\d186:\º:g;s:\d187:\»:g;s:\d188:\¼:g;s:\d189:\½:g;s:\d190:\¾:g;s:\d191:\¿:g;s:\d192:\À:g;s:\d193:\Á:g;s:\d194:\Â:g;s:\d195:\Ã:g;s:\d196:\Ä:g;s:\d197:\Å:g;s:\d198:\Æ:g;s:\d199:\Ç:g;s:\d200:\È:g;s:\d201:\É:g;s:\d202:\Ê:g;s:\d203:\Ë:g;s:\d204:\Ì:g;s:\d205:\Í:g;s:\d206:\Î:g;s:\d207:\Ï:g;s:\d208:\Ð:g;s:\d209:\Ñ:g;s:\d210:\Ò:g;s:\d211:\Ó:g;s:\d212:\Ô:g;s:\d213:\Õ:g;s:\d214:\Ö:g;s:\d215:\×:g;s:\d216:\Ø:g;s:\d217:\Ù:g;s:\d218:\Ú:g;s:\d219:\Û:g;s:\d220:\Ü:g;s:\d221:\Ý:g;s:\d222:\Þ:g;s:\d223:\ß:g;s:\d224:\à:g;s:\d225:\á:g;s:\d226:\â:g;s:\d227:\ã:g;s:\d228:\ä:g;s:\d229:\å:g;s:\d230:\æ:g;s:\d231:\ç:g;s:\d232:\è:g;s:\d233:\é:g;s:\d234:\ê:g;s:\d235:\ë:g;s:\d236:\ì:g;s:\d237:\í:g;s:\d238:\î:g;s:\d239:\ï:g;s:\d240:\ð:g;s:\d241:\ñ:g;s:\d242:\ò:g;s:\d243:\ó:g;s:\d244:\ô:g;s:\d245:\õ:g;s:\d246:\ö:g;s:\d247:\÷:g;s:\d248:\ø:g;s:\d249:\ù:g;s:\d250:\ú:g;s:\d251:\û:g;s:\d252:\ü:g;s:\d253:\ý:g;s:\d254:\þ:g;s:\d255:\ÿ:g;"
"${foo}">/tmp/_ds_$$;diff -Naur "${foo}" "/tmp/_ds_$$" >/dev/null 2>&1;if [
"${?}" != "0" ];then echo "File that probably needs to be fixed:
${foo}";fi;done;rm "/tmp/_ds_$$"
File that probably needs to be fixed:
./webui/cgi-bin/templates/de/nav_admin_preferences.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/de/nav_alerts.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/de/nav_preferences.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_admin_error.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_admin_preferences.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_admin_status.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_admin_user.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_alerts.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_analysis.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_error.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_fragment.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_history.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_performance.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_preferences.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_quarantine.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/he/nav_viewmessage.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_admin_error.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_admin_preferences.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_admin_status.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_admin_user.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_alerts.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_error.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_history.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_performance.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_preferences.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_quarantine.html
File that probably needs to be fixed:
./webui/cgi-bin/templates/ro/nav_viewmessage.html
File that probably needs to be fixed: ./webui/cgi-bin/templates/pt-br/strings.pl
theia dspam #
----------------------
The Hebrew version is out of scope since it uses dual byte unicode and I did
not take care of that. The de, ro and pt-br are probably the ones we should fix.
> Warm Regards, cause it's cold! =)
>
The heart is what needs to be warm. Anything else can be heaten up :)
> Hugo Monteiro.
>
Steve
> --
> ci.fct.unl.pt:~# cat .signature
>
> Hugo Monteiro
> Email : hugo.monte...@fct.unl.pt
> Telefone : +351 212948300 Ext.15307
> Web : http://hmonteiro.net
>
> Centro de Informática
> Faculdade de Ciências e Tecnologia da
> Universidade Nova de Lisboa
> Quinta da Torre 2829-516 Caparica Portugal
> Telefone: +351 212948596 Fax: +351 212948548
> www.ci.fct.unl.pt ap...@fct.unl.pt
>
> ci.fct.unl.pt:~# _
>
>
> ------------------------------------------------------------------------------
> Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
> trial. Simplify your report design, integration and deployment - and focus on
> what you do best, core application coding. Discover what's new with
> Crystal Reports now. http://p.sf.net/sfu/bobj-july
> _______________________________________________
> Dspam-devel mailing list
> Dspam-devel@lists.sourceforge.net
> https://lists.sourceforge.net/lists/listinfo/dspam-devel
>
------------------------------------------------------------------------------
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
_______________________________________________
Dspam-devel mailing list
Dspam-devel@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dspam-devel
