I wrote at 15:03 (EST):
> Tonight, I'll work up a quick patch with tests, etc. based on Johan's
> suggestion below.
>> -            myGetTextFunc = gettext.translation('poker-engine', 
>> languages=[lang]).gettext
>> +            myGetTextFunc = gettext.translation('poker-engine', 
>> languages=[lang], codeset="iso-8859-1").gettext
> I suspect to post something around 20:00 US/Eastern.

Only a few hours late. :)

Attached please find a patch that implements this basic suggestion,
including test coverage and somewhat of plan for future changes that
will better support and handle the issue of string conversion for
non-ISO-8859 locales.

The only client-visible change should be that you for the time being
*not* pass the encoding portion of the locale setting.  All encodings
are being sent out over the JSON interface as Unicode entities, not
UTF-8 per se.  Even worse, the encoding would *have* to be set to
ISO-8859-1 for things to work properly if that support were currently
left in, and that's just plain confusing given that the JSON interface
is going to give Unicode entities back.

This patch will get earthquake up and running, and it provides a
springboard for a more correct change to actually handle locale
encodings correctly.

I'd like to commit to poker-network trunk later this morning (Saturday
27 November).  I'll be online on #pokersource around 11:30 UTC.  This
patch also is r5068 on the locale-server-side branch.


This patch is copyrighted by me and licensed AGPLv3-or-later.

diff --git a/poker-network/ChangeLog b/poker-network/ChangeLog
index be27ff9..d521556 100644
--- a/poker-network/ChangeLog
+++ b/poker-network/ChangeLog
@@ -1,3 +1,52 @@
+2008-11-28  Bradley M. Kuhn  <[EMAIL PROTECTED]>
+
+	* tests/test-pokeravatar.py.in
+	(PokerAvatarNoClientServerTestCase.MockService.__init__): Added
+	encoding option to transFunc.
+	(PokerAvatarTestCase.test71_setLocaleAlwaysValid): Changed locale
+	string.
+	(PokerAvatarTestCase.test15_handPlay): Changed locale strings.
+	(PokerAvatarTestCase.test15a_handPlay_dbLocalesAndOverridesThereof):
+	Changed locale strings.
+	(PokerAvatarTestCase.startHandAndReceiveCards): Changed locale
+	strings.
+	(PokerAvatarNoClientServerTestCase.test03_relogin_localeNotFound):
+	Rewrote lambda as transFunc; added arg.  Removed expectation of
+	message.
+	(PokerAvatarTestCase.setLocale): Added check for verbosity message
+	on invalid locale.
+
+	* pokernetwork/pokersite.py (fromutf8, toutf8): Added FIXME
+	comment.
+
+	* pokernetwork/pokerpackets.py (PacketPokerSetLocale): Changed
+	documentation string to match what is actually expected.
+
+	* pokernetwork/pokeravatar.py (PokerAvatar.setLocale): Removed
+	error function; locale2translationFunc now has verbose message.
+	Added default encoding call and comment.
+
+	* conf/poker.server.xml.in: Added non-country supported languages.
+
+	* tests/test-pokerservice.py.in
+	(PokerServiceTestCase.test23_localeChecks): Wrote test.
+
+	* pokernetwork/pokerservice.py
+	(PokerService.locale2translationFunc): Added codeset argument and
+	auto-append it before lookup.  Added warning message when locale
+	not found.
+
+	* tests/test-pokerservice.py.in
+	(PokerServiceTestCase.test00_12_badEncoding): Wrote test.
+
+	* pokernetwork/pokerservice.py
+	(PokerService._separateCodesetFromLocale): Wrote function.
+	(PokerService._lookupTranslationFunc): Added proper codeset
+	support with error handling.
+
+	* tests/test-pokerservice.py.in
+	(PokerServiceTestCase.test23_localeChecks): Added stub for test.
+
 2008-11-28  Loic Dachary <[EMAIL PROTECTED]>
 
  	* Release 1.7.1
diff --git a/poker-network/conf/poker.server.xml.in b/poker-network/conf/poker.server.xml.in
index d4287c7..f46509f 100644
--- a/poker-network/conf/poker.server.xml.in
+++ b/poker-network/conf/poker.server.xml.in
@@ -5,24 +5,39 @@
      max_missed_round defaults to 10 if you leave it out.
      max_joined defaults to 4000 if you leave it out.  -->
 
-  <language value="en_US.UTF-8"/>
-  <language value="fr_FR.UTF-8"/>
-  <language value="fr_FX.UTF-8"/>
-  <language value="de_DE.UTF-8"/>
-  <language value="en_GB.UTF-8"/>
-  <language value="es_ES.UTF-8"/>
-  <language value="nl_NL.UTF-8"/>
-  <language value="fr_BE.UTF-8"/>
-  <language value="en_CA.UTF-8"/>
-  <language value="fr_CA.UTF-8"/>
-  <language value="it_IT.UTF-8"/>
-  <language value="pt_PT.UTF-8"/>
-  <language value="da_DK.UTF-8"/>
-  <language value="fi_FI.UTF-8"/>
-  <language value="nb_NO.UTF-8"/>
-  <language value="sv_SE.UTF-8"/>
-
-<!-- language settings are for client-side localization support, set via the PacketPokerSetLocale() -->
+  <language value="en.ISO-8859-1"/>
+  <language value="en_US.ISO-8859-1"/>
+  <language value="en_GB.ISO-8859-1"/>
+  <language value="en_CA.ISO-8859-1"/>
+  <language value="fr_FR.ISO-8859-1"/>
+  <language value="fr.ISO-8859-1"/>
+  <language value="fr_FX.ISO-8859-1"/>
+  <language value="fr_BE.ISO-8859-1"/>
+  <language value="fr_CA.ISO-8859-1"/>
+  <language value="de.ISO-8859-1"/>
+  <language value="de_DE.ISO-8859-1"/>
+  <language value="es.ISO-8859-1"/>
+  <language value="es_ES.ISO-8859-1"/>
+  <language value="nl.ISO-8859-1"/>
+  <language value="nl_NL.ISO-8859-1"/>
+  <language value="it.ISO-8859-1"/>
+  <language value="it_IT.ISO-8859-1"/>
+  <language value="pt.ISO-8859-1"/>
+  <language value="pt_PT.ISO-8859-1"/>
+  <language value="da.ISO-8859-1"/>
+  <language value="da_DK.ISO-8859-1"/>
+  <language value="fi.ISO-8859-1"/>
+  <language value="fi_FI.ISO-8859-1"/>
+  <language value="nb.ISO-8859-1"/>
+  <language value="nb_NO.ISO-8859-1"/>
+  <language value="sv.ISO-8859-1"/>
+  <language value="sv_SE.ISO-8859-1"/>
+
+<!-- language settings above are for client-side localization support, set
+     via the PacketPokerSetLocale().  Note that *only* ISO-8859-1 codesets
+     are currently supported!  If you use ISO-8859-1 codsets, clients
+     using the JSON interface (and perhaps even the packet interface) will
+     receive *incorrectly encoded* strings.  -->
 
   <stats type="RankPercentile"/>
 
diff --git a/poker-network/pokernetwork/pokeravatar.py b/poker-network/pokernetwork/pokeravatar.py
index c382018..0ef2c7d 100644
--- a/poker-network/pokernetwork/pokeravatar.py
+++ b/poker-network/pokernetwork/pokeravatar.py
@@ -91,9 +91,16 @@ class PokerAvatar:
             
     def setLocale(self, locale):
         if locale:
-            self.localeFunc = self.service.locale2translationFunc(locale)
-            if not self.localeFunc:
-                self.error("locale not found: '%s'" % locale)
+            # 'ISO-8859-1' is currently enforced for all setLocale()
+            # requests.  This is primarily because the JSON interface
+            # implemented in pokersite.py *assumes* that all strings must
+            # be 'ISO-8859-1' and encodes them to unicode.  We should
+            # actually find a way to be knowledgeable about the needed
+            # encoding, because there is probably pointless conversion
+            # between ISO and UTF-8 happening in this process.  Also, we
+            # cannot currently support any languages that are non
+            # 'ISO-8859-1', so that's a huge FIXME!  --bkuhn, 2008-10-28
+            self.localeFunc = self.service.locale2translationFunc(locale, 'ISO-8859-1')
         return self.localeFunc
 
     def setProtocol(self, protocol):
diff --git a/poker-network/pokernetwork/pokerpackets.py b/poker-network/pokernetwork/pokerpackets.py
index da48e8f..1548f61 100644
--- a/poker-network/pokernetwork/pokerpackets.py
+++ b/poker-network/pokernetwork/pokerpackets.py
@@ -4145,11 +4145,12 @@ returned, and future PokerExplain strings will be localized to the
 requested language.  Otherwise a PacketError is returned with other_type
 set to PACKET_POKER_SET_LOCALE.
 
-locale: string representing fully qualified locale and encoding, such as "fr_FR.UTF-8"
+locale: string representing a valid locale supported by the server configuration (e.g.,  "fr_FR" or "fr")
 serial: integer uniquely identifying a player.
 """
+# locale: string representing fully qualified locale and encoding, such as "fr_FR.UTF-8"
 
-    info = PacketSerial.info + ( ('locale', 'en_US.UTF-8', 's'), )
+    info = PacketSerial.info + ( ('locale', 'en_US', 's'), )
 
 Packet.infoDeclare(globals(), PacketPokerSetLocale, PacketSerial, "POKER_SET_LOCALE", 153) # 153 # 0x99 # %SEQ%
 
diff --git a/poker-network/pokernetwork/pokerservice.py b/poker-network/pokernetwork/pokerservice.py
index 9dc7880..9af899f 100644
--- a/poker-network/pokernetwork/pokerservice.py
+++ b/poker-network/pokernetwork/pokerservice.py
@@ -352,30 +352,81 @@ class PokerService(service.Service):
     def getClientQueuedPacketMax(self):
         return self.client_queued_packet_max
 
-    def _lookupTranslationFunc(self, lang):
+    def _separateCodesetFromLocale(self, lang_with_codeset):
+        lang = lang_with_codeset
+        codeset = ""
+        dotLoc = lang.find('.')
+        if dotLoc > 0:
+            lang = lang_with_codeset[:dotLoc]
+            codeset = lang_with_codeset[dotLoc+1:]
+
+        if len(codeset) <= 0:
+            self.error('Unable to find codeset string in language value: %s' % lang_with_codeset)
+        if len(lang) <= 0:
+            self.error('Unable to find locale string in language value: %s' % lang_with_codeset)
+        return (lang, codeset)
+
+    def _lookupTranslationFunc(self, lang_with_codeset):
         # Start by defaulting to just returning the string...
         myGetTextFunc = lambda text:text
+
+        (lang, codeset) = self._separateCodesetFromLocale(lang_with_codeset)
+
+# I now believe that changing the locale in this way for each language is
+# completely uneeded given the set of features we are looking for.
+# Ultimately, we aren't currently doing localization operations other than
+# gettext() string lookup, so the need to actually switch locales does not
+# exist.  Long term, we may want to format numbers properly for remote
+# users, and then we'll need more involved locale changes, probably
+# handled by avatar and stored in the server object.  In the meantime,
+# this can be commented out and makes testing easier.  --bkuhn, 2008-11-28
+
+#         try:
+#             locale.setlocale(locale.LC_ALL, lang)
+#         except locale.Error, le:
+#             self.error('Unable to support locale, "%s", due to locale error: %s'
+#                        % (lang_with_codeset, le))
+#             return myGetTextFunc
+
+# I believe calling these was completely pointless in the first place,
+#  since we never want to install these using the standard GNU API.
+#  --bkuhn, 2008-11-28
+
+#        gettext.bind_textdomain_codeset("poker-engine", lang)
+#        gettext.install("poker-engine")
         try:
-            locale.setlocale(locale.LC_ALL, lang)
-        except locale.Error, le:
-            self.error('Unable to support locale, "%s", due to locale error: %s'
-                       % (lang, le))
-            return myGetTextFunc
-
-        # I am not completely sure poker-engine should be hardcoded here like this...
-        gettext.bind_textdomain_codeset("poker-engine", lang)
-        gettext.install("poker-engine")
-        try:
-            myGetTextFunc = gettext.translation('poker-engine', languages=[lang]).gettext
+            # I am not completely sure poker-engine should be hardcoded here like this...
+            myGetTextFunc = gettext.translation('poker-engine', 
+                                                languages=[lang], codeset=codeset).gettext
+            # This test call of the function *must* be a string in the
+            # poker-engine domain.  The idea is to force a throw of
+            # LookupError, which will be thrown if the codeset doesn't
+            # exist.  Unfortunately, gettext doesn't throw it until you
+            # call it with a string that it can translate (gibberish
+            # doesn't work!).  We want to fail to support this
+            # language/encoding pair here so the server can send the error
+            # early and still support clients with this codec, albeit by
+            # sending untranslated strings.
+            myGetTextFunc("Aces")
         except IOError, e:
-            self.error("No translation to locale %s in poker-engine; locale ignored: %s"
-                       % (lang, e))
+            self.error("No translation for language %s for %s in poker-engine; locale ignored: %s"
+                       % (lang, lang_with_codeset, e))
+            myGetTextFunc = lambda text:text
+        except LookupError, l:
+            self.error("Unsupported codeset %s for %s in poker-engine; locale ignored: %s"
+                       % (codeset, lang_with_codeset, l))
+            myGetTextFunc = lambda text:text
+
         return myGetTextFunc
 
-    def locale2translationFunc(self, locale):
+    def locale2translationFunc(self, locale, codeset = ""):
+        if len(codeset) > 0:
+            locale += "." + codeset
         if self.gettextFuncs.has_key(locale):
             return self.gettextFuncs[locale]
         else:
+            if self.verbose > 2:
+                self.message("Locale, \"%s\" not available.  %s must not have been provide via <language/> tag in settings, or errors occured during loading." % (locale, locale))
             return None
 
     def shutdown(self):
diff --git a/poker-network/pokernetwork/pokersite.py b/poker-network/pokernetwork/pokersite.py
index 0fb89c2..fc20fda 100644
--- a/poker-network/pokernetwork/pokersite.py
+++ b/poker-network/pokernetwork/pokersite.py
@@ -38,6 +38,9 @@ from pokernetwork import pokermemcache
 def uid2last_modified(uid):
     return 'L' + uid
 
+# FIXME: I don't think these next two functions should assume 'ISO-8859-1'
+# like they do.  This is related to another FIXME about this issue you'll
+# find in pokeravatar.py -- bkuhn, 2008-11-28
 def fromutf8(tree, encoding = 'ISO-8859-1'):
     return __walk(tree, lambda x: x.encode(encoding))
 
diff --git a/poker-network/tests/test-pokeravatar.py.in b/poker-network/tests/test-pokeravatar.py.in
index b04333f..5c4cd0d 100644
--- a/poker-network/tests/test-pokeravatar.py.in
+++ b/poker-network/tests/test-pokeravatar.py.in
@@ -48,14 +48,14 @@ from tests.testmessages import restore_all_messages, silence_all_messages, searc
 verbose = int(os.environ.get('VERBOSE_T', '-1'))
 if verbose < 0: silence_all_messages()
 
-# Note that we use the locale fr_FR.UTF-8 here for testing.  This must be
+# Note that we use the locale fr_FR.ISO-8859-1 here for testing.  This must be
 # a valid locale on the system!  Important tests are skipped otherwise!!!
 
 localLocale = locale.getlocale(locale.LC_ALL)
-MESSAGE_WHEN_FRENCH_LOCALE_MISSING = "WARNING: Locale fr_FR.UTF-8 missing, so certain tests will be SKIPPED and coverage will be INCOMPLETE!  FAIL!  fail!"
+MESSAGE_WHEN_FRENCH_LOCALE_MISSING = "WARNING: Locale fr_FR.ISO-8859-1 missing, so certain tests will be SKIPPED and coverage will be INCOMPLETE!  FAIL!  fail!"
 FRENCH_LOCALE_MISSING = False
 try:
-    locale.setlocale(locale.LC_ALL, "fr_FR.UTF-8")
+    locale.setlocale(locale.LC_ALL, "fr_FR.ISO-8859-1")
 except locale.Error, le:
     print MESSAGE_WHEN_FRENCH_LOCALE_MISSING
     FRENCH_LOCALE_MISSING = True
@@ -91,8 +91,8 @@ settings_xml_server = """<?xml version="1.0" encoding="ISO-8859-1"?>
 <server verbose="3" ping="300000" autodeal="yes" simultaneous="4" chat="yes" >
   <delays autodeal="20" round="0" position="0" showdown="0" autodeal_max="1" finish="0" messages="60" />
 
-  <language value="en_US.UTF-8"/>
-  <language value="fr_FR.UTF-8"/>
+  <language value="en_US.ISO-8859-1"/>
+  <language value="fr_FR.ISO-8859-1"/>
 
   <stats type="RankPercentile"/>
 
@@ -796,15 +796,15 @@ class PokerAvatarTestCase(unittest.TestCase):
         lang2strings = { 'default' : [ "Dealer: user1 pays 1 blind\n",
                                        "Dealer: user0 pays 2 blind\n",
                                        "Dealer: pre-flop, 2 players\n" ], 
-                         'en_US.UTF-8' : [ "Dealer: user1 pays 1 blind\n",
+                         'en_US' : [ "Dealer: user1 pays 1 blind\n",
                                            "Dealer: user0 pays 2 blind\n",
                                            "Dealer: pre-flop, 2 players\n" ], 
-                         'fr_FR.UTF-8' : [ "Dealer: user1 paye 1 de blind\n",
+                         'fr_FR' : [ "Dealer: user1 paye 1 de blind\n",
                                            "Dealer: user0 paye 2 de blind\n",
                                            "Dealer: pre-flop, 2 joueurs\n" ] }
         if FRENCH_LOCALE_MISSING:
             print MESSAGE_WHEN_FRENCH_LOCALE_MISSING
-            lang2strings['fr_FR.UTF-8'] = lang2strings['default']
+            lang2strings['fr_FR'] = lang2strings['default']
 
         avatar = []
         avatar.append(self.service.avatars[0])
@@ -927,7 +927,7 @@ class PokerAvatarTestCase(unittest.TestCase):
         # labelled "How to sit at a cash game table ?"
         d = self.client_factory[0].established_deferred
         d.addCallback(self.sendExplain)
-        d.addCallback(self.setLocale, "fr_FR.UTF-8")
+        d.addCallback(self.setLocale, "fr_FR")
         d.addCallback(self.sendRolePlay)
         d.addCallback(self.login, 0)
         d.addCallback(self.createRankDBTable, 2, rank = 50, percentile = 30)
@@ -990,10 +990,10 @@ class PokerAvatarTestCase(unittest.TestCase):
         d = self.client_factory[0].established_deferred
         d.addCallback(self.sendExplain)
         d.addCallback(self.forceDbToLocaleValue, avid = 0, 
-                      setTo = 'fr_FR.UTF-8', expect = "RESET_BY_SET_LOCALE")
+                      setTo = 'fr_FR', expect = "RESET_BY_SET_LOCALE")
         d.addCallback(self.sendRolePlay)
         d.addCallback(self.login, 0)
-        d.addCallback(self.setLocale, "en_US.UTF-8")
+        d.addCallback(self.setLocale, "en_US")
         d.addCallback(self.createRankDBTable, 2, rank = 50, percentile = 30)
         d.addCallback(self.joinTable, 0, 2, 'Table2', '2-4-limit')
         d.addCallback(self.seatTable, 0, 2, 50, 30)
@@ -1005,7 +1005,7 @@ class PokerAvatarTestCase(unittest.TestCase):
 
         d2.addCallback(self.sendExplain)
         d2.addCallback(self.forceDbToLocaleValue, avid = 1, 
-                       setTo = 'fr_FR.UTF-8', expect = "fr_FR.UTF-8")
+                       setTo = 'fr_FR', expect = "fr_FR")
         d2.addCallback(self.sendRolePlay)
         d2.addCallback(self.login, 1)
         d2.addCallback(self.joinTable, 1, 2, 'Table2', '2-4-limit',
@@ -1379,6 +1379,8 @@ class PokerAvatarTestCase(unittest.TestCase):
                 self.assertEquals(packet.serial, client.getSerial())
                 self.assertEquals(packet.game_id, gameId)
                 found = True
+            elif packet.type == PACKET_POKER_CHAT:
+                print packet
         self.assertEquals(found, True)
         return (client, packet)
     # -------------------------------------------------------------------------
@@ -3131,12 +3133,13 @@ class PokerAvatarTestCase(unittest.TestCase):
         return d
     # -------------------------------------------------------------------------
     def setLocale(self, (client, packet), myLocale, avid = 0, expectSucceed=True):
-        if FRENCH_LOCALE_MISSING and myLocale == 'fr_FR.UTF-8':
+        if FRENCH_LOCALE_MISSING and myLocale == 'fr_FR':
             print MESSAGE_WHEN_FRENCH_LOCALE_MISSING
             expectSucceed = False
 
         avatar = self.service.avatars[avid]
         avatar.queuePackets()
+        clear_all_messages()
         avatar.handlePacketLogic(PacketPokerSetLocale(serial = client.getSerial(), 
                                                       locale = myLocale))
         foundCount = 0
@@ -3156,6 +3159,7 @@ class PokerAvatarTestCase(unittest.TestCase):
                     self.assertEquals(packet.serial, client.getSerial())
                     self.assertEquals(packet.other_type, PACKET_POKER_SET_LOCALE)
                     foundCount += 1
+                    self.failUnless(search_output('Locale, "Klingon_Kronos.ISO-8859-1" not available.  Klingon_Kronos.ISO-8859-1 must not have been provide via <language/> tag in settings, or errors occured during loading.'))
         self.assertEquals(foundCount, 1)
         return (client, packet)
     # -------------------------------------------------------------------------
@@ -3163,7 +3167,7 @@ class PokerAvatarTestCase(unittest.TestCase):
         """Tests setting of roles"""
         d = self.client_factory[0].established_deferred
         d.addCallback(self.sendExplain)
-        d.addCallback(self.setLocale, "en_US.UTF-8", 0)
+        d.addCallback(self.setLocale, "en_US", 0)
         d.addCallback(self.quit)
         return d
     # -------------------------------------------------------------------------
@@ -3171,7 +3175,7 @@ class PokerAvatarTestCase(unittest.TestCase):
         """Tests setting of roles"""
         d = self.client_factory[0].established_deferred
         d.addCallback(self.sendExplain)
-        d.addCallback(self.setLocale, "Klingon_Kronos.UTF-8", 0, False)
+        d.addCallback(self.setLocale, "Klingon_Kronos", 0, False)
         d.addCallback(self.quit)
         return d
     # ------------------------------------------------------------------------
@@ -3702,7 +3706,7 @@ class PokerAvatarNoClientServerTestCase(unittest.TestCase):
             mpiSelf.name = 'Doyle Brunson'
     class MockService:
         def __init__(msSelf):
-            def transFunc(locale):
+            def transFunc(locale, encoding = ''):
                 if locale == 'mylocale':
                     return lambda s: "MYTRANSLATION"
                 else:
@@ -3797,14 +3801,13 @@ class PokerAvatarNoClientServerTestCase(unittest.TestCase):
         from pokernetwork import pokeravatar
 
         service = PokerAvatarNoClientServerTestCase.MockService()
-        service.locale2translationFunc = lambda l: None
+        def transFunc(l, codeset = ''): return None
+        service.locale2translationFunc = transFunc
         avatar = pokeravatar.PokerAvatar(service)
         explain = PokerAvatarNoClientServerTestCase.MockExplain()
         saveExplain = avatar.explain
         avatar.explain = explain
 
-        errors = []
-        avatar.error = lambda e: errors.append(e)
         avatar.relogin(1042)
         self.assertEquals(avatar.user.serial, 1042)
         self.assertEquals(avatar.user.name, 'Doyle Brunson')
@@ -3823,11 +3826,10 @@ class PokerAvatarNoClientServerTestCase(unittest.TestCase):
         self.assertEquals(explain.handleSerialPackets[0].type, PACKET_SERIAL)
 
         self.assertEquals(None, avatar.localeFunc)
-        self.assertEquals("locale not found: 'mylocale'", errors[0])
 ##############################################################################
 def Run():
     loader = runner.TestLoader()
-#    loader.methodPrefix = "test15a"
+#    loader.methodPrefix = "test72"
     suite = loader.suiteFactory()
     suite.addTest(loader.loadClass(PokerAvatarTestCase))
     suite.addTest(loader.loadClass(PokerAvatarNoClientServerTestCase))
diff --git a/poker-network/tests/test-pokerservice.py.in b/poker-network/tests/test-pokerservice.py.in
index eafc17f..88b50a1 100644
--- a/poker-network/tests/test-pokerservice.py.in
+++ b/poker-network/tests/test-pokerservice.py.in
@@ -83,14 +83,14 @@ TABLE1 = 1
 TABLE2 = 2
 TABLE3 = 3
 
-# Note that we use the locale fr_FR.UTF-8 here for testing.  This must be
+# Note that we use the locale fr_FR.ISO-8859-1 here for testing.  This must be
 # a valid locale on the system!  Important tests are skipped otherwise!!!
 
 localLocale = locale.getlocale(locale.LC_ALL)
 FRENCH_LOCALE_MISSING = False
-MESSAGE_WHEN_FRENCH_LOCALE_MISSING = "WARNING: Locale fr_FR.UTF-8 missing, so certain tests will be SKIPPED and coverage will be INCOMPLETE! FAIL!  fail!"
+MESSAGE_WHEN_FRENCH_LOCALE_MISSING = "WARNING: Locale fr_FR.ISO-8859-1 missing, so certain tests will be SKIPPED and coverage will be INCOMPLETE! FAIL!  fail!"
 try:
-    locale.setlocale(locale.LC_ALL, "fr_FR.UTF-8")
+    locale.setlocale(locale.LC_ALL, "fr_FR.ISO-8859-1")
 except locale.Error, le:
     print MESSAGE_WHEN_FRENCH_LOCALE_MISSING
     FRENCH_LOCALE_MISSING = True
@@ -107,8 +107,22 @@ settings_xml = """<?xml version="1.0" encoding="ISO-8859-1"?>
 
   <listen tcp="19480" />
 
-  <language value="en_US.UTF-8"/>
-  <language value="fr_FR.UTF-8"/>
+  <language value="en_US.ISO-8859-1"/>
+  <language value="fr_FR.ISO-8859-1"/>
+  <language value="fr_FX.ISO-8859-1"/>
+  <language value="de_DE.ISO-8859-1"/>
+  <language value="en_GB.ISO-8859-1"/>
+  <language value="es_ES.ISO-8859-1"/>
+  <language value="nl_NL.ISO-8859-1"/>
+  <language value="fr_BE.ISO-8859-1"/>
+  <language value="en_CA.ISO-8859-1"/>
+  <language value="fr_CA.ISO-8859-1"/>
+  <language value="it_IT.ISO-8859-1"/>
+  <language value="pt_PT.ISO-8859-1"/>
+  <language value="da_DK.ISO-8859-1"/>
+  <language value="fi_FI.ISO-8859-1"/>
+  <language value="nb_NO.ISO-8859-1"/>
+  <language value="sv_SE.ISO-8859-1"/>
   <language value="this_locale_does_not_exist"/>
 
   <stats type="RankPercentile"/>
@@ -616,29 +630,25 @@ class PokerServiceTestCase(PokerServiceTestCaseBase):
                             ('tables', 2), ('tourney_table_serial', 1),
                             ('shutting_down', False), ('avatars', []),
                             ('serial2client', {}), ('simultaneous', 4),
-                            ('monitors', []), ('gettextFuncs', 3) ]
+                            ('monitors', []), ('gettextFuncs', 17) ]
         for (instanceVar, val) in startedOnlyVars:
             self.assertEquals(self.service.__dict__.has_key(instanceVar), False)
 
         clear_all_messages()
         self.service.startService()
-        self.assertEquals(search_output("*ERROR* No translation to locale en_US.UTF-8 in poker-engine; locale ignored: [Errno 2] No translation file found for domain: 'poker-engine'"), True)
-        self.assertEquals(search_output('*ERROR* Unable to support locale, "this_locale_does_not_exist", due to locale error: unsupported locale setting'), True)
+        self.failUnless(search_output('*ERROR* Unable to find codeset string in language value: this_locale_does_not_exist'))
+        self.failUnless(search_output("*ERROR* No translation for language this_locale_does_not_exist for this_locale_does_not_exist in poker-engine; locale ignored: [Errno 2] No translation file found for domain: 'poker-engine'"))
         for (instanceVar, val) in startedOnlyVars:
             self.assertEquals(self.service.__dict__.has_key(instanceVar), True)
             if instanceVar == "tables":
                 self.assertEquals(len(self.service.__dict__[instanceVar]), val)
 
-        for ii in ('this_locale_does_not_exist', 'en_US.UTF-8', 'fr_FR.UTF-8'):
+        for ii in ('this_locale_does_not_exist', 'en_US.ISO-8859-1', 'fr_FR.ISO-8859-1'):
             self.assertEquals(self.service.gettextFuncs.has_key(ii), True)
             self.assertEquals(callable(self.service.locale2translationFunc(ii)), True)
-        for ii in ('this_locale_does_not_exist', 'en_US.UTF-8'):
+        for ii in ('this_locale_does_not_exist', 'en_US.ISO-8859-1'):
             self.assertEquals(self.service.gettextFuncs[ii]("Aces"), "Aces")
-        if FRENCH_LOCALE_MISSING:
-            print MESSAGE_WHEN_FRENCH_LOCALE_MISSING
-            self.assertEquals(self.service.gettextFuncs['fr_FR.UTF-8']("Aces"), "Aces")
-        else:
-            self.assertEquals(self.service.gettextFuncs['fr_FR.UTF-8']("Aces"), "d'As")
+        self.assertEquals(self.service.gettextFuncs['fr_FR.ISO-8859-1']("Aces"), "d'As")
 
         for ii in ('nothing', 'unknown'):
             self.assertEquals(self.service.locale2translationFunc(ii), None)
@@ -682,11 +692,13 @@ class PokerServiceTestCase(PokerServiceTestCaseBase):
         self.service = pokerservice.PokerService(settings)
         clear_all_messages()
         self.service.startService()
-        self.assertEquals(len(self.service.gettextFuncs), 3)
-        for ii in ('en_US.UTF-8', 'fr_FR.UTF-8', 'this_locale_does_not_exist'):
-            self.assertEquals(search_output('*ERROR* Unable to support locale, "%s", due to locale error: testing bad setlocale' % ii), True)
+        self.assertEquals(len(self.service.gettextFuncs), 17)
+        self.failUnless(search_output('*ERROR* Unable to find codeset string in language value: this_locale_does_not_exist'))
+        self.failUnless(search_output("*ERROR* No translation for language this_locale_does_not_exist for this_locale_does_not_exist in poker-engine; locale ignored: [Errno 2] No translation file found for domain: 'poker-engine'"))
+        for ii in ('en_US.ISO-8859-1', 'fr_FR.ISO-8859-1'):
             self.assertEquals(self.service.gettextFuncs.has_key(ii), True)
-            self.assertEquals(self.service.gettextFuncs[ii]("Aces"), "Aces")
+        self.assertEquals(self.service.gettextFuncs['en_US.ISO-8859-1']("Aces"), "Aces")
+        self.assertEquals(self.service.gettextFuncs['fr_FR.ISO-8859-1']("Aces"), "d'As")
         self.assertEquals(search_output("*ERROR* Unable to restore original locale: testing bad setlocale"), True)
         locale.setlocale = saveLocale
 
@@ -713,6 +725,28 @@ class PokerServiceTestCase(PokerServiceTestCaseBase):
     def test00_11_configValues(self):
         self.configValues(settings_xml.replace('<tourney_attrs type="SponsoredPrizes"/>', ''), 1000,
                           tourneyAttrsLookupType = AttrsLookup)
+    def test00_12_badEncoding(self):
+        """test00_12_badEncoding
+        Check the case where an error occurs due to an encoding requested
+        for a config language value has an unknown encoding."""
+
+        new_settings_xml = settings_xml.replace('<language value="fr_FR.ISO-8859-1"/>',
+                                                '<language value="fr_FR.MAGIC-PIXIE-DUST"/>')
+
+        settings = pokernetworkconfig.Config([])
+        settings.doc = libxml2.parseMemory(new_settings_xml, len(new_settings_xml))
+        settings.header = settings.doc.xpathNewContext()
+        self.service = pokerservice.PokerService(settings)
+        clear_all_messages()
+        self.service.startService()
+        self.assertEquals(len(self.service.gettextFuncs), 17)
+        self.failUnless(search_output('*ERROR* Unsupported codeset MAGIC-PIXIE-DUST for fr_FR.MAGIC-PIXIE-DUST in poker-engine; locale ignored: unknown encoding: MAGIC-PIXIE-DUST'))
+        self.failUnless(search_output('*ERROR* Unable to find codeset string in language value: this_locale_does_not_exist'))
+        self.failUnless(search_output("*ERROR* No translation for language this_locale_does_not_exist for this_locale_does_not_exist in poker-engine; locale ignored: [Errno 2] No translation file found for domain: 'poker-engine'"))
+        for ii in ('en_US.ISO-8859-1', 'fr_FR.MAGIC-PIXIE-DUST'):
+            self.assertEquals(self.service.gettextFuncs.has_key(ii), True)
+            self.assertEquals(self.service.gettextFuncs[ii]("Aces"), "Aces")
+
     # ----------------------------------------------------------------
     def test01_auth(self):
         self.service.startService()
@@ -1217,7 +1251,29 @@ class PokerServiceTestCase(PokerServiceTestCaseBase):
         cursor.execute("UPDATE tourneys_schedule SET currency_serial_from_date_format = 'NaN666' WHERE name = 'sitngo2'")
         cursor.close()
         self.assertRaises(UserWarning, self.service.startService)
+    # ----------------------------------------------------------------
+    def test23_localeChecks(self):
+        self.service.startService()
+        for enlo in ('this_locale_does_not_exist', 'en_GB.ISO-8859-1',
+                     'en_US.ISO-8859-1', 'en_CA.ISO-8859-1'):
+            self.assertEquals(self.service.locale2translationFunc(enlo)("Aces"), "Aces")
 
+        clear_all_messages()
+        self.assertEquals(self.service.locale2translationFunc('fr_FR', 'UTF-8'),
+                         None)
+        self.assertEquals(get_messages(), ['Locale, "fr_FR.UTF-8" not available.  fr_FR.UTF-8 must not have been provide via <language/> tag in settings, or errors occured during loading.'])
+        enc = 'ISO-8859-1'
+        self.assertEquals(self.service.locale2translationFunc('da_DK', enc)("%(name)s mucks loosing hand"), "%(name)s mucker tabende h\xe5nd")
+        self.assertEquals(self.service.locale2translationFunc('de_DE', enc)("%(name)s raises %(amount)s"), '%(name)s erh\xf6ht %(amount)s' )
+        self.assertEquals(self.service.locale2translationFunc('fi_FI', enc)("Board: %(board)s"), 'P\xf6yt\xe4kortit: %(board)s')
+        for fr in ('fr_FR', 'fr_FX', 'fr_BE', 'fr_CA'):
+            self.assertEquals(self.service.locale2translationFunc(fr, enc)("%(name)s receives %(amount)s"), '%(name)s re\xe7oit %(amount)s')
+        self.assertEquals(self.service.locale2translationFunc('it_IT', enc)("High card %(card)s"),'Carta pi\xf9 alta: %(card)s')
+        self.assertEquals(self.service.locale2translationFunc('nb_NO', enc)("%(name)s mucks loosing hand"), '%(name)s skjuler tapende h\xe5nd')
+        self.assertEquals(self.service.locale2translationFunc('nl_NL', enc)("Four of a kind %(card)s"), 'Carr\xe9 %(card)s')
+        self.assertEquals(self.service.locale2translationFunc('pt_PT', enc)("Rake %(amount)s"), 'Comiss\xe3o %(amount)s')
+        self.assertEquals(self.service.locale2translationFunc('sv_SE', enc)("winners share a pot of %(pot)s"), 'vinnarna delar p\xe5 potten %(pot)s')
+##############################################################################
 class RefillTestCase(unittest.TestCase):
 
     def destroyDb(self):
@@ -2396,8 +2452,8 @@ class PokerServiceCoverageTests(unittest.TestCase):
         self.service.db = oldDb
 # ----------------------------------------------------------------
 def Run():
-    loader = runner.TestLoader()
-#    loader.methodPrefix = "test00"
+    loader = runner.TestLoader() 
+#    loader.methodPrefix = "test23"
     suite = loader.suiteFactory()
     suite.addTest(loader.loadClass(PokerServiceTestCase))
     suite.addTest(loader.loadClass(RefillTestCase))
-- 

   -- bkuhn
_______________________________________________
Pokersource-users mailing list
[email protected]
https://mail.gna.org/listinfo/pokersource-users

Reply via email to