It's fair to say that the Javadocs for the Properties class is a little ambiguous about what should be Unicode-escaped, and what should not be.
The Properties Javadoc states that properties files written by store() should be written in ISO 8859-1 encoding. However, it also states that "Characters less than \u0020 and characters greater than \u007E are written as \uxxxx for the appropriate hexadecimal value xxxx." That is what I implemented -- the more conservative implementation that Sun itself uses. But Janne's got a good point. It would be easy enough to modify the behavior slightly to render characters with values A0-FF rather than Unicode-escape them. That would certainly cover the unlauts. :) In fact, I think I'll check that in tonight -- assuming nobody's got any objection. ;) Harry -- did you notice that I checked in fixes for the Russian files last night? As you noted, some (3?) of the keys had unescaped line separators. I checked all of the other files, and those were the only ones. Other than that, BundleMigrator seems to actually work now. Should help me radically accelerate Stripes-enabling the JSPs. Andrew On Thu, Feb 5, 2009 at 5:15 PM, Janne Jalkanen <[email protected]> wrote: > > For me, yes, this is a problem. It complicates translation (and is kinda > useless, since property files are defined to contain Latin1). > > /Janne > > On 5 Feb 2009, at 20:41, Harry Metske wrote: > >> looking at the _de and _fi files, I notice that UTF-8 chars are replaced >> by >> Unicode chars, for example : >> >> pääsyä => p\u00e4\u00e4sy\u00e4 >> können => k\u00f6nnen >> >> Janne/Florian, is that a problem ? >> >> regards, >> Harry >> >> 2009/2/3 Harry Metske <[email protected]> >> >>> ok, I'll fix these properties, run the bundlemigrator again, and commit >>> the >>> changes >>> >>> thanks, >>> Harry >>> >>> 2009/2/3 Andrew Jaquith <[email protected]> >>> >>> I had not noticed that; good catch. >>>> >>>> That is almost certainly not the proper format. Property values, if >>>> they span lines, need to be escaped with a \ character at the end of >>>> the line, just before the newline character. So I would consider this >>>> a bug in the properties file, not a bug in the code. We should fix >>>> this. >>>> >>>> The re-worked CommentedProperties code, by the way, simply delegates >>>> parsing of keys and values to the Properties superclass, so the >>>> parsing results should be the same for both Properties and >>>> CommentedProperties. >>>> >>>> On Tue, Feb 3, 2009 at 1:21 PM, Harry Metske <[email protected]> >>>> wrote: >>>>> >>>>> One of the things I noticed (and maybe you too), is that the >>>>> default_ru.properties contains properties whose value start on a new >>>> >>>> line, >>>>> >>>>> for example this snippet where common.nopage has that : >>>>> >>>>> common.nopage= >>>>> \u042d\u0442\u0430 \u0441\u0442\u0440\u0430\u043d\u0438\u0446\u0430 >>>>> \u043d\u0435 >>>> >>>> \u0441\u0443\u0449\u0435\u0441\u0442\u0432\u0443\u0435\u0442. >>>>> >>>>> \u041f\u043e\u0447\u0435\u043c\u0443 \u0431\u044b \u0432\u0430\u043c >>>>> \u0435\u0435 \u043d\u0435 \u0441\u043e\u0437\u0434\u0430\u0442\u044c >>>> >>>> {0}? >>>>> >>>>> common.createit=\u0441\u043e\u0437\u0434\u0430\u0442\u044c >>>>> common.more=\u0411\u043e\u043b\u044c\u0448\u0435... >>>>> >>>>> >>>>> I don't understand the reason for doing this, and I'm not sure if this >>>> >>>> is >>>>> >>>>> valid Java properties format. >>>>> Anyway, the output from BundleMigrator in this case is : >>>>> >>>>> >>>>> #Copied from etc/i18n/templates/default.properties. >>>>> common.nopage= >>>>> >>>>> Harry >>>>> >>>>> 2009/1/30 Andrew Jaquith <[email protected]> >>>>> >>>>>> I've checked in a new & improved CommentedProperties that preserves >>>>>> encodings correctly. There are a few cosmetic issues that probably >>>>>> should be fixed related to whitespace fidelity in comments, but other >>>>>> that this, it's looking pretty good. >>>>>> >>>>>> At this point we can basically say, "it works." Will tweak a little >>>>>> bit more in the next few days. >>>>>> >>>>>> On 1/29/09, Andrew Jaquith <[email protected]> wrote: >>>>>>> >>>>>>> Harry -- I've concluded that it's probably easiest just to >>>>>>> re-implement Properties.store(). The encoding rules are fairly clear, >>>>>>> so it shouldn't be too hard to do. I will take a whack at this over >>>>>>> the next few days. >>>>>>> >>>>>>> Andrew >>>>>>> >>>>>>> On Thu, Jan 29, 2009 at 3:50 PM, Harry Metske < >>>> >>>> [email protected]> >>>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>> hmmm, indeed, it runs fine, a few minor differences like blanks >>>> >>>> around >>>>>>>> >>>>>>>> the >>>>>>>> equals sign, and the property values convert to one line. >>>>>>>> but the Russian and Romanian files get corrupted. >>>>>>>> >>>>>>>> This is something weird in java.util.Properties, I noticed that >>>>>>>> properties >>>>>>>> that are commented are translated fine, for example : >>>>>>>> >>>>>>>> # Login.jsp >>>>>>>> >>>> #login.error.capslock=\u041d\u0435\u0432\u0435\u0440\u043d\u044b\u0439 >>>>>>>> >>>>>>>> \u043b\u043e\u0433\u0438\u043d >>>>>>>> (\u043f\u0440\u043e\u0432\u0435\u0440\u0442\u0435 >>>>>>>> \u043a\u043b\u0430\u0432\u0438\u0448\u0443 Caps Lock) #obsolete >>>>>>>> login.error.password = ???????? ?????. >>>>>>>> login.error.noaccess = ? ??? ??? ??????? ? ?????. ????????. >>>>>>>> >>>>>>>> You see that the commented login.error.capslock remains fine. >>>>>>>> I was just on my way finding the sources of JDK5, but run out of >>>> >>>> time, >>>>>>>> >>>>>>>> I'll >>>>>>>> dig further next few days (or you get there before me :-) ) >>>>>>>> >>>>>>>> regards, >>>>>>>> Harry >>>>>>>> >>>>>>>> 2009/1/28 Andrew Jaquith <[email protected]> >>>>>>>> >>>>>>>>> Yes. This is one of those weird cases. We have three choices: >>>>>>>>> >>>>>>>>> 1. Duplicate the message key in both places >>>>>>>>> 2. Remove the fmt:message tags in InfoContent.jsp and PageTab.jsp >>>> >>>> (the >>>>>>>>> >>>>>>>>> ones that reference common.nopage) and refactor their functions >>>> >>>> into >>>>>>>>> >>>>>>>>> the ActionBean handler methods >>>>>>>>> 3. Rename the key that WikiPageTypeConverter relies on, or figure >>>> >>>> out >>>>>>>>> >>>>>>>>> a way to eliminate the reference. >>>>>>>>> >>>>>>>>> At the moment (1) is the best option because it is the simplest; >>>> >>>> I'm >>>>>>>>> >>>>>>>>> also investigating (3). But when we encounter stuff like this we >>>> >>>> need >>>>>>>>> >>>>>>>>> to figure out if we can reduce duplication when possible on a >>>>>>>>> case-by-case basis. >>>>>>>>> >>>>>>>>> Could you try the BundleMigrator tool to do this? The command... >>>>>>>>> >>>>>>>>> "java -cp build/JSPWiki.jar >>>>>>>>> com.ecyrd.jspwiki.ui.migrator.BundleMigrator" >>>>>>>>> >>>>>>>>> ...should work. >>>>>>>>> >>>>>>>>> Note: there is one bug in BundleMigrator that I'm still trying to >>>>>>>>> solve, related to output file encoding. If you've got any insights >>>> >>>> on >>>>>>>>> >>>>>>>>> how to solve it, let me know... I'll be looking at this >>>>>>>>> tonight/tomorrow. >>>>>>>>> >>>>>>>>> On Wed, Jan 28, 2009 at 12:04 PM, Harry Metske < >>>> >>>> [email protected] >>>>>>> >>>>>>>>> wrote: >>>>>>>>>> >>>>>>>>>> Andrew, >>>>>>>>>> >>>>>>>>>> nice, during of the tests last week I stumbled into missing >>>> >>>> resource >>>>>>>>>> >>>>>>>>>> common.nopage : >>>>>>>>>> >>>>>>>>>> 2009-01-26 21:51:15,780 [http-8080-6] ERROR >>>>>>>>>> com.ecyrd.jspwiki.tags.WikiTagBase - Tag failed >>>>>>>>>> java.util.MissingResourceException: Could not find an error >>>> >>>> message >>>>>>>>>> >>>>>>>>>> with >>>>>>>>>> key: common.nopage >>>>>>>>>> at >>>>>>>>>> >>>>>>>>> >>>>>> >>>> >>>> net.sourceforge.stripes.validation.LocalizableError.getMessageTemplate(LocalizableError.java:109) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> >>>>>>>>> >>>>>> >>>> >>>> net.sourceforge.stripes.action.SimpleMessage.getMessage(SimpleMessage.java:91) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> >>>>>>>>> >>>>>> >>>> >>>> net.sourceforge.stripes.validation.SimpleError.getMessage(SimpleError.java:102) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> >>>>>> >>>>>> com.ecyrd.jspwiki.tags.MessagesTag.doWikiStartTag(MessagesTag.java:117) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> >>>> com.ecyrd.jspwiki.tags.WikiTagBase.doStartTag(WikiTagBase.java:119) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> >>>>>>>>> >>>>>> >>>> >>>> org.apache.jspwiki.jsp.templates.default_.AttachmentTab_jsp._jspx_meth_wiki_Messages_0(Unknown >>>>>>>>>> >>>>>>>>>> Source) >>>>>>>>>> at >>>>>>>>>> >>>>>>>>> >>>>>> >>>> >>>> org.apache.jspwiki.jsp.templates.default_.AttachmentTab_jsp._jspService(Unknown >>>>>>>>>> >>>>>>>>>> Source) >>>>>>>>>> at >>>>>>>>>> >>>> org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94) >>>>>>>>>> >>>>>>>>>> at >>>>>> >>>>>> javax.servlet.http.HttpServlet.service(HttpServlet.java:803) >>>>>>>>>> >>>>>>>>>> at >>>>>>>>>> >>>>>>>>> >>>>>> >>>> >>>> org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:290) >>>>>>>>>> >>>>>>>>>> ................ >>>>>>>>>> >>>>>>>>>> I read the whole discussion on resourcebundles in >>>>>>>>>> https://issues.apache.org/jira/browse/JSPWIKI-351 . >>>>>>>>>> >>>>>>>>>> The common.nopage is currently in default.properties, but is used >>>> >>>> in >>>>>>>>>> >>>>>>>>>> both >>>>>>>>>> stripes messages, as in plain JSP's, what to do with that ? >>>>>>>>>> >>>>>>>>>> regards, >>>>>>>>>> Harry >>>>>>>>>> >>>>>>>>>> 2009/1/27 Andrew Jaquith <[email protected]> >>>>>>>>>> >>>>>>>>>>> All -- >>>>>>>>>>> >>>>>>>>>>> In the process of writing the BundleMigrator tool, I discovered >>>> >>>> a >>>>>>>>>>> >>>>>>>>>>> serious logic flaw in the CommentedProperties class. >>>>>>>>>>> CommentedProperties reads/writes Properties files while >>>> >>>> preserving >>>>>>>>>>> >>>>>>>>>>> comments. As currently written, it doesn't correctly parse >>>> >>>> property >>>>>>>>>>> >>>>>>>>>>> values that span multiple lines. So, I am re-writing it. It's >>>>>>>>>>> actually >>>>>>>>>>> a pretty hard thing to re-write: the original version took some >>>>>>>>>>> shortcuts that, as it turned out, didn't work out so well. >>>>>>>>>>> >>>>>>>>>>> Anyway, I thought I'd let everybody, especially Harry, know >>>> >>>> what's >>>>>>>>>>> >>>>>>>>>>> going on. It's blocking me from finishing BundleMigrator, and >>>> >>>> thus >>>>>>>>>>> >>>>>>>>>>> from doing more JSP migration work -- so I wanted to get it >>>> >>>> solved. >>>>>>>>>>> >>>>>>>>>>> Andrew >>>>>>>>>>> >>>>>>>>>> >>>>>>>>> >>>>>>>> >>>>>>> >>>>>> >>>>> >>>> >>> >>> > >
