Hi Stefan,
Do you mind opening a jira issue and attaching these patches there?
For NumberFormatTransformer, I guess we should add another attribute for
locale which should be used for parsing. The reason is that the data you are
processing may be coming from a different locale than the system locale.
What do you think?
On Thu, Aug 14, 2008 at 2:33 AM, Stefan Oestreicher <
[EMAIL PROTECTED]> wrote:
> Hi,
>
> I just ran the unit tests on my system (uses a german locale) and the
> DateFieldTest and LegacyDateFieldTest failed because the expected decimal
> format used "." as separator but the actual result was formatted using the
> "," separator.
> This happens because the ISO8601CanonicalDateFormat class which is defined
> inside the DateField class doesn't specify the US locale when creating the
> formatter used for milliseconds.
> I attached a patch which fixes that problem
> (schema.DateField-locale.patch.txt).
>
> TestNumberFormatTransformer in contrib/dataimporthandler also failed
> because handler.dataimport.NumberFormatTransformer uses a NumberFormatter
> which relies on the system locale. But I think in this case it's
> intentional, so I modified the test case to use the grouping separator of
> the system locale
> (handler.dataimport.TestNumberFormatTransformer-locale.patch.txt).
>
> After applying those changes all tests succeeded.
>
> kind regards,
>
> Stefan
>
> ### Eclipse Workspace Patch 1.0
> #P solr
> Index:
> contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
> ===================================================================
> ---
> contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
> (revision 685675)
> +++
> contrib/dataimporthandler/src/test/java/org/apache/solr/handler/dataimport/TestNumberFormatTransformer.java
> (working copy)
> @@ -19,6 +19,7 @@
> import org.junit.Assert;
> import org.junit.Test;
>
> +import java.text.DecimalFormatSymbols;
> import java.util.ArrayList;
> import java.util.List;
> import java.util.Map;
> @@ -32,6 +33,9 @@
> * @since solr 1.3
> */
> public class TestNumberFormatTransformer {
> + private static final char GRP_SEP = new DecimalFormatSymbols()
> + .getGroupingSeparator();
> +
> @Test
> @SuppressWarnings("unchecked")
> public void testTransformRow_SingleNumber() {
> @@ -40,7 +44,7 @@
> NumberFormatTransformer.FORMAT_STYLE,
> NumberFormatTransformer.NUMBER));
> Context c = AbstractDataImportHandlerTest.getContext(null, null, null,
> 0,
> l, null);
> - Map m = AbstractDataImportHandlerTest.createMap("num", "123,567");
> + Map m = AbstractDataImportHandlerTest.createMap("num",
> "123"+GRP_SEP+"567");
> new NumberFormatTransformer().transformRow(m, c);
> Assert.assertEquals(new Long(123567), m.get("num"));
> }
> @@ -56,8 +60,8 @@
> NumberFormatTransformer.FORMAT_STYLE,
> NumberFormatTransformer.NUMBER));
>
> List inputs = new ArrayList();
> - inputs.add("123,567");
> - inputs.add("245,678");
> + inputs.add("123"+GRP_SEP+"567");
> + inputs.add("245"+GRP_SEP+"678");
> Map row = AbstractDataImportHandlerTest.createMap("inputs", inputs);
>
> VariableResolverImpl resolver = new VariableResolverImpl();
>
> ### Eclipse Workspace Patch 1.0
> #P solr
> Index: src/java/org/apache/solr/schema/DateField.java
> ===================================================================
> --- src/java/org/apache/solr/schema/DateField.java (revision 685675)
> +++ src/java/org/apache/solr/schema/DateField.java (working copy)
> @@ -31,6 +31,7 @@
> import java.util.Date;
> import java.util.TimeZone;
> import java.util.Locale;
> +import java.text.DecimalFormatSymbols;
> import java.text.SimpleDateFormat;
> import java.text.DateFormat;
> import java.text.NumberFormat;
> @@ -245,7 +246,8 @@
> protected NumberFormat millisParser
> = NumberFormat.getIntegerInstance(Locale.US);
>
> - protected NumberFormat millisFormat = new DecimalFormat(".###");
> + protected NumberFormat millisFormat = new DecimalFormat(".###",
> + new DecimalFormatSymbols(Locale.US));
>
> public ISO8601CanonicalDateFormat() {
> super("yyyy-MM-dd'T'HH:mm:ss", Locale.US);
> @@ -295,7 +297,8 @@
> ISO8601CanonicalDateFormat c
> = (ISO8601CanonicalDateFormat) super.clone();
> c.millisParser = NumberFormat.getIntegerInstance(Locale.US);
> - c.millisFormat = new DecimalFormat(".###");
> + c.millisFormat = new DecimalFormat(".###",
> + new DecimalFormatSymbols(Locale.US));
> return c;
> }
> }
>
>
--
Regards,
Shalin Shekhar Mangar.
