On 7/7/06, Jens Kraemer <[EMAIL PROTECTED]> wrote:
> On Fri, Jul 07, 2006 at 05:29:50AM +0200, Ian Zabel wrote:
> > Thanks for the response!
> >
> > > I put this statement at the very top of the file, outside of the block.
> > > Maybe that will do the trick.
> > >
> > > You also should make sure the locale exists on your system. On
> > > a Debian-based system, you could do
> > > dpkg-reconfigure locales
> > > and make sure the box before "en_US.UTF-8" is ticked.
> >
> > I determined with `locale -a` that the locale on the box is called
> > "en_US.utf8", so I added "ENV['LANG'] = 'en_US.utf8'" at the top of my
> > environment.rb (right after "ENV['RAILS_ENV'] ||= 'production'".
> >
> > Still getting the same error: "Error decoding input string. Check that
> > you have the locale set correctly"
> >
> > :(
> >
> > It may be worth noting that it seems to only be a problem with this
> > particular model. I am able to index a different model without any
> > issues. So it's gotta be something with the data.
> >
> > I noticed that my InnoDB topics table was set to latin1 charset, so I
> > changed it to utf8. I still get the same error.
>
> Imho changing the default charset of a table doesn't change the encoding
> of the data stored in it. So that's still latin1 what you get from your
> DB.
>
> > Not sure where to go next.
>
> The ENV['LANG'] value has to correspond to the encoding of the data you
> want to index, so if your data is latin1, Ferret needs to run with such
> a locale, i.e. ISO-8859-1.
>
> In such cases I dump the data as text, convert to utf8 (usually
> with vim :set fileencoding=utf8), re-create the table with DEFAULT
> CHARSET UTF-8 and re-import the data.
>
> With large data sets other solutions might be more efficient, though.
Here's one way you can convert ISO-8859-1 to UTF-8;
str = str.unpack("C*).map {|c|
if c < 0x80
next c.chr
elsif c < 0xC0
next "\xC2" + c.chr
else
next "\xC3" + (c - 64).chr
end
}.join("")
That may help.
Cheers,
Dave
_______________________________________________
Ferret-talk mailing list
[email protected]
http://rubyforge.org/mailman/listinfo/ferret-talk
