[sqlite] FTS3 Unicode support

2009-03-26 Thread waterq
hi all: I'm working on full text search with sqlite-fts3 on chinese characters. I know that "For those wishing to perform full-text searching for non-English text, there is some basic functionality provided via the ICU library. The ICU tokenizer will break words on boundaries as identified by th

Re: [sqlite] FTS3 Unicode support

2008-01-30 Thread Myk Melez
Scott Hess wrote: The [3] status is ... pending, sorry :-(. But it is more along the lines of adding stuff to ICU rather than adding ICU-less stuff to SQLite, so it sounds like that is not relevant to what you're doing. Hi Scott, Thanks for the info. Indeed, enhancements to ICU don't sound

Re: [sqlite] FTS3 Unicode support

2008-01-30 Thread Scott Hess
The [3] status is ... pending, sorry :-(. But it is more along the lines of adding stuff to ICU rather than adding ICU-less stuff to SQLite, so it sounds like that is not relevant to what you're doing. As Dan mentioned, there's stuff in there for supporting alternate tokenizers, including an ICU-

Re: [sqlite] FTS3 Unicode support

2008-01-24 Thread Shawn Wilsher
The problem with ICU is that it's a rather large library, and mozilla already has it's own unicode system. That's we we opted on doing unicode support ourselves (less code duplication, and a smaller binary). Cheers, Shawn Wilsher On Jan 24, 2008 11:35 PM, Dan <[EMAIL PROTECTED]> wrote: > > On J

Re: [sqlite] FTS3 Unicode support

2008-01-24 Thread Dan
On Jan 25, 2008, at 7:26 AM, Myk Melez wrote: Hi all, I'm working to enable FTS3 in the next version of Firefox [1] so that extenders can take advantage of it, although Firefox itself isn't using it for the next release. Given Firefox's international audience, it would be useful for FTS3

[sqlite] FTS3 Unicode support

2008-01-24 Thread Myk Melez
Hi all, I'm working to enable FTS3 in the next version of Firefox [1] so that extenders can take advantage of it, although Firefox itself isn't using it for the next release. Given Firefox's international audience, it would be useful for FTS3 to support Unicode. We currently do this for upp