Re: [sqlite] FTS4 languageid : not sure I understand this correctly

[Richard Hipp]

On Thu, Jan 31, 2013 at 3:18 PM, Gert Van Assche  wrote:

> All,
>
> I have the feeling this is the most stupid question ever, but...
> If I create a FTS4 table, put text in it, could I use the languageid to
> figure out what Language that text actually is?
> Is that how langID works?
>

No.  The languageID simply allows the same FTS4 table to hold separate and
independent content for multiple languages.  It is the same as having a
separate table for each languange, except that with languageID you can
choose which language in between sqlite3_prepare() and sqlite3_step()
whereas with separate tables, you must make the choice prior to
sqlite3_prepare().  That difference, though subtle, is very important for
some applications.



> I did some tests, but the LangID seems to be 0 all the time, so or I'm
> doing something wrong, or I misunderstand the process.
>
> thanks
>
> gert
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>



-- 
D. Richard Hipp
d...@sqlite.org
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] FTS4 languageid : not sure I understand this correctly

[Gert Van Assche]

thanks for this link. Most of these tools I tested in the past, but I was
hoping there was an easier path. The best langID tool is not in this list,
by the way. that is:
https://code.google.com/p/chromium-compact-language-detector/

gert

2013/1/31 Petite Abeille 

>
> On Jan 31, 2013, at 9:27 PM, Gert Van Assche  wrote:
>
> > Thanks Michael. Not what I hoped for but now I understand it.
>
> Perhaps of interest:
>
> Language Identification Tools
> http://www.let.rug.nl/~vannoord/TextCat/competitors.html
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] FTS4 languageid : not sure I understand this correctly

[Petite Abeille]

On Jan 31, 2013, at 9:27 PM, Gert Van Assche  wrote:

> Thanks Michael. Not what I hoped for but now I understand it.

Perhaps of interest:

Language Identification Tools
http://www.let.rug.nl/~vannoord/TextCat/competitors.html

___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users

Re: [sqlite] FTS4 languageid : not sure I understand this correctly

[Gert Van Assche]

Thanks Michael. Not what I hoped for but now I understand it.

2013/1/31 Michael Black 

> According to the docs:
> http://www.sqlite.org/fts3.html#section_6_3
>
> It's YOUR choice as to what to put in there.  A separate index is created
> for each language id.
> So it's designed to prevent cross-lang contamination in the index.
>
> -Original Message-
> From: sqlite-users-boun...@sqlite.org
> [mailto:sqlite-users-boun...@sqlite.org] On Behalf Of Gert Van Assche
> Sent: Thursday, January 31, 2013 2:19 PM
> To: sqlite-users
> Subject: [sqlite] FTS4 languageid : not sure I understand this correctly
>
> All,
>
> I have the feeling this is the most stupid question ever, but...
> If I create a FTS4 table, put text in it, could I use the languageid to
> figure out what Language that text actually is?
> Is that how langID works?
> I did some tests, but the LangID seems to be 0 all the time, so or I'm
> doing something wrong, or I misunderstand the process.
>
> thanks
>
> gert
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
> ___
> sqlite-users mailing list
> sqlite-users@sqlite.org
> http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users
>
___
sqlite-users mailing list
sqlite-users@sqlite.org
http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users