[
https://issues.apache.org/jira/browse/TIKA-1696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chris A. Mattmann resolved TIKA-1696.
-------------------------------------
Resolution: Fixed
This is now done, Ken's Optimaize langdetect, N-gram langdetect and Text.jl
from MIT are all now integrated:
{noformat}
LMC-053601:tika1.13 mattmann$ git commit -m "Resolve conflicts in CHANGES.txt"
[master 2caf3da] Resolve conflicts in CHANGES.txt
LMC-053601:tika1.13 mattmann$ git push -u origin master
Counting objects: 477, done.
Delta compression using up to 8 threads.
Compressing objects: 100% (237/237), done.
Writing objects: 100% (477/477), 113.91 KiB | 0 bytes/s, done.
Total 477 (delta 134), reused 320 (delta 67)
remote: tika git commit: Resolve conflicts in CHANGES.txt
remote: tika git commit: Update with information about TIKA-1872, TIKA-1696 and
TIKA-1723.
remote: tika git commit: Merge branch 'TIKA-1872'
remote: tika git commit: Merge branch 'TIKA-1872' of
https://github.com/trevorlewis/tika into TIKA-1872
remote: tika git commit: Updated TextLangDetector and fixed build errors
remote: tika git commit: Merge branch 'master' of
https://git-wip-us.apache.org/repos/asf/tika into TIKA-1872
remote: tika git commit: Depend on 1.13-SNAPSHOT, not 2.0.
remote: tika git commit: Merge branch 'TIKA-1872' of
https://github.com/trevorlewis/tika into TIKA-1872
remote: tika git commit: Added missing license headers
remote: tika git commit: Add missing license headers
remote: tika git commit: fix for TIKA-1872 contributed by trevorlewis
remote: tika git commit: Make detector "discoverable", use that everywhere
remote: tika git commit: Move base lang detect classes to core
remote: tika git commit: Remove built-in lang detector
remote: tika git commit: Add tika-langdetect dependency in other modules
remote: tika git commit: Add project.build.sourceEncoding to properties
remote: tika git commit: Roll in new lang detect support in new module
remote: tika git commit: Add missing dependency on tika-test-resources
To https://git-wip-us.apache.org/repos/asf/tika.git
c9d508d..2caf3da master -> master
Branch master set up to track remote
{noformat}
Thanks [~kkrugler] and [~lewistre]!
> Language Identification with Text Processing Toolkit from MITLL
> ---------------------------------------------------------------
>
> Key: TIKA-1696
> URL: https://issues.apache.org/jira/browse/TIKA-1696
> Project: Tika
> Issue Type: New Feature
> Components: languageidentifier
> Reporter: Paul Ramirez
> Assignee: Chris A. Mattmann
> Fix For: 1.13
>
>
> The aim here is to extend the methods for language identification within
> text. MIT Lincoln Labs has an open source library [1] written in Julia.
> Having spoken with the MITLL guys there is a possibility that there is a
> scala version of this library which would make it easier to package in with
> Tika.
> At this point I'm not quite sure how many languages this library supports by
> default but it can be extended when provided some training data.
> [1] https://github.com/mit-nlp/Text.jl
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
