[
https://issues.apache.org/jira/browse/TIKA-4745?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18086077#comment-18086077
]
Hudson commented on TIKA-4745:
------------------------------
UNSTABLE: Integrated in Jenkins build Tika ยป tika-main-jdk17 #1402 (See
[https://ci-builds.apache.org/job/Tika/job/tika-main-jdk17/1402/])
TIKA-4745 - charset/junk/tika-eval improvements (#2861) (github:
[https://github.com/apache/tika/commit/4ce5c7011bba18889826d32799720768b5e6d42f])
* (edit)
tika-ml/tika-ml-junkdetect/src/main/java/org/apache/tika/ml/junkdetect/JunkFilterEncodingDetector.java
* (edit)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/resources/org/apache/tika/ml/chardetect/nb-bigram.bin
* (edit)
tika-ml/tika-ml-junkdetect/src/test/java/org/apache/tika/ml/junkdetect/JunkFilterEncodingDetectorTest.java
* (edit) .skills/tika-eval-encoding-regression.md
* (add)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/resources/org/apache/tika/ml/chardetect/cosine-profiles.bin
* (edit)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/java/org/apache/tika/ml/chardetect/MojibusterEncodingDetector.java
* (edit)
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ProfilerBase.java
* (edit)
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractComparer.java
* (add) tika-core/src/test/java/org/apache/tika/detect/CharsetSupersetsTest.java
* (add)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/textstats/ReplacementCharCounter.java
* (add)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/java/org/apache/tika/ml/chardetect/CosineFamilyArbiter.java
* (edit)
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/reports/MarkdownSummaryWriter.java
* (edit)
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractProfileRunner.java
* (edit) tika-core/src/main/java/org/apache/tika/detect/CharsetSupersets.java
* (edit)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/java/org/apache/tika/ml/chardetect/NaiveBayesBigramEncodingDetector.java
* (edit)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/langid/LanguageIDWrapper.java
* (add)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/main/java/org/apache/tika/ml/chardetect/CjkDecodeValidator.java
* (edit)
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/db/Cols.java
* (add) tika-core/src/main/java/org/apache/tika/detect/HighByteLetterStats.java
* (add)
tika-eval/tika-eval-core/src/main/java/org/apache/tika/eval/core/textstats/NonAsciiCharCounter.java
* (edit)
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractProfiler.java
* (edit)
docs/modules/ROOT/pages/advanced/integration-testing/tika-eval-regression.adoc
* (add)
tika-ml/tika-ml-junkdetect/src/test/java/org/apache/tika/ml/junkdetect/LatinLetterGateTest.java
* (edit) .skills/tika-eval-compare.md
* (add)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/test/java/org/apache/tika/ml/chardetect/CjkDecodeValidatorTest.java
* (add)
tika-encoding-detectors/tika-encoding-detector-mojibuster/src/test/java/org/apache/tika/ml/chardetect/Iso2022DetectionTest.java
* (edit)
tika-eval/tika-eval-app/src/main/java/org/apache/tika/eval/app/ExtractComparerRunner.java
* (edit) docs/modules/ROOT/pages/advanced/charset-detection-design.adoc
* (add)
tika-core/src/test/java/org/apache/tika/detect/HighByteLetterStatsTest.java
* (edit) .skills/tika-eval-h2-query.md
> Small improvements to lang detection, charset detection and junk detection
> --------------------------------------------------------------------------
>
> Key: TIKA-4745
> URL: https://issues.apache.org/jira/browse/TIKA-4745
> Project: Tika
> Issue Type: Task
> Reporter: Tim Allison
> Priority: Minor
>
> I ran a regression test in prep for the 4.0.0-beta-1 release. There are a
> number of smallish things that we can clean up in the components listed in
> the title.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
