[jira] [Commented] (JAMES-2302) SMTP configuration documentation refers to servername which does not exist anymore
[ https://issues.apache.org/jira/browse/JAMES-2302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16682245#comment-16682245 ] ASF GitHub Bot commented on JAMES-2302: --- GitHub user Daniel-Dos opened a pull request: https://github.com/apache/james-project/pull/143 JAMES-2302:SMTP configuration documentation refers to servername PR with fix to item https://issues.apache.org/jira/browse/JAMES-2302 You can merge this pull request into a Git repository by running: $ git pull https://github.com/Daniel-Dos/james-project JAMES-2302 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/james-project/pull/143.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #143 commit 56ae3ba1d88294e23c7d8925606324f7a6887a99 Author: Daniel Dias Date: 2018-11-10T06:51:51Z JAMES-2302:SMTP configuration documentation refers to servername which does not exist anymore > SMTP configuration documentation refers to servername which does not exist > anymore > -- > > Key: JAMES-2302 > URL: https://issues.apache.org/jira/browse/JAMES-2302 > Project: James Server > Issue Type: Bug >Reporter: Raphaël Ouazana >Priority: Major > Labels: documentation, easyfix > > The page https://james.apache.org/server/3/config-smtp-lmtp.html is > referencing the servername option which does not exist anymore. > It should be fixed, probably by replacing it by domains handling (see > domainlist.xml). -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Closed] (JAMES-2581) Configurable ContentType blacklist for Tika
[ https://issues.apache.org/jira/browse/JAMES-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antoine Duprat closed JAMES-2581. - > Configurable ContentType blacklist for Tika > --- > > Key: JAMES-2581 > URL: https://issues.apache.org/jira/browse/JAMES-2581 > Project: James Server > Issue Type: Improvement >Reporter: Dat Pham >Priority: Major > > Enhanced production logging upon Tika failing call highlight the fact that > our installation of Tika **can not** handle some kinds of attachments. > Here is a log example: > ``` > org.apache.http.client.HttpResponseException: Unsupported Media Type > at > org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:70) > at org.apache.http.client.fluent.Response.handleResponse(Response.java:90) > at org.apache.http.client.fluent.Response.returnContent(Response.java:97) > at > org.apache.james.mailbox.tika.TikaHttpClientImpl.recursiveMetaDataAsJson(TikaHttpClientImpl.java:62) > at > org.apache.james.mailbox.tika.TikaTextExtractor.performContentExtraction(TikaTextExtractor.java:86) > at > org.apache.james.mailbox.tika.TikaTextExtractor.lambda$extractContent$0(TikaTextExtractor.java:81) > ``` > (131 matches in the last 2 days) > Here is a list if Content types we recurringly fail on: > - application/ics > - application/zip > - application/pgp-signature > - image/jpg > - image/jpeg > - image/png > - message/delivery-status > As an admin, I should be able to specify in `tika.properties` file a coma > separated list of Content type to blacklist. > Benefits: > - Avoid known-to-be-failing Tika calls - reduce log output > - Avoid transmitting potentially big payload over the network for nothing - > performance -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Resolved] (JAMES-2581) Configurable ContentType blacklist for Tika
[ https://issues.apache.org/jira/browse/JAMES-2581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antoine Duprat resolved JAMES-2581. --- Resolution: Fixed merged > Configurable ContentType blacklist for Tika > --- > > Key: JAMES-2581 > URL: https://issues.apache.org/jira/browse/JAMES-2581 > Project: James Server > Issue Type: Improvement >Reporter: Dat Pham >Priority: Major > > Enhanced production logging upon Tika failing call highlight the fact that > our installation of Tika **can not** handle some kinds of attachments. > Here is a log example: > ``` > org.apache.http.client.HttpResponseException: Unsupported Media Type > at > org.apache.http.impl.client.AbstractResponseHandler.handleResponse(AbstractResponseHandler.java:70) > at org.apache.http.client.fluent.Response.handleResponse(Response.java:90) > at org.apache.http.client.fluent.Response.returnContent(Response.java:97) > at > org.apache.james.mailbox.tika.TikaHttpClientImpl.recursiveMetaDataAsJson(TikaHttpClientImpl.java:62) > at > org.apache.james.mailbox.tika.TikaTextExtractor.performContentExtraction(TikaTextExtractor.java:86) > at > org.apache.james.mailbox.tika.TikaTextExtractor.lambda$extractContent$0(TikaTextExtractor.java:81) > ``` > (131 matches in the last 2 days) > Here is a list if Content types we recurringly fail on: > - application/ics > - application/zip > - application/pgp-signature > - image/jpg > - image/jpeg > - image/png > - message/delivery-status > As an admin, I should be able to specify in `tika.properties` file a coma > separated list of Content type to blacklist. > Benefits: > - Avoid known-to-be-failing Tika calls - reduce log output > - Avoid transmitting potentially big payload over the network for nothing - > performance -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[4/4] james-project git commit: JAMES-2581 Using contentType blacklist from Tika configuration to Text Extractor
JAMES-2581 Using contentType blacklist from Tika configuration to Text Extractor
Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/501d3436
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/501d3436
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/501d3436
Branch: refs/heads/master
Commit: 501d3436e224a2d38a783425dc8117f200b8c656
Parents: 51f1d7b
Author: datph
Authored: Tue Nov 6 18:34:00 2018 +0700
Committer: datph
Committed: Fri Nov 9 16:28:02 2018 +0700
--
.../tika/ContentTypeFilteringTextExtractor.java | 14 ++-
.../tika/TextExtractorConfiguration.java| 94
.../james/mailbox/tika/TikaConfiguration.java | 16 ++--
.../ContentTypeFilteringTextExtractorTest.java | 10 +--
.../tika/TextExtractorConfigurationTest.java| 60 -
.../mailbox/TikaConfigurationReader.java| 10 ++-
.../modules/mailbox/TikaMailboxModule.java | 26 +-
.../mailbox/TikaConfigurationReaderTest.java| 8 +-
8 files changed, 30 insertions(+), 208 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/james-project/blob/501d3436/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java
--
diff --git
a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java
b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java
index 9b18a2f..c648610 100644
---
a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java
+++
b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/ContentTypeFilteringTextExtractor.java
@@ -20,19 +20,20 @@
package org.apache.james.mailbox.tika;
import java.io.InputStream;
-import java.util.Objects;
import org.apache.james.mailbox.extractor.ParsedContent;
import org.apache.james.mailbox.extractor.TextExtractor;
+import com.google.common.collect.ImmutableSet;
+
public class ContentTypeFilteringTextExtractor implements TextExtractor {
private final TextExtractor textExtractor;
-private final TextExtractorConfiguration textExtractorConfiguration;
+private final ImmutableSet contentTypeBlacklist;
-public ContentTypeFilteringTextExtractor(TextExtractor textExtractor,
TextExtractorConfiguration textExtractorConfiguration) {
+public ContentTypeFilteringTextExtractor(TextExtractor textExtractor,
ImmutableSet contentTypeBlacklist) {
this.textExtractor = textExtractor;
-this.textExtractorConfiguration = textExtractorConfiguration;
+this.contentTypeBlacklist = contentTypeBlacklist;
}
@Override
@@ -44,10 +45,7 @@ public class ContentTypeFilteringTextExtractor implements
TextExtractor {
}
private boolean isBlacklisted(String contentType) {
-return textExtractorConfiguration
-.getContentTypeBlacklist()
-.stream()
-.anyMatch(blackListItem -> Objects.equals(blackListItem,
contentType));
+return contentTypeBlacklist.contains(contentType);
}
}
http://git-wip-us.apache.org/repos/asf/james-project/blob/501d3436/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TextExtractorConfiguration.java
--
diff --git
a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TextExtractorConfiguration.java
b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TextExtractorConfiguration.java
deleted file mode 100644
index df5400a..000
---
a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TextExtractorConfiguration.java
+++ /dev/null
@@ -1,94 +0,0 @@
-/
- * Licensed to the Apache Software Foundation (ASF) under one *
- * or more contributor license agreements. See the NOTICE file *
- * distributed with this work for additional information*
- * regarding copyright ownership. The ASF licenses this file *
- * to you under the Apache License, Version 2.0 (the*
- * "License"); you may not use this file except in compliance *
- * with the License. You may obtain a copy of the License at *
- * *
- * http://www.apache.org/licenses/LICENSE-2.0 *
- * *
- * Unless required by applicable law or agreed to in writing, *
- * software distributed under the License is distributed on an *
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
- * KIND, either express or implied. See the License for the*
- * specific language
[2/4] james-project git commit: JAMES-2581 Add contentType blacklist for Tika Configuration
JAMES-2581 Add contentType blacklist for Tika Configuration
Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/51f1d7ba
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/51f1d7ba
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/51f1d7ba
Branch: refs/heads/master
Commit: 51f1d7ba254c4a8d1aef4056a5299e4d54df6171
Parents: d91fe69
Author: datph
Authored: Tue Nov 6 18:24:52 2018 +0700
Committer: datph
Committed: Fri Nov 9 16:28:02 2018 +0700
--
.../james/mailbox/tika/TikaConfiguration.java | 27 +--
.../mailbox/tika/TikaConfigurationTest.java | 33 +
.../mailbox/TikaConfigurationReader.java| 10 +++
.../mailbox/TikaConfigurationReaderTest.java| 74
4 files changed, 140 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/james-project/blob/51f1d7ba/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java
--
diff --git
a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java
b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java
index 113c1e9..ce978ab 100644
---
a/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java
+++
b/mailbox/tika/src/main/java/org/apache/james/mailbox/tika/TikaConfiguration.java
@@ -20,6 +20,7 @@
package org.apache.james.mailbox.tika;
import java.time.Duration;
+import java.util.List;
import java.util.Objects;
import java.util.Optional;
import java.util.concurrent.TimeUnit;
@@ -27,6 +28,7 @@ import java.util.concurrent.TimeUnit;
import org.apache.james.util.Port;
import com.google.common.base.Preconditions;
+import com.google.common.collect.ImmutableList;
import com.google.common.primitives.Ints;
public class TikaConfiguration {
@@ -39,6 +41,7 @@ public class TikaConfiguration {
private Optional timeoutInMillis;
private Optional cacheEvictionPeriod;
private Optional cacheWeightInBytes;
+private ImmutableList.Builder contentTypeBlacklist;
private Builder() {
isEnabled = Optional.empty();
@@ -48,6 +51,7 @@ public class TikaConfiguration {
timeoutInMillis = Optional.empty();
cacheEvictionPeriod = Optional.empty();
cacheWeightInBytes = Optional.empty();
+contentTypeBlacklist = ImmutableList.builder();
}
public Builder enable(Optional isEnabled) {
@@ -136,6 +140,12 @@ public class TikaConfiguration {
return this;
}
+public Builder contentTypeBlacklist(List contentTypeBlacklist)
{
+Preconditions.checkNotNull(contentTypeBlacklist);
+this.contentTypeBlacklist.addAll(contentTypeBlacklist);
+return this;
+}
+
public TikaConfiguration build() {
port.ifPresent(Port::assertValid);
@@ -146,7 +156,8 @@ public class TikaConfiguration {
port.orElse(DEFAULT_PORT),
timeoutInMillis.orElse(DEFAULT_TIMEOUT_IN_MS),
cacheEvictionPeriod.orElse(DEFAULT_CACHE_EVICTION_PERIOD),
-cacheWeightInBytes.orElse(DEFAULT_CACHE_LIMIT_100_MB));
+cacheWeightInBytes.orElse(DEFAULT_CACHE_LIMIT_100_MB),
+contentTypeBlacklist.build());
}
}
@@ -168,8 +179,9 @@ public class TikaConfiguration {
private final int timeoutInMillis;
private final Duration cacheEvictionPeriod;
private final long cacheWeightInBytes;
+private final ImmutableList contentTypeBlacklist;
-private TikaConfiguration(boolean enabled, boolean cacheEnabled, String
host, int port, int timeoutInMillis, Duration cacheEvictionPeriod, long
cacheWeightInBytes) {
+private TikaConfiguration(boolean enabled, boolean cacheEnabled, String
host, int port, int timeoutInMillis, Duration cacheEvictionPeriod, long
cacheWeightInBytes, ImmutableList contentTypeBlacklist) {
this.enabled = enabled;
this.cacheEnabled = cacheEnabled;
this.host = host;
@@ -177,6 +189,7 @@ public class TikaConfiguration {
this.timeoutInMillis = timeoutInMillis;
this.cacheEvictionPeriod = cacheEvictionPeriod;
this.cacheWeightInBytes = cacheWeightInBytes;
+this.contentTypeBlacklist = contentTypeBlacklist;
}
public boolean isEnabled() {
@@ -207,24 +220,30 @@ public class TikaConfiguration {
return cacheWeightInBytes;
}
+public List getContentTypeBlacklist() {
+return contentTypeBlacklist;
+}
+
@Override
public final boolean equals(Object o) {
if (o instanceof TikaConfiguration) {
[1/4] james-project git commit: JAMES-2581 Add contentType blacklist for text extractor
Repository: james-project
Updated Branches:
refs/heads/master 16ee6 -> cf2febf0c
JAMES-2581 Add contentType blacklist for text extractor
Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/d91fe698
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/d91fe698
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/d91fe698
Branch: refs/heads/master
Commit: d91fe6987fb6d1e7ff48c01a7d584bfb7c191dc8
Parents: 16eebbb
Author: datph
Authored: Tue Nov 6 11:34:01 2018 +0700
Committer: datph
Committed: Fri Nov 9 16:28:01 2018 +0700
--
.../james/mailbox/extractor/ParsedContent.java | 24 -
.../mailbox/extractor/ParsedContentTest.java| 33 +++
mailbox/tika/pom.xml| 5 ++
.../tika/ContentTypeFilteringTextExtractor.java | 53 +++
.../tika/TextExtractorConfiguration.java| 94
.../ContentTypeFilteringTextExtractorTest.java | 76
.../tika/TextExtractorConfigurationTest.java| 60 +
.../modules/mailbox/TikaMailboxModule.java | 37 ++--
8 files changed, 373 insertions(+), 9 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/james-project/blob/d91fe698/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/ParsedContent.java
--
diff --git
a/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/ParsedContent.java
b/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/ParsedContent.java
index 8ba90ab..4be6646 100644
---
a/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/ParsedContent.java
+++
b/mailbox/api/src/main/java/org/apache/james/mailbox/extractor/ParsedContent.java
@@ -21,8 +21,11 @@ package org.apache.james.mailbox.extractor;
import java.util.List;
import java.util.Map;
+import java.util.Objects;
import java.util.Optional;
+import com.google.common.collect.ImmutableMap;
+
public class ParsedContent {
private final Optional textualContent;
@@ -40,5 +43,24 @@ public class ParsedContent {
public Map> getMetadata() {
return metadata;
}
-
+
+public static ParsedContent empty() {
+return new ParsedContent(Optional.empty(), ImmutableMap.of());
+}
+
+@Override
+public final boolean equals(Object o) {
+if (o instanceof ParsedContent) {
+ParsedContent that = (ParsedContent) o;
+
+return Objects.equals(this.textualContent, that.textualContent)
+&& Objects.equals(this.metadata, that.metadata);
+}
+return false;
+}
+
+@Override
+public final int hashCode() {
+return Objects.hash(textualContent, metadata);
+}
}
http://git-wip-us.apache.org/repos/asf/james-project/blob/d91fe698/mailbox/api/src/test/java/org/apache/james/mailbox/extractor/ParsedContentTest.java
--
diff --git
a/mailbox/api/src/test/java/org/apache/james/mailbox/extractor/ParsedContentTest.java
b/mailbox/api/src/test/java/org/apache/james/mailbox/extractor/ParsedContentTest.java
new file mode 100644
index 000..be65e10
--- /dev/null
+++
b/mailbox/api/src/test/java/org/apache/james/mailbox/extractor/ParsedContentTest.java
@@ -0,0 +1,33 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one *
+ * or more contributor license agreements. See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership. The ASF licenses this file *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance *
+ * with the License. You may obtain a copy of the License at *
+ * *
+ * http://www.apache.org/licenses/LICENSE-2.0 *
+ * *
+ * Unless required by applicable law or agreed to in writing, *
+ * software distributed under the License is distributed on an *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY *
+ * KIND, either express or implied. See the License for the*
+ * specific language governing permissions and limitations *
+ * under the License. *
+ /
+
+package org.apache.james.mailbox.extractor;
+
+import org.junit.Test;
+
+import nl.jqno.equalsverifier.EqualsVerifier;
+
+public class ParsedContentTest {
+
+@Test
+public void shouldMatchBeanContract() {
+
[3/4] james-project git commit: JAMES-2581 Document the contentType blacklist
JAMES-2581 Document the contentType blacklist Project: http://git-wip-us.apache.org/repos/asf/james-project/repo Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/cf2febf0 Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/cf2febf0 Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/cf2febf0 Branch: refs/heads/master Commit: cf2febf0c32ead029d8f368f02901d0b52524b9b Parents: 501d343 Author: datph Authored: Thu Nov 8 18:14:39 2018 +0700 Committer: datph Committed: Fri Nov 9 16:28:02 2018 +0700 -- src/site/xdoc/server/config-elasticsearch.xml | 3 +++ 1 file changed, 3 insertions(+) -- http://git-wip-us.apache.org/repos/asf/james-project/blob/cf2febf0/src/site/xdoc/server/config-elasticsearch.xml -- diff --git a/src/site/xdoc/server/config-elasticsearch.xml b/src/site/xdoc/server/config-elasticsearch.xml index f6d9ead..e8e1373 100644 --- a/src/site/xdoc/server/config-elasticsearch.xml +++ b/src/site/xdoc/server/config-elasticsearch.xml @@ -153,6 +153,9 @@ A value of 0 disables the cache Please note that units are supported (K for KB, M for MB, G for GB). Defaults is no units, so in bytes. Default value is 100 MB. + +tika.contentType.blacklist +Blacklist of content type is known-to-be-failing with Tika. Specify the list with comma separator. Note: You can launch a tika server using this command line: - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
Re: Call for vote: Apache James 3.2.0
+1 downloaded tar from https://repository.apache.org/content/repositories/orgapachejames-1041/org/apache/james/james-server-app/3.2.0 an ran a few basic operations (send, receive mails) thx to all for this. On 11/6/18 4:25 PM, Benoit Tellier wrote: Hi, I would like to propose the 3.2.0 release of the Apache James server. You can see changes proposed to the website at the occasion of that release, as well as communication on this GitHub pull request: https://github.com/linagora/james-project/pull/1869 You can find: - The maven release staged in http://repository.apache.org/ as the artifact #1041 - 3.2.0 sources and binaries are uploaded on Apache dist. Please note that we included JPA Guice and JPA SMTP guice. - The changelog for 3.2.0: https://github.com/chibenwa/james-project/blob/3.2.0-release-doc/CHANGELOG.md - The compatibility instructions/upgrade recommendation: https://github.com/chibenwa/james-project/blob/3.2.0-release-doc/upgrade-instructions.md Voting rules: - This is a majority approval: this release may not be vetoed. - A quorum of 3 binding votes is required - The vote starts at Tuesday 6th of November 2018, 3pm30 UTC - The vote ends at Tuesday 13th of November 2018, 3pm30 UTC You can answer to it just with +1 and -1. Down-votes may be motivated. Cheers, Benoit Tellier - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
[jira] [Closed] (JAMES-2584) Define the subset of our tests which is specific to Cassandra
[ https://issues.apache.org/jira/browse/JAMES-2584?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Antoine Duprat closed JAMES-2584. - > Define the subset of our tests which is specific to Cassandra > - > > Key: JAMES-2584 > URL: https://issues.apache.org/jira/browse/JAMES-2584 > Project: James Server > Issue Type: New Feature > Components: integrationTests >Reporter: Antoine Duprat >Priority: Major > > This issue is linked to #JAMES-2580. > Here the goal is to isolate Cassandra releated integration tests. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org For additional commands, e-mail: server-dev-h...@james.apache.org
