[jira] [Created] (JAMES-2560) ICS sanitizing

2018-10-10 Thread Tellier Benoit (JIRA)
Tellier Benoit created JAMES-2560:
-

 Summary: ICS sanitizing
 Key: JAMES-2560
 URL: https://issues.apache.org/jira/browse/JAMES-2560
 Project: James Server
  Issue Type: Improvement
  Components: Mailet Contributions
Reporter: Tellier Benoit


Some ICS sender sends text/calendar as headers of an empty body part, causing 
the ICS to be ignored by the ICS pipeline.

We should sanitize such ICS in order to make them available as regular ICS 
attachment and processed by the ICS workflow.

This can, for instance, be done by a specific mailet.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[08/10] james-project git commit: JAMES-2525 adds createContainer operation to ObjectStorageBlobsDAO

2018-10-10 Thread btellier
JAMES-2525 adds createContainer operation to ObjectStorageBlobsDAO


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/a2c7e989
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/a2c7e989
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/a2c7e989

Branch: refs/heads/master
Commit: a2c7e989f662c70eb5d0f46031eba2e2433ed381
Parents: 7c57ae1
Author: Jean Helou 
Authored: Tue Sep 11 13:19:38 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:27:00 2018 +0700

--
 .../objectstorage/ObjectStorageBlobsDAO.java| 13 
 .../ObjectStorageBlobsDAOContract.java  |  9 ++
 .../ObjectStorageBlobsDAOTest.java  | 34 
 3 files changed, 42 insertions(+), 14 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/a2c7e989/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
index 176e5a4..1e0d12a 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAO.java
@@ -33,6 +33,7 @@ import 
org.apache.james.blob.objectstorage.swift.SwiftKeystone3ObjectStorage;
 import org.apache.james.blob.objectstorage.swift.SwiftTempAuthObjectStorage;
 import org.jclouds.blobstore.domain.Blob;
 import org.jclouds.blobstore.options.CopyOptions;
+import org.jclouds.domain.Location;
 
 import com.github.fge.lambdas.Throwing;
 import com.google.common.base.Preconditions;
@@ -41,6 +42,7 @@ import com.google.common.hash.HashingInputStream;
 
 public class ObjectStorageBlobsDAO implements BlobStore {
 private static final InputStream EMPTY_STREAM = new 
ByteArrayInputStream(new byte[0]);
+public static final Location DEFAULT_LOCATION = null;
 
 
 private final BlobId.Factory blobIdFactory;
@@ -67,6 +69,17 @@ public class ObjectStorageBlobsDAO implements BlobStore {
 return SwiftKeystone3ObjectStorage.daoBuilder(testConfig);
 }
 
+public CompletableFuture createContainer(ContainerName 
name) {
+return CompletableFuture.supplyAsync(() -> 
blobStore.createContainerInLocation(DEFAULT_LOCATION,
+name.value())).thenApply(created -> {
+if (created) {
+return name;
+} else {
+throw new ObjectStoreException("Unable to create container " + 
name.value());
+}
+});
+}
+
 @Override
 public CompletableFuture save(byte[] data) {
 return save(new ByteArrayInputStream(data));

http://git-wip-us.apache.org/repos/asf/james-project/blob/a2c7e989/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAOContract.java
--
diff --git 
a/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAOContract.java
 
b/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAOContract.java
index a6cb4b4..36501f3 100644
--- 
a/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAOContract.java
+++ 
b/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/ObjectStorageBlobsDAOContract.java
@@ -29,22 +29,17 @@ import java.util.concurrent.ExecutionException;
 import java.util.concurrent.TimeUnit;
 import java.util.concurrent.TimeoutException;
 
-
 import org.apache.james.blob.api.BlobId;
-import org.jclouds.blobstore.BlobStore;
-import org.jclouds.domain.Location;
 
-public interface ObjectStorageBlobsDAOContract {
 
-Location DEFAULT_LOCATION = null;
+public interface ObjectStorageBlobsDAOContract {
 
 ContainerName containerName();
 
 default void 
assertBlobsDAOCanStoreAndRetrieve(ObjectStorageBlobsDAOBuilder builder)
 throws InterruptedException, ExecutionException, TimeoutException {
-BlobStore blobStore = builder.getSupplier().get();
-blobStore.createContainerInLocation(DEFAULT_LOCATION, 
containerName().value());
 ObjectStorageBlobsDAO dao = builder.build();
+dao.createContainer(containerName());
 byte[] bytes = "content".getBytes(StandardCharsets.UTF_8);
 CompletableFuture save = dao.save(bytes);
 InputStream inputStream = save.thenApply(dao::read

[06/10] james-project git commit: JAMES-2525 moves common test components to guice-common

2018-10-10 Thread btellier
JAMES-2525 moves common test components to guice-common


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/5dff052d
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/5dff052d
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/5dff052d

Branch: refs/heads/master
Commit: 5dff052d10b0dbaa689e60614c5d5d54259ae51f
Parents: 8ecd8bb
Author: Jean Helou 
Authored: Mon Sep 10 14:53:55 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:26:15 2018 +0700

--
 .../james/AggregateGuiceModuleTestRule.java | 82 
 .../org/apache/james/GuiceModuleTestRule.java   | 30 ---
 .../apache/james/TempFilesystemTestRule.java| 54 -
 .../james/AggregateGuiceModuleTestRule.java | 82 
 .../org/apache/james/GuiceModuleTestRule.java   | 30 +++
 .../apache/james/TempFilesystemTestRule.java| 54 +
 6 files changed, 166 insertions(+), 166 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/5dff052d/server/container/guice/cassandra-guice/src/test/java/org/apache/james/AggregateGuiceModuleTestRule.java
--
diff --git 
a/server/container/guice/cassandra-guice/src/test/java/org/apache/james/AggregateGuiceModuleTestRule.java
 
b/server/container/guice/cassandra-guice/src/test/java/org/apache/james/AggregateGuiceModuleTestRule.java
deleted file mode 100644
index b3c4f71..000
--- 
a/server/container/guice/cassandra-guice/src/test/java/org/apache/james/AggregateGuiceModuleTestRule.java
+++ /dev/null
@@ -1,82 +0,0 @@
-/
- * Licensed to the Apache Software Foundation (ASF) under one   *
- * or more contributor license agreements.  See the NOTICE file *
- * distributed with this work for additional information*
- * regarding copyright ownership.  The ASF licenses this file   *
- * to you under the Apache License, Version 2.0 (the*
- * "License"); you may not use this file except in compliance   *
- * with the License.  You may obtain a copy of the License at   *
- *  *
- *   http://www.apache.org/licenses/LICENSE-2.0 *
- *  *
- * Unless required by applicable law or agreed to in writing,   *
- * software distributed under the License is distributed on an  *
- * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY   *
- * KIND, either express or implied.  See the License for the*
- * specific language governing permissions and limitations  *
- * under the License.   *
- /
-
-package org.apache.james;
-
-import java.util.List;
-
-import org.junit.rules.RuleChain;
-import org.junit.runner.Description;
-import org.junit.runners.model.Statement;
-
-import com.github.steveash.guavate.Guavate;
-import com.google.common.collect.ImmutableList;
-import com.google.inject.Module;
-import com.google.inject.util.Modules;
-
-
-public class AggregateGuiceModuleTestRule implements GuiceModuleTestRule {
-
-public static AggregateGuiceModuleTestRule of(GuiceModuleTestRule... 
subrule) {
-return new AggregateGuiceModuleTestRule(ImmutableList.copyOf(subrule));
-}
-
-private final List subrule;
-private final RuleChain chain;
-
-private AggregateGuiceModuleTestRule(List subrule) {
-this.subrule = subrule;
-this.chain = subrule
-.stream()
-.reduce(RuleChain.emptyRuleChain(),
-RuleChain::around,
-RuleChain::around);
-}
-
-public AggregateGuiceModuleTestRule aggregate(GuiceModuleTestRule... 
subrule) {
-List guiceModules = 
ImmutableList.builder()
-.addAll(this.subrule)
-.add(subrule)
-.build();
-
-return new AggregateGuiceModuleTestRule(guiceModules);
-}
-
-@Override
-public Module getModule() {
-List modules = subrule
-.stream()
-.map(GuiceModuleTestRule::getModule)
-.collect(Guavate.toImmutableList());
-
-return Modules.combine(modules);
-}
-
-@Override
-public Statement apply(Statement base, Description description) {
-return chain.apply(base, description);
-}
-
-@Override
-public void await() {
-subrule
-.parallelStream()
-.forEach(GuiceModuleTestRule::await);
-}
-}

http://git-wip-us.apache.org/repos/asf/james-project/blob/5dff052d/server/container/guice/cassan

[07/10] james-project git commit: JAMES-2525 enforce singleton swift container for test

2018-10-10 Thread btellier
JAMES-2525 enforce singleton swift container for test


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/7c57ae17
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/7c57ae17
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/7c57ae17

Branch: refs/heads/master
Commit: 7c57ae178a75ccbb63e12df59ed431f31ee644d6
Parents: 5dff052
Author: Jean Helou 
Authored: Mon Sep 10 16:08:07 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:26:49 2018 +0700

--
 .../objectstorage/DockerSwiftContainer.java | 134 +++
 .../objectstorage/DockerSwiftExtension.java |  52 +--
 .../blob/objectstorage/DockerSwiftRule.java |  49 +++
 .../objectstorage/DockerSwiftSingleton.java |  31 +
 4 files changed, 217 insertions(+), 49 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/7c57ae17/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/DockerSwiftContainer.java
--
diff --git 
a/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/DockerSwiftContainer.java
 
b/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/DockerSwiftContainer.java
new file mode 100644
index 000..b073eed
--- /dev/null
+++ 
b/server/blob/blob-objectstorage/src/test/java/org/apache/james/blob/objectstorage/DockerSwiftContainer.java
@@ -0,0 +1,134 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one   *
+ * or more contributor license agreements.  See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership.  The ASF licenses this file   *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance   *
+ * with the License.  You may obtain a copy of the License at   *
+ *  *
+ *   http://www.apache.org/licenses/LICENSE-2.0 *
+ *  *
+ * Unless required by applicable law or agreed to in writing,   *
+ * software distributed under the License is distributed on an  *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY   *
+ * KIND, either express or implied.  See the License for the*
+ * specific language governing permissions and limitations  *
+ * under the License.   *
+ /
+
+package org.apache.james.blob.objectstorage;
+
+import static org.testcontainers.containers.wait.strategy.Wait.forHttp;
+
+import java.net.URI;
+
+import org.apache.james.util.Host;
+import org.apache.james.util.docker.RateLimiters;
+import org.slf4j.Logger;
+import org.slf4j.LoggerFactory;
+import org.testcontainers.containers.Container;
+import org.testcontainers.containers.GenericContainer;
+import org.testcontainers.containers.output.OutputFrame;
+import org.testcontainers.containers.wait.strategy.WaitAllStrategy;
+
+import com.github.fge.lambdas.Throwing;
+
+public class DockerSwiftContainer {
+private static final Logger LOGGER = 
LoggerFactory.getLogger(DockerSwiftContainer.class);
+private static final String SWIFT_DOCKER_IMAGE = 
"jeantil/openstack-keystone-swift:pike";
+private static final int SWIFT_PORT = 8080;
+private static final int KEYSTONE_ADMIN_PORT = 35357;
+private final GenericContainer swiftContainer;
+private DockerSwift dockerSwift;
+
+public DockerSwiftContainer() {
+this.swiftContainer = new GenericContainer<>(SWIFT_DOCKER_IMAGE)
+.withExposedPorts(KEYSTONE_ADMIN_PORT)
+.withExposedPorts(SWIFT_PORT)
+.withLogConsumer(DockerSwiftContainer::displayDockerLog)
+.waitingFor(
+new WaitAllStrategy()
+.withStrategy(
+forHttp("/v3")
+.forPort(KEYSTONE_ADMIN_PORT)
+.forStatusCode(200)
+.withRateLimiter(RateLimiters.DEFAULT)
+).withStrategy(
+forHttp("/info")
+.forPort(SWIFT_PORT)
+.forStatusCode(200)
+.withRateLimiter(RateLimiters.DEFAULT)
+)
+);
+
+}
+
+public void start() {
+swiftContainer.start();
+Integer swiftPort = swiftContainer.getMappedPort(SWIFT_PORT);
+String containerIpAddress = swiftContainer.getContainerIpAddress();
+Container

[04/10] james-project git commit: JAMES-2525 adds blob-objectstorage-guice module to maven

2018-10-10 Thread btellier
JAMES-2525 adds blob-objectstorage-guice module to maven


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/7bb6e802
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/7bb6e802
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/7bb6e802

Branch: refs/heads/master
Commit: 7bb6e80223d8ab504cd1c5360e4876baf99dc816
Parents: 53eeb3f
Author: Jean Helou 
Authored: Wed Sep 5 11:59:28 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:26:14 2018 +0700

--
 pom.xml | 22 +++
 .../guice/blob-objectstorage-guice/pom.xml  | 62 
 server/container/guice/pom.xml  |  1 +
 3 files changed, 85 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/7bb6e802/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 0b43776..c93c920 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1061,6 +1061,28 @@
 
 
 ${james.groupId}
+blob-objectstorage
+${project.version}
+
+
+${james.groupId}
+blob-objectstorage
+${project.version}
+test-jar
+
+
+${james.groupId}
+blob-objectstorage-guice
+${project.version}
+
+
+${james.groupId}
+blob-objectstorage-guice
+${project.version}
+test-jar
+
+
+${james.groupId}
 blob-cassandra
 ${project.version}
 

http://git-wip-us.apache.org/repos/asf/james-project/blob/7bb6e802/server/container/guice/blob-objectstorage-guice/pom.xml
--
diff --git a/server/container/guice/blob-objectstorage-guice/pom.xml 
b/server/container/guice/blob-objectstorage-guice/pom.xml
new file mode 100644
index 000..24e96d4
--- /dev/null
+++ b/server/container/guice/blob-objectstorage-guice/pom.xml
@@ -0,0 +1,62 @@
+
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd";>
+
+4.0.0
+
+
+org.apache.james
+james-server-guice
+3.2.0-SNAPSHOT
+../pom.xml
+
+
+blob-objectstorage-guice
+jar
+
+Apache James :: Server :: Blob Store in object storage - guice 
injection
+An advanced email server - Object storage based Blob Store 
bindings
+
+
+
+${james.groupId}
+blob-objectstorage
+
+
+${james.groupId}
+james-server-guice-common
+
+
+org.assertj
+assertj-core
+test
+
+
+org.junit.jupiter
+junit-jupiter-engine
+test
+
+
+org.junit.platform
+junit-platform-launcher
+test
+
+
+
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/james-project/blob/7bb6e802/server/container/guice/pom.xml
--
diff --git a/server/container/guice/pom.xml b/server/container/guice/pom.xml
index 73a6a93..e57a43f 100644
--- a/server/container/guice/pom.xml
+++ b/server/container/guice/pom.xml
@@ -34,6 +34,7 @@
 
 
 blob-api-guice
+blob-objectstorage-guice
 cassandra-guice
 cassandra-ldap-guice
 configuration


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Commented] (JAMES-2525) Implement swift Object Store

2018-10-10 Thread Tellier Benoit (JIRA)


[ 
https://issues.apache.org/jira/browse/JAMES-2525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16645866#comment-16645866
 ] 

Tellier Benoit commented on JAMES-2525:
---

https://github.com/linagora/james-project/pull/1782 Provides Guice bindings for 
object stores

> Implement swift Object Store
> 
>
> Key: JAMES-2525
> URL: https://issues.apache.org/jira/browse/JAMES-2525
> Project: James Server
>  Issue Type: New Feature
>  Components: James Core
>Reporter: Jean Helou
>Priority: Major
> Fix For: 3.2.0
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[09/10] james-project git commit: JAMES-2525 creates guice rule and cucumber singleton for swift docker container

2018-10-10 Thread btellier
JAMES-2525 creates guice rule and cucumber singleton for swift docker container


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/7554b2e0
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/7554b2e0
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/7554b2e0

Branch: refs/heads/master
Commit: 7554b2e0caaed7f3cfed15a49a2e9b6ee52fe3a0
Parents: a2c7e98
Author: Jean Helou 
Authored: Tue Sep 11 13:20:53 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:27:08 2018 +0700

--
 .../guice/blob-objectstorage-guice/pom.xml  | 18 
 .../guice/DockerSwiftTestRule.java  | 87 
 .../cassandra-jmap-integration-testing/pom.xml  | 17 
 .../cucumber/CucumberSwiftSingleton.java| 27 ++
 4 files changed, 149 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/7554b2e0/server/container/guice/blob-objectstorage-guice/pom.xml
--
diff --git a/server/container/guice/blob-objectstorage-guice/pom.xml 
b/server/container/guice/blob-objectstorage-guice/pom.xml
index 24e96d4..20e74f0 100644
--- a/server/container/guice/blob-objectstorage-guice/pom.xml
+++ b/server/container/guice/blob-objectstorage-guice/pom.xml
@@ -41,9 +41,22 @@
 
 
 ${james.groupId}
+blob-objectstorage
+${project.version}
+test-jar
+test
+
+
+${james.groupId}
 james-server-guice-common
 
 
+${james.groupId}
+james-server-guice-common
+test-jar
+test
+
+
 org.assertj
 assertj-core
 test
@@ -58,5 +71,10 @@
 junit-platform-launcher
 test
 
+
+org.testcontainers
+testcontainers
+test
+
 
 
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/james-project/blob/7554b2e0/server/container/guice/blob-objectstorage-guice/src/test/java/org/apache/james/modules/objectstorage/guice/DockerSwiftTestRule.java
--
diff --git 
a/server/container/guice/blob-objectstorage-guice/src/test/java/org/apache/james/modules/objectstorage/guice/DockerSwiftTestRule.java
 
b/server/container/guice/blob-objectstorage-guice/src/test/java/org/apache/james/modules/objectstorage/guice/DockerSwiftTestRule.java
new file mode 100644
index 000..ac5c459
--- /dev/null
+++ 
b/server/container/guice/blob-objectstorage-guice/src/test/java/org/apache/james/modules/objectstorage/guice/DockerSwiftTestRule.java
@@ -0,0 +1,87 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one   *
+ * or more contributor license agreements.  See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership.  The ASF licenses this file   *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance   *
+ * with the License.  You may obtain a copy of the License at   *
+ *  *
+ *   http://www.apache.org/licenses/LICENSE-2.0 *
+ *  *
+ * Unless required by applicable law or agreed to in writing,   *
+ * software distributed under the License is distributed on an  *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY   *
+ * KIND, either express or implied.  See the License for the*
+ * specific language governing permissions and limitations  *
+ * under the License.   *
+ /
+
+package org.apache.james.modules.objectstorage.guice;
+
+import java.util.UUID;
+
+import org.apache.james.GuiceModuleTestRule;
+import org.apache.james.blob.api.BlobStore;
+import org.apache.james.blob.api.HashBlobId;
+import org.apache.james.blob.objectstorage.ContainerName;
+import org.apache.james.blob.objectstorage.ObjectStorageBlobsDAO;
+import org.apache.james.blob.objectstorage.swift.Credentials;
+import org.apache.james.blob.objectstorage.swift.SwiftKeystone2ObjectStorage;
+import org.apache.james.blob.objectstorage.swift.TenantName;
+import org.apache.james.blob.objectstorage.swift.UserName;
+import org.junit.runner.Description;
+import org.junit.runners.model.Statement;
+import org.testcontainers.containers.GenericContainer;
+
+import com.github.fge.lambdas.Throwing;
+import com.google.inject.Mod

[03/10] james-project git commit: JAMES-2525 Provider and bindings for ObjectStorageDAO

2018-10-10 Thread btellier
JAMES-2525 Provider and bindings for ObjectStorageDAO


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/8ecd8bbf
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/8ecd8bbf
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/8ecd8bbf

Branch: refs/heads/master
Commit: 8ecd8bbf967970bd0c19b3eeea61283d5ea008a3
Parents: a4872d6
Author: Jean Helou 
Authored: Wed Sep 5 15:34:36 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:26:14 2018 +0700

--
 .../swift/SwiftKeystone2ObjectStorage.java  |   2 +
 .../swift/SwiftKeystone3ObjectStorage.java  |   2 +
 .../swift/SwiftTempAuthObjectStorage.java   |   2 +
 .../ObjectStorageBlobStoreModule.java   |  35 ++
 .../ObjectStorageBlobsDAOProvider.java  | 116 +++
 5 files changed, 157 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/8ecd8bbf/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
index bf75bcd..fcb1833 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
@@ -38,6 +38,8 @@ import com.google.common.collect.ImmutableSet;
 import com.google.inject.Module;
 
 public class SwiftKeystone2ObjectStorage {
+public static final String AUTH_API_NAME = "keystone2";
+
 private static final Iterable JCLOUDS_MODULES =
 ImmutableSet.of(new SLF4JLoggingModule());
 

http://git-wip-us.apache.org/repos/asf/james-project/blob/8ecd8bbf/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
index c0dcee3..13350ea 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
@@ -40,6 +40,8 @@ import com.google.common.collect.ImmutableSet;
 import com.google.inject.Module;
 
 public class SwiftKeystone3ObjectStorage {
+public static final String AUTH_API_NAME = "keystone2";
+
 private static final Iterable JCLOUDS_MODULES =
 ImmutableSet.of(new SLF4JLoggingModule());
 

http://git-wip-us.apache.org/repos/asf/james-project/blob/8ecd8bbf/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
index b5b1c22..631d63d 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
@@ -39,6 +39,8 @@ import com.google.common.collect.ImmutableSet;
 import com.google.inject.Module;
 
 public class SwiftTempAuthObjectStorage {
+public static final String AUTH_API_NAME = "tmpauth";
+
 private static final Iterable JCLOUDS_MODULES =
 ImmutableSet.of(new SLF4JLoggingModule());
 

http://git-wip-us.apache.org/repos/asf/james-project/blob/8ecd8bbf/server/container/guice/blob-objectstorage-guice/src/main/java/org/apache/james/modules/objectstorage/ObjectStorageBlobStoreModule.java
--
diff --git 
a/server/container/guice/blob-objectstorage-guice/src/main/java/org/apache/james/modules/objectstorage/ObjectStorageBlobStoreModule.java
 
b/server/container/guice/blob-objectstorage-guice/src/main/java/org/apache/james/modules/objectstorage/ObjectStorageBlobStoreModule.java
new file mode 100644
index 000..85ceb

[05/10] james-project git commit: JAMES-2525 extracts common blob store guice module

2018-10-10 Thread btellier
JAMES-2525 extracts common blob store guice module


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/53eeb3fb
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/53eeb3fb
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/53eeb3fb

Branch: refs/heads/master
Commit: 53eeb3fbb0f4c5a392b03d6c1a9f3e8d1c8c26d2
Parents: 27ef335
Author: Jean Helou 
Authored: Tue Sep 4 10:25:21 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:26:14 2018 +0700

--
 pom.xml | 11 
 server/container/guice/blob-api-guice/pom.xml   | 58 
 .../modules/mailbox/BlobStoreAPIModule.java | 34 
 server/container/guice/cassandra-guice/pom.xml  |  5 ++
 .../apache/james/CassandraJamesServerMain.java  |  2 +
 .../mailbox/CassandraObjectStoreModule.java |  4 --
 server/container/guice/pom.xml  |  1 +
 7 files changed, 111 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/53eeb3fb/pom.xml
--
diff --git a/pom.xml b/pom.xml
index 6a3dda1..0b43776 100644
--- a/pom.xml
+++ b/pom.xml
@@ -1050,6 +1050,17 @@
 
 
 ${james.groupId}
+blob-api-guice
+${project.version}
+
+
+${james.groupId}
+blob-api-guice
+${project.version}
+test-jar
+
+
+${james.groupId}
 blob-cassandra
 ${project.version}
 

http://git-wip-us.apache.org/repos/asf/james-project/blob/53eeb3fb/server/container/guice/blob-api-guice/pom.xml
--
diff --git a/server/container/guice/blob-api-guice/pom.xml 
b/server/container/guice/blob-api-guice/pom.xml
new file mode 100644
index 000..41f5881
--- /dev/null
+++ b/server/container/guice/blob-api-guice/pom.xml
@@ -0,0 +1,58 @@
+
+
+http://maven.apache.org/POM/4.0.0"; 
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"; 
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 
http://maven.apache.org/maven-v4_0_0.xsd";>
+
+4.0.0
+
+
+org.apache.james
+james-server-guice
+3.2.0-SNAPSHOT
+../pom.xml
+
+
+blob-api-guice
+jar
+
+Apache James :: Server :: Blob Store API - guice injection
+An advanced email server - Common object storage 
bindings
+
+
+
+${james.groupId}
+blob-api
+
+
+${james.groupId}
+james-server-guice-common
+
+
+
+
+
+org.apache.maven.plugins
+maven-surefire-plugin
+
+true
+
+
+
+
+
\ No newline at end of file

http://git-wip-us.apache.org/repos/asf/james-project/blob/53eeb3fb/server/container/guice/blob-api-guice/src/main/java/org/apache/james/modules/mailbox/BlobStoreAPIModule.java
--
diff --git 
a/server/container/guice/blob-api-guice/src/main/java/org/apache/james/modules/mailbox/BlobStoreAPIModule.java
 
b/server/container/guice/blob-api-guice/src/main/java/org/apache/james/modules/mailbox/BlobStoreAPIModule.java
new file mode 100644
index 000..4887488
--- /dev/null
+++ 
b/server/container/guice/blob-api-guice/src/main/java/org/apache/james/modules/mailbox/BlobStoreAPIModule.java
@@ -0,0 +1,34 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one   *
+ * or more contributor license agreements.  See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership.  The ASF licenses this file   *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance   *
+ * with the License.  You may obtain a copy of the License at   *
+ *  *
+ *   http://www.apache.org/licenses/LICENSE-2.0 *
+ *  *
+ * Unless required by applicable law or agreed to in writing,   *
+ * software distributed under the License is distributed on an  *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY   *
+ * KIND, either express or implied.  See the License for the*
+ * specific language governing permissions and limitations  *
+ * under the License.  

[10/10] james-project git commit: JAMES-2558 Upgrade to Apache Camel 2.22.1

2018-10-10 Thread btellier
JAMES-2558 Upgrade to Apache Camel 2.22.1

Fixes CVE-2018-8041


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/8f5df497
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/8f5df497
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/8f5df497

Branch: refs/heads/master
Commit: 8f5df4978450810394c2d420cd33a04262379920
Parents: 7554b2e
Author: Benoit Tellier 
Authored: Wed Oct 10 15:31:22 2018 +0700
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:27:29 2018 +0700

--
 pom.xml | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/8f5df497/pom.xml
--
diff --git a/pom.xml b/pom.xml
index c93c920..80d1b71 100644
--- a/pom.xml
+++ b/pom.xml
@@ -588,7 +588,7 @@
 5.15.5
 0.8.2
 3.0.0
-2.22.0
+2.22.1
 10.14.2.0
 1.1.1
 0.94.27


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[01/10] james-project git commit: JAMES-2525 fixes broken equality introduced in previous commit

2018-10-10 Thread btellier
Repository: james-project
Updated Branches:
  refs/heads/master 34f102990 -> 8f5df4978


JAMES-2525 fixes broken equality introduced in previous commit


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/27ef3356
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/27ef3356
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/27ef3356

Branch: refs/heads/master
Commit: 27ef335611e3d162755f4c5ce7237ab3d30a9158
Parents: 34f1029
Author: Jean Helou 
Authored: Fri Oct 5 17:02:42 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:25:13 2018 +0700

--
 .../blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java  | 2 +-
 .../james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/27ef3356/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
index 2e1986d..f5e7272 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
@@ -209,7 +209,7 @@ public class SwiftKeystone3ObjectStorage {
 
 @Override
 public boolean equals(Object o) {
-if (o instanceof SwiftKeystone2ObjectStorage.Configuration) {
+if (o instanceof SwiftKeystone3ObjectStorage.Configuration) {
 Configuration that = (Configuration) o;
 return Objects.equal(endpoint, that.endpoint) &&
 Objects.equal(identity, that.identity) &&

http://git-wip-us.apache.org/repos/asf/james-project/blob/27ef3356/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
index 48eeeb2..e8f1129 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
@@ -195,7 +195,7 @@ public class SwiftTempAuthObjectStorage {
 
 @Override
 public boolean equals(Object o) {
-if (o instanceof SwiftKeystone2ObjectStorage.Configuration) {
+if (o instanceof SwiftTempAuthObjectStorage.Configuration) {
 Configuration that = (Configuration) o;
 return Objects.equal(endpoint, that.endpoint) &&
 Objects.equal(identity, that.identity) &&


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[02/10] james-project git commit: JAMES-2525 implements swift auth protocol config readers

2018-10-10 Thread btellier
JAMES-2525 implements swift auth protocol config readers


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/a4872d69
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/a4872d69
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/a4872d69

Branch: refs/heads/master
Commit: a4872d69a9d62aad11fb3331e7daa192536ebd04
Parents: 7bb6e80
Author: Jean Helou 
Authored: Wed Sep 5 12:00:01 2018 +0200
Committer: Benoit Tellier 
Committed: Thu Oct 11 09:26:14 2018 +0700

--
 .../swift/SwiftKeystone2ObjectStorage.java  |   4 +-
 .../swift/SwiftKeystone3ObjectStorage.java  |  14 +-
 .../swift/SwiftTempAuthObjectStorage.java   |  15 ++
 .../objectstorage/SwiftConfigurationReader.java |  26 ++
 .../SwiftKeystone2ConfigurationReader.java  |  74 ++
 .../SwiftKeystone3ConfigurationReader.java  | 134 ++
 .../SwiftTmpAuthConfigurationReader.java|  90 +++
 .../SwiftKeystone2ConfigurationReaderTest.java  | 150 +++
 .../SwiftKeystone3ConfigurationReaderTest.java  | 254 +++
 .../SwiftTmpAuthConfigurationReaderTest.java| 183 +
 10 files changed, 940 insertions(+), 4 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/a4872d69/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
index 5ace285..bf75bcd 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone2ObjectStorage.java
@@ -114,8 +114,8 @@ public class SwiftKeystone2ObjectStorage {
 return this;
 }
 
-public Builder region(Region region) {
-this.region = Optional.of(region);
+public Builder region(Optional region) {
+this.region = region;
 return this;
 }
 

http://git-wip-us.apache.org/repos/asf/james-project/blob/a4872d69/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
index f5e7272..c0dcee3 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftKeystone3ObjectStorage.java
@@ -110,8 +110,13 @@ public class SwiftKeystone3ObjectStorage {
 return this;
 }
 
-public Builder region(Region region) {
-this.region = Optional.of(region);
+public Builder region(Optional region) {
+this.region = region;
+return this;
+}
+
+public Builder domainId(Optional domainId) {
+this.domainId = domainId;
 return this;
 }
 
@@ -120,6 +125,11 @@ public class SwiftKeystone3ObjectStorage {
 return this;
 }
 
+public Builder project(Optional project) {
+this.project = project;
+return this;
+}
+
 public Builder project(Project project) {
 this.project = Optional.of(project);
 return this;

http://git-wip-us.apache.org/repos/asf/james-project/blob/a4872d69/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
--
diff --git 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
index e8f1129..b5b1c22 100644
--- 
a/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/objectstorage/swift/SwiftTempAuthObjectStorage.java
+++ 
b/server/blob/blob-objectstorage/src/main/java/org/apache/james/blob/

[jira] [Created] (JAMES-2559) Branch merge should be on a specific original branch

2018-10-10 Thread Antoine Duprat (JIRA)
Antoine Duprat created JAMES-2559:
-

 Summary: Branch merge should be on a specific original branch
 Key: JAMES-2559
 URL: https://issues.apache.org/jira/browse/JAMES-2559
 Project: James Server
  Issue Type: New Feature
  Components: docker
Reporter: Antoine Duprat


Actually, the branch merging tool is merging one branch into `master`.
In order to be more flexible, we have to introduce a new parameter and merge 
the branch into the given original branch.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[13/14] james-project git commit: JAMES-2554 Add test on Cassandra MailRepository storing implementation The aim of those tests is to prouve the order of storing data: MimeMessage -> message blob part

2018-10-10 Thread btellier
JAMES-2554 Add test on Cassandra MailRepository storing implementation
The aim of those tests is to prouve the order of storing data:
MimeMessage -> message blob parts -> keys -> mail repository size


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/e536e62f
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/e536e62f
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/e536e62f

Branch: refs/heads/master
Commit: e536e62fabe75a5463f345a4b72db5f8dcadc162
Parents: 33bfd0d
Author: Antoine Duprat 
Authored: Mon Oct 8 15:20:36 2018 +0200
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:47:38 2018 +0700

--
 ...ilRepositoryWithFakeImplementationsTest.java | 291 +++
 1 file changed, 291 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/e536e62f/server/mailrepository/mailrepository-cassandra/src/test/java/org/apache/james/mailrepository/cassandra/CassandraMailRepositoryWithFakeImplementationsTest.java
--
diff --git 
a/server/mailrepository/mailrepository-cassandra/src/test/java/org/apache/james/mailrepository/cassandra/CassandraMailRepositoryWithFakeImplementationsTest.java
 
b/server/mailrepository/mailrepository-cassandra/src/test/java/org/apache/james/mailrepository/cassandra/CassandraMailRepositoryWithFakeImplementationsTest.java
new file mode 100644
index 000..75aa16a
--- /dev/null
+++ 
b/server/mailrepository/mailrepository-cassandra/src/test/java/org/apache/james/mailrepository/cassandra/CassandraMailRepositoryWithFakeImplementationsTest.java
@@ -0,0 +1,291 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one
+ * or more contributor license agreements.  See the NOTICE file
+ * distributed with this work for additional information
+ * regarding copyright ownership.  The ASF licenses this file
+ * to you under the Apache License, Version 2.0 (the
+ * "License"); you may not use this file except in compliance
+ * with the License.  You may obtain a copy of the License at
+ *
+ *   http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing,
+ * software distributed under the License is distributed on an
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+ * KIND, either express or implied.  See the License for the
+ * specific language governing permissions and limitations
+ * under the License.
+ */
+
+package org.apache.james.mailrepository.cassandra;
+
+import com.datastax.driver.core.ResultSet;
+import com.datastax.driver.core.Session;
+import com.google.common.collect.ImmutableList;
+import org.apache.james.backends.cassandra.CassandraCluster;
+import org.apache.james.backends.cassandra.CassandraClusterExtension;
+import org.apache.james.backends.cassandra.components.CassandraModule;
+import org.apache.james.backends.cassandra.init.CassandraTypesProvider;
+import org.apache.james.backends.cassandra.utils.CassandraUtils;
+import org.apache.james.blob.api.BlobId;
+import org.apache.james.blob.api.HashBlobId;
+import org.apache.james.blob.api.Store;
+import org.apache.james.blob.cassandra.BlobTable;
+import org.apache.james.blob.cassandra.CassandraBlobModule;
+import org.apache.james.blob.cassandra.CassandraBlobsDAO;
+import org.apache.james.blob.mail.MimeMessagePartsId;
+import org.apache.james.blob.mail.MimeMessageStore;
+import org.apache.james.core.MailAddress;
+import org.apache.james.core.builder.MimeMessageBuilder;
+import org.apache.james.mailrepository.api.MailKey;
+import org.apache.james.mailrepository.api.MailRepositoryUrl;
+import org.apache.james.server.core.MailImpl;
+import org.apache.mailet.Mail;
+import org.junit.jupiter.api.BeforeEach;
+import org.junit.jupiter.api.Nested;
+import org.junit.jupiter.api.Test;
+import org.junit.jupiter.api.extension.ExtendWith;
+import org.junit.jupiter.api.extension.ExtensionContext;
+
+import javax.mail.MessagingException;
+import javax.mail.internet.MimeMessage;
+import java.util.List;
+import java.util.concurrent.CompletableFuture;
+
+import static com.datastax.driver.core.querybuilder.QueryBuilder.select;
+import static org.assertj.core.api.Assertions.assertThat;
+import static org.assertj.core.api.Assertions.assertThatThrownBy;
+
+@ExtendWith(CassandraMailRepositoryWithFakeImplementationsTest.MailRepositoryCassandraClusterExtension.class)
+class CassandraMailRepositoryWithFakeImplementationsTest {
+static final MailRepositoryUrl URL = MailRepositoryUrl.from("proto://url");
+static final HashBlobId.Factory BLOB_ID_FACTORY = new HashBlobId.Factory();
+
+static class MailRepositoryCassandraClusterExtension extends 
CassandraClusterExtension {
+public MailRepositoryCassandraClusterExtension() {
+   

[04/14] james-project git commit: JAMES-2556 Fix reprocessing when several repositories with same path

2018-10-10 Thread btellier
JAMES-2556 Fix reprocessing when several repositories with same path

We need to aggregate repositories to check for mail existence before 
reprocessing


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/9d52d569
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/9d52d569
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/9d52d569

Branch: refs/heads/master
Commit: 9d52d569f749a28163719ea0ee709a55153ebbc4
Parents: f15655b
Author: Benoit Tellier 
Authored: Wed Oct 10 10:43:41 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:19 2018 +0700

--
 .../james/webadmin/service/ReprocessingService.java  | 15 +--
 .../webadmin/routes/MailRepositoriesRoutesTest.java  |  2 --
 2 files changed, 9 insertions(+), 8 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/9d52d569/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
index c316b95..6ccb045 100644
--- 
a/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
@@ -25,6 +25,7 @@ import java.util.function.Consumer;
 import javax.inject.Inject;
 import javax.mail.MessagingException;
 
+import org.apache.commons.lang3.tuple.Pair;
 import org.apache.james.mailrepository.api.MailKey;
 import org.apache.james.mailrepository.api.MailRepository;
 import org.apache.james.mailrepository.api.MailRepositoryPath;
@@ -90,13 +91,15 @@ public class ReprocessingService {
 public void reprocess(MailRepositoryPath path, MailKey key, 
Optional targetProcessor, String targetQueue) throws 
MailRepositoryStore.MailRepositoryStoreException, MessagingException {
 Reprocessor reprocessor = new Reprocessor(getMailQueue(targetQueue), 
targetProcessor);
 
-mailRepositoryStoreService
+Pair mailPair = mailRepositoryStoreService
 .getRepositories(path)
-.forEach(Throwing.consumer((MailRepository repository) ->
-reprocessor.reprocess(repository,
-Optional.ofNullable(repository.retrieve(key))
-.orElseThrow(() -> new MissingKeyException(key
-.sneakyThrow());
+.map(Throwing.function(repository -> Pair.of(repository, 
Optional.ofNullable(repository.retrieve(key)
+.filter(pair -> pair.getRight().isPresent())
+.map(pair -> Pair.of(pair.getLeft(), pair.getRight().get()))
+.findFirst()
+.orElseThrow(() -> new MissingKeyException(key));
+
+reprocessor.reprocess(mailPair.getKey(), mailPair.getValue());
 }
 
 private MailQueue getMailQueue(String targetQueue) {

http://git-wip-us.apache.org/repos/asf/james-project/blob/9d52d569/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
index ea78e4c..7d98451 100644
--- 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
@@ -83,7 +83,6 @@ import org.apache.mailet.base.test.FakeMail;
 import org.eclipse.jetty.http.HttpStatus;
 import org.junit.After;
 import org.junit.Before;
-import org.junit.Ignore;
 import org.junit.Test;
 
 import com.google.common.collect.Sets;
@@ -1506,7 +1505,6 @@ public class MailRepositoriesRoutesTest {
 .body("completedDate", is(notNullValue()));
 }
 
-@Ignore
 @Test
 public void 
reprocessingOneTaskShouldNotFailWhenSeveralRepositoryWithSamePath() throws 
Exception {
 MailRepository mailRepository = 
mailRepositoryStore.create(URL_MY_REPO);


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additi

[jira] [Commented] (JAMES-2554) Coherence issues between mail repository list and mails

2018-10-10 Thread Tellier Benoit (JIRA)


[ 
https://issues.apache.org/jira/browse/JAMES-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644660#comment-16644660
 ] 

Tellier Benoit commented on JAMES-2554:
---

https://github.com/linagora/james-project/pull/1786 contributed some 
consistency related tests for CassandraRepositories

> Coherence issues between mail repository list and mails
> ---
>
> Key: JAMES-2554
> URL: https://issues.apache.org/jira/browse/JAMES-2554
> Project: James Server
>  Issue Type: New Feature
>  Components: cassandra, MailStore & MailRepository
>Reporter: Antoine Duprat
>Priority: Major
>
> When getting the list of mails from a repository, some of them does not exist.
> I don't know what happened with them, but GETting them fails with not found.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Closed] (JAMES-2556) Impossible to reprocess a mailqueue when a key is not found

2018-10-10 Thread Tellier Benoit (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tellier Benoit closed JAMES-2556.
-

> Impossible to reprocess a mailqueue when a key is not found
> ---
>
> Key: JAMES-2556
> URL: https://issues.apache.org/jira/browse/JAMES-2556
> Project: James Server
>  Issue Type: Bug
>  Components: MailStore & MailRepository, webadmin
>Affects Versions: master, 3.1.0
>Reporter: Tellier Benoit
>Priority: Minor
> Fix For: 3.2.0
>
>
> We miss a null check on mailRepository.retrieve(key)
> This can happen for instance in the face of concurrent deletes
> {code:java}
> 11:56:43.579 [ERROR] o.a.j.t.MemoryTaskManager - Error while running task
> java.lang.NullPointerException: null
>   at 
> org.apache.james.webadmin.service.ReprocessingService.reprocess(ReprocessingService.java:72)
>   at 
> org.apache.james.webadmin.service.ReprocessingService.lambda$null$0(ReprocessingService.java:59)
>   at 
> com.github.fge.lambdas.consumers.ConsumerChainer.doAccept(ConsumerChainer.java:20)
>   at 
> com.github.fge.lambdas.consumers.ThrowingConsumer.accept(ThrowingConsumer.java:22)
>   at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
>   at 
> java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
>   at java.util.Iterator.forEachRemaining(Iterator.java:116)
>   at 
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
>   at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
>   at 
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
>   at 
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
>   at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
>   at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>   at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
>   at 
> org.apache.james.webadmin.service.ReprocessingService.lambda$reprocessAll$1(ReprocessingService.java:59)
>   at 
> com.github.fge.lambdas.consumers.ConsumerChainer.lambda$sneakyThrow$9(ConsumerChainer.java:73)
>   at 
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
>   at 
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
>   at 
> org.apache.james.webadmin.service.ReprocessingService.reprocessAll(ReprocessingService.java:56)
>   at 
> org.apache.james.webadmin.service.ReprocessingAllMailsTask.run(ReprocessingAllMailsTask.java:99)
>   at 
> org.apache.james.task.MemoryTaskManager.run(MemoryTaskManager.java:84)
>   at 
> org.apache.james.task.MemoryTaskManager.lambda$runWithMdc$2(MemoryTaskManager.java:77)
>   at org.apache.james.util.MDCBuilder.lambda$withMdc$0(MDCBuilder.java:60)
>   at org.apache.james.util.MDCBuilder.withMdc(MDCBuilder.java:48)
>   at org.apache.james.util.MDCBuilder.withMdc(MDCBuilder.java:59)
>   at 
> org.apache.james.task.MemoryTaskManager.runWithMdc(MemoryTaskManager.java:72)
>   at 
> org.apache.james.task.MemoryTaskManager.lambda$submit$1(MemoryTaskManager.java:67)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Commented] (JAMES-2545) Implement a HealthCheck for RabbitMQ component

2018-10-10 Thread Tellier Benoit (JIRA)


[ 
https://issues.apache.org/jira/browse/JAMES-2545?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644658#comment-16644658
 ] 

Tellier Benoit commented on JAMES-2545:
---

https://github.com/linagora/james-project/pull/1783 stabilises some RabbitMQ 
tests

> Implement a HealthCheck for RabbitMQ component
> --
>
> Key: JAMES-2545
> URL: https://issues.apache.org/jira/browse/JAMES-2545
> Project: James Server
>  Issue Type: Improvement
>  Components: Queue
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.2.0
>
>
> This health check will simply try to open a connection to rabbitMQ



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Resolved] (JAMES-2556) Impossible to reprocess a mailqueue when a key is not found

2018-10-10 Thread Tellier Benoit (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tellier Benoit resolved JAMES-2556.
---
Resolution: Fixed

https://github.com/linagora/james-project/pull/1791 solved this

> Impossible to reprocess a mailqueue when a key is not found
> ---
>
> Key: JAMES-2556
> URL: https://issues.apache.org/jira/browse/JAMES-2556
> Project: James Server
>  Issue Type: Bug
>  Components: MailStore & MailRepository, webadmin
>Affects Versions: master, 3.1.0
>Reporter: Tellier Benoit
>Priority: Minor
> Fix For: 3.2.0
>
>
> We miss a null check on mailRepository.retrieve(key)
> This can happen for instance in the face of concurrent deletes
> {code:java}
> 11:56:43.579 [ERROR] o.a.j.t.MemoryTaskManager - Error while running task
> java.lang.NullPointerException: null
>   at 
> org.apache.james.webadmin.service.ReprocessingService.reprocess(ReprocessingService.java:72)
>   at 
> org.apache.james.webadmin.service.ReprocessingService.lambda$null$0(ReprocessingService.java:59)
>   at 
> com.github.fge.lambdas.consumers.ConsumerChainer.doAccept(ConsumerChainer.java:20)
>   at 
> com.github.fge.lambdas.consumers.ThrowingConsumer.accept(ThrowingConsumer.java:22)
>   at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:184)
>   at 
> java.util.stream.ReferencePipeline$11$1.accept(ReferencePipeline.java:373)
>   at java.util.Iterator.forEachRemaining(Iterator.java:116)
>   at 
> java.util.Spliterators$IteratorSpliterator.forEachRemaining(Spliterators.java:1801)
>   at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
>   at 
> java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
>   at 
> java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
>   at 
> java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
>   at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
>   at 
> java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
>   at 
> org.apache.james.webadmin.service.ReprocessingService.lambda$reprocessAll$1(ReprocessingService.java:59)
>   at 
> com.github.fge.lambdas.consumers.ConsumerChainer.lambda$sneakyThrow$9(ConsumerChainer.java:73)
>   at 
> java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1382)
>   at 
> java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:580)
>   at 
> org.apache.james.webadmin.service.ReprocessingService.reprocessAll(ReprocessingService.java:56)
>   at 
> org.apache.james.webadmin.service.ReprocessingAllMailsTask.run(ReprocessingAllMailsTask.java:99)
>   at 
> org.apache.james.task.MemoryTaskManager.run(MemoryTaskManager.java:84)
>   at 
> org.apache.james.task.MemoryTaskManager.lambda$runWithMdc$2(MemoryTaskManager.java:77)
>   at org.apache.james.util.MDCBuilder.lambda$withMdc$0(MDCBuilder.java:60)
>   at org.apache.james.util.MDCBuilder.withMdc(MDCBuilder.java:48)
>   at org.apache.james.util.MDCBuilder.withMdc(MDCBuilder.java:59)
>   at 
> org.apache.james.task.MemoryTaskManager.runWithMdc(MemoryTaskManager.java:72)
>   at 
> org.apache.james.task.MemoryTaskManager.lambda$submit$1(MemoryTaskManager.java:67)
>   at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
>   at java.util.concurrent.FutureTask.run(FutureTask.java:266)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {code}



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Closed] (JAMES-2557) NULL_SENDER throws an error in Dlp

2018-10-10 Thread Tellier Benoit (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tellier Benoit closed JAMES-2557.
-

> NULL_SENDER throws an error in Dlp
> --
>
> Key: JAMES-2557
> URL: https://issues.apache.org/jira/browse/JAMES-2557
> Project: James Server
>  Issue Type: Bug
>  Components: Mailet Contributions
>Affects Versions: master, 3.1.0
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.2.0
>
>
> Watching Kibana logs I got this recurring error:
> {code:java}
>   Exception calling dlp.Dlp: NULL sender '<>' do not have domain part
> java.lang.IllegalStateException: NULL sender '<>' do not have domain part
>   at org.apache.james.core.MailAddress$1.getDomain(MailAddress.java:87)
>   at org.apache.james.transport.matchers.dlp.Dlp.matchingRule(Dlp.java:76)
>   at 
> org.apache.james.transport.matchers.dlp.Dlp.lambda$findFirstMatchingRule$0(Dlp.java:72)
>   at java.util.Optional.flatMap(Optional.java:241)
>   at 
> org.apache.james.transport.matchers.dlp.Dlp.findFirstMatchingRule(Dlp.java:72)
>   at org.apache.james.transport.matchers.dlp.Dlp.match(Dlp.java:55)
>   at 
> org.apache.james.mailetcontainer.impl.camel.MatcherSplitter.split(MatcherSplitter.java:109)
>   at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.camel.component.bean.MethodInfo.invoke(MethodInfo.java:481)
>   at 
> org.apache.camel.component.bean.MethodInfo$1.doProceed(MethodInfo.java:300)
>   at 
> org.apache.camel.component.bean.MethodInfo$1.proceed(MethodInfo.java:273)
>   at 
> org.apache.camel.component.bean.AbstractBeanProcessor.process(AbstractBeanProcessor.java:187)
>   at 
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
>   at 
> org.apache.camel.component.bean.AbstractBeanProcessor.process(AbstractBeanProcessor.java:70)
>   at 
> org.apache.camel.language.bean.BeanExpression.invokeBean(BeanExpression.java:200)
>   at 
> org.apache.camel.language.bean.BeanExpression.evaluate(BeanExpression.java:124)
>   at 
> org.apache.camel.language.bean.BeanExpression.evaluate(BeanExpression.java:135)
>   at 
> org.apache.camel.processor.Splitter.createProcessorExchangePairs(Splitter.java:127)
>   at 
> org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:241)
>   at org.apache.camel.processor.Splitter.process(Splitter.java:122)
>   at 
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
>   at 
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
>   at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
>   at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
>   at 
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
>   at 
> org.apache.camel.component.direct.DirectBlockingProducer.process(DirectBlockingProducer.java:53)
>   at 
> org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:186)
>   at 
> org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:86)
>   at 
> org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:541)
>   at 
> org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:506)
>   at 
> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:369)
>   at 
> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:506)
>   at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:229)
>   at 
> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:144)
>   at 
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:161)
>   at 
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:168)
>   at 
> org.apache.james.mailetcontainer.impl.camel.CamelMailetProcessor.service(CamelMailetProcessor.java:68)
>   at 
> org.apache.james.mailetcontainer.lib.AbstractStateCompositeProcessor.service(AbstractStateCompositeProcessor.java:84)
>   at 
> org.apache.james.mailetcontainer.impl.JamesMailSpooler.lambda$run$0(JamesMailSpooler.java:163)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(Thread.java:748)
> {code}
> It cames from the Dlp part (Dlp.java):
> {code:java}
> pri

[jira] [Resolved] (JAMES-2557) NULL_SENDER throws an error in Dlp

2018-10-10 Thread Tellier Benoit (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tellier Benoit resolved JAMES-2557.
---
Resolution: Fixed

https://github.com/linagora/james-project/pull/1789 fixed this

> NULL_SENDER throws an error in Dlp
> --
>
> Key: JAMES-2557
> URL: https://issues.apache.org/jira/browse/JAMES-2557
> Project: James Server
>  Issue Type: Bug
>  Components: Mailet Contributions
>Affects Versions: master, 3.1.0
>Reporter: Tellier Benoit
>Priority: Major
> Fix For: 3.2.0
>
>
> Watching Kibana logs I got this recurring error:
> {code:java}
>   Exception calling dlp.Dlp: NULL sender '<>' do not have domain part
> java.lang.IllegalStateException: NULL sender '<>' do not have domain part
>   at org.apache.james.core.MailAddress$1.getDomain(MailAddress.java:87)
>   at org.apache.james.transport.matchers.dlp.Dlp.matchingRule(Dlp.java:76)
>   at 
> org.apache.james.transport.matchers.dlp.Dlp.lambda$findFirstMatchingRule$0(Dlp.java:72)
>   at java.util.Optional.flatMap(Optional.java:241)
>   at 
> org.apache.james.transport.matchers.dlp.Dlp.findFirstMatchingRule(Dlp.java:72)
>   at org.apache.james.transport.matchers.dlp.Dlp.match(Dlp.java:55)
>   at 
> org.apache.james.mailetcontainer.impl.camel.MatcherSplitter.split(MatcherSplitter.java:109)
>   at sun.reflect.GeneratedMethodAccessor70.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.apache.camel.component.bean.MethodInfo.invoke(MethodInfo.java:481)
>   at 
> org.apache.camel.component.bean.MethodInfo$1.doProceed(MethodInfo.java:300)
>   at 
> org.apache.camel.component.bean.MethodInfo$1.proceed(MethodInfo.java:273)
>   at 
> org.apache.camel.component.bean.AbstractBeanProcessor.process(AbstractBeanProcessor.java:187)
>   at 
> org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
>   at 
> org.apache.camel.component.bean.AbstractBeanProcessor.process(AbstractBeanProcessor.java:70)
>   at 
> org.apache.camel.language.bean.BeanExpression.invokeBean(BeanExpression.java:200)
>   at 
> org.apache.camel.language.bean.BeanExpression.evaluate(BeanExpression.java:124)
>   at 
> org.apache.camel.language.bean.BeanExpression.evaluate(BeanExpression.java:135)
>   at 
> org.apache.camel.processor.Splitter.createProcessorExchangePairs(Splitter.java:127)
>   at 
> org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:241)
>   at org.apache.camel.processor.Splitter.process(Splitter.java:122)
>   at 
> org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:548)
>   at 
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
>   at org.apache.camel.processor.Pipeline.process(Pipeline.java:138)
>   at org.apache.camel.processor.Pipeline.process(Pipeline.java:101)
>   at 
> org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:201)
>   at 
> org.apache.camel.component.direct.DirectBlockingProducer.process(DirectBlockingProducer.java:53)
>   at 
> org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:186)
>   at 
> org.apache.camel.processor.SharedCamelInternalProcessor.process(SharedCamelInternalProcessor.java:86)
>   at 
> org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:541)
>   at 
> org.apache.camel.impl.ProducerCache$1.doInProducer(ProducerCache.java:506)
>   at 
> org.apache.camel.impl.ProducerCache.doInProducer(ProducerCache.java:369)
>   at 
> org.apache.camel.impl.ProducerCache.sendExchange(ProducerCache.java:506)
>   at org.apache.camel.impl.ProducerCache.send(ProducerCache.java:229)
>   at 
> org.apache.camel.impl.DefaultProducerTemplate.send(DefaultProducerTemplate.java:144)
>   at 
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:161)
>   at 
> org.apache.camel.impl.DefaultProducerTemplate.sendBody(DefaultProducerTemplate.java:168)
>   at 
> org.apache.james.mailetcontainer.impl.camel.CamelMailetProcessor.service(CamelMailetProcessor.java:68)
>   at 
> org.apache.james.mailetcontainer.lib.AbstractStateCompositeProcessor.service(AbstractStateCompositeProcessor.java:84)
>   at 
> org.apache.james.mailetcontainer.impl.JamesMailSpooler.lambda$run$0(JamesMailSpooler.java:163)
>   at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
>   at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
>   at java.lang.Thread.run(

[jira] [Commented] (JAMES-2544) Manageable RabbitMQ: browse

2018-10-10 Thread Tellier Benoit (JIRA)


[ 
https://issues.apache.org/jira/browse/JAMES-2544?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16644649#comment-16644649
 ] 

Tellier Benoit commented on JAMES-2544:
---

https://github.com/linagora/james-project/pull/1781 Added a Retrier on top of 
management API

> Manageable RabbitMQ: browse
> ---
>
> Key: JAMES-2544
> URL: https://issues.apache.org/jira/browse/JAMES-2544
> Project: James Server
>  Issue Type: Improvement
>  Components: Queue, rabbitmq
>Reporter: Trần Tiến Đức
>Priority: Major
>
> retrieve last message timestamp from Cassandra
> read messages from cassandra with slide queries
> filter messages the same way as Dequeue (noop for now)



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[14/14] james-project git commit: JAMES-2554 Use logback in tests

2018-10-10 Thread btellier
JAMES-2554 Use logback in tests


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/34f10299
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/34f10299
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/34f10299

Branch: refs/heads/master
Commit: 34f10299003853fc30a87812ba693fcc0a3c6234
Parents: e536e62
Author: Antoine Duprat 
Authored: Tue Oct 9 09:50:47 2018 +0200
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:47:38 2018 +0700

--
 .../mailrepository-cassandra/pom.xml|  5 +++
 .../src/test/resources/logback-test.xml | 43 
 2 files changed, 48 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/34f10299/server/mailrepository/mailrepository-cassandra/pom.xml
--
diff --git a/server/mailrepository/mailrepository-cassandra/pom.xml 
b/server/mailrepository/mailrepository-cassandra/pom.xml
index 479a90f..506fe9e 100644
--- a/server/mailrepository/mailrepository-cassandra/pom.xml
+++ b/server/mailrepository/mailrepository-cassandra/pom.xml
@@ -78,6 +78,11 @@
 test
 
 
+ch.qos.logback
+logback-classic
+test
+
+
 org.assertj
 assertj-core
 test

http://git-wip-us.apache.org/repos/asf/james-project/blob/34f10299/server/mailrepository/mailrepository-cassandra/src/test/resources/logback-test.xml
--
diff --git 
a/server/mailrepository/mailrepository-cassandra/src/test/resources/logback-test.xml
 
b/server/mailrepository/mailrepository-cassandra/src/test/resources/logback-test.xml
new file mode 100644
index 000..cf76022
--- /dev/null
+++ 
b/server/mailrepository/mailrepository-cassandra/src/test/resources/logback-test.xml
@@ -0,0 +1,43 @@
+
+
+
+
+
+
+true
+
+
+
+
+%d{HH:mm:ss.SSS} [%-5level] %logger{15} - 
%msg%n%rEx
+false
+
+
+
+
+
+
+
+
+
+
+
+
+


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[11/14] james-project git commit: JAMES-2334 Disable nodeKillingWhenConsuming

2018-10-10 Thread btellier
JAMES-2334 Disable nodeKillingWhenConsuming


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/86961f3f
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/86961f3f
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/86961f3f

Branch: refs/heads/master
Commit: 86961f3fc818ab3fdee2ddc76dcd16ff06e61dff
Parents: de7d0e0
Author: Benoit Tellier 
Authored: Mon Oct 8 10:32:50 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:36 2018 +0700

--
 .../org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java| 3 +++
 1 file changed, 3 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/86961f3f/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
--
diff --git 
a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
 
b/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
index 081f0d1..b72b45b 100644
--- 
a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
+++ 
b/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
@@ -245,6 +245,9 @@ class RabbitMQClusterTest {
 }
 }
 
+@Disabled("JAMES-2334 For some reason, we are unable to recover 
topology when reconnecting" +
+"See https://github.com/rabbitmq/rabbitmq-server/issues/959"; +
+"This test have roughly 4% chance to fail")
 @Test
 void nodeKillingWhenConsuming(DockerRabbitMQCluster cluster) throws 
Exception {
 resilientChannel.exchangeDeclare(EXCHANGE_NAME, DIRECT, DURABLE);


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[06/14] james-project git commit: JAMES-2556 Add tests for multi-path repository reprocessing

2018-10-10 Thread btellier
JAMES-2556 Add tests for multi-path repository reprocessing


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/6cf97c9d
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/6cf97c9d
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/6cf97c9d

Branch: refs/heads/master
Commit: 6cf97c9d5bbe32c77e4923767f9cc547a6c9e784
Parents: 9d1d697
Author: Benoit Tellier 
Authored: Wed Oct 10 10:08:38 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:19 2018 +0700

--
 .../routes/MailRepositoriesRoutesTest.java  | 120 +++
 .../src/test/resources/mailrepositorystore.xml  |   5 +
 2 files changed, 125 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/6cf97c9d/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
index 4ea84f7..7294348 100644
--- 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
@@ -83,6 +83,7 @@ import org.apache.mailet.base.test.FakeMail;
 import org.eclipse.jetty.http.HttpStatus;
 import org.junit.After;
 import org.junit.Before;
+import org.junit.Ignore;
 import org.junit.Test;
 
 import com.google.common.collect.Sets;
@@ -95,6 +96,7 @@ import io.restassured.parsing.Parser;
 public class MailRepositoriesRoutesTest {
 
 private static final MailRepositoryUrl URL_MY_REPO = 
MailRepositoryUrl.from("memory://myRepo");
+private static final MailRepositoryUrl URL_MY_REPO_OTHER = 
MailRepositoryUrl.from("other://myRepo");
 private static final MailRepositoryPath PATH_MY_REPO = 
MailRepositoryPath.from("myRepo");
 private static final String PATH_ESCAPED_MY_REPO = "myRepo";
 private static final String MY_REPO_MAILS = "myRepo/mails";
@@ -1045,6 +1047,36 @@ public class MailRepositoriesRoutesTest {
 }
 
 @Test
+public void 
reprocessingAllTaskShouldNotFailWhenSeveralRepositoriesWithSamePath() throws 
Exception {
+MailRepository mailRepository = 
mailRepositoryStore.create(URL_MY_REPO);
+mailRepositoryStore.create(URL_MY_REPO_OTHER);
+String name1 = "name1";
+String name2 = "name2";
+mailRepository.store(FakeMail.builder()
+.name(name1)
+.build());
+mailRepository.store(FakeMail.builder()
+.name(name2)
+.build());
+
+String transport = "transport";
+String taskId = with()
+.param("action", "reprocess")
+.param("queue", CUSTOM_QUEUE)
+.param("processor", transport)
+.patch(PATH_ESCAPED_MY_REPO + "/mails")
+.jsonPath()
+.get("taskId");
+
+given()
+.basePath(TasksRoutes.BASE)
+.when()
+.get(taskId + "/await")
+.then()
+.body("status", is("completed"));
+}
+
+@Test
 public void reprocessingAllTaskShouldClearMailRepository() throws 
Exception {
 MailRepository mailRepository = 
mailRepositoryStore.create(URL_MY_REPO);
 String name1 = "name1";
@@ -1073,6 +1105,36 @@ public class MailRepositoriesRoutesTest {
 }
 
 @Test
+public void 
reprocessingAllTaskShouldClearBothMailRepositoriesWhenSamePath() throws 
Exception {
+MailRepository mailRepository1 = 
mailRepositoryStore.create(URL_MY_REPO);
+MailRepository mailRepository2 = 
mailRepositoryStore.create(URL_MY_REPO_OTHER);
+String name1 = "name1";
+String name2 = "name2";
+mailRepository1.store(FakeMail.builder()
+.name(name1)
+.build());
+mailRepository2.store(FakeMail.builder()
+.name(name2)
+.build());
+
+String transport = "transport";
+String taskId = with()
+.param("action", "reprocess")
+.param("queue", CUSTOM_QUEUE)
+.param("processor", transport)
+.patch(PATH_ESCAPED_MY_REPO + "/mails")
+.jsonPath()
+.get("taskId");
+
+with()
+.basePath(TasksRoutes.BASE)
+.get(taskId + "/await");
+
+assertThat(mailRepository1.list()).isEmpty();
+assertThat(mailRepository2.list()).isEmpty();
+}
+
+   

[10/14] james-project git commit: JAMES-2551 Fasting up RabbitMQ cluster startup

2018-10-10 Thread btellier
JAMES-2551 Fasting up RabbitMQ cluster startup

 - No need to start App and await node 1, which is never stopper
 - Doing a single big step run in parallel is more efficient than several 
little parallel steps (-30 s)


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/33bfd0d7
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/33bfd0d7
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/33bfd0d7

Branch: refs/heads/master
Commit: 33bfd0d7aa5bb30c37decee8eee6b80218300540
Parents: 86961f3
Author: Benoit Tellier 
Authored: Mon Oct 8 13:23:59 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:36 2018 +0700

--
 .../backend/rabbitmq/DockerClusterRabbitMQExtension.java  | 10 --
 .../org/apache/james/backend/rabbitmq/DockerRabbitMQ.java |  2 ++
 2 files changed, 2 insertions(+), 10 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/33bfd0d7/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerClusterRabbitMQExtension.java
--
diff --git 
a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerClusterRabbitMQExtension.java
 
b/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerClusterRabbitMQExtension.java
index 811c768..aede7c6 100644
--- 
a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerClusterRabbitMQExtension.java
+++ 
b/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerClusterRabbitMQExtension.java
@@ -64,17 +64,7 @@ public class DockerClusterRabbitMQExtension implements 
BeforeEachCallback, After
 Throwing.runnable(() -> rabbitMQ2.join(rabbitMQ1)),
 Throwing.runnable(() -> rabbitMQ3.join(rabbitMQ1)));
 
-Runnables.runParallel(
-Throwing.runnable(rabbitMQ1::startApp),
-Throwing.runnable(rabbitMQ2::startApp),
-Throwing.runnable(rabbitMQ3::startApp));
-
 cluster = new DockerRabbitMQCluster(rabbitMQ1, rabbitMQ2, rabbitMQ3);
-
-Runnables.runParallel(
-rabbitMQ1::waitForReadyness,
-rabbitMQ2::waitForReadyness,
-rabbitMQ3::waitForReadyness);
 }
 
 @Override

http://git-wip-us.apache.org/repos/asf/james-project/blob/33bfd0d7/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerRabbitMQ.java
--
diff --git 
a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerRabbitMQ.java
 
b/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerRabbitMQ.java
index a546939..29add81 100644
--- 
a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerRabbitMQ.java
+++ 
b/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/DockerRabbitMQ.java
@@ -140,6 +140,8 @@ public class DockerRabbitMQ {
 public void join(DockerRabbitMQ rabbitMQ) throws Exception {
 stopApp();
 joinCluster(rabbitMQ);
+startApp();
+waitForReadyness();
 }
 
 public void stopApp() throws java.io.IOException, InterruptedException {


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[08/14] james-project git commit: JAMES-2556 Add tests for multi-path repository management

2018-10-10 Thread btellier
JAMES-2556 Add tests for multi-path repository management

This proves limit and offset to be failing


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/ea70abcd
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/ea70abcd
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/ea70abcd

Branch: refs/heads/master
Commit: ea70abcd16910b63822061f0e5858f7d8be1cbd3
Parents: 6cf97c9
Author: Benoit Tellier 
Authored: Wed Oct 10 10:16:52 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:19 2018 +0700

--
 .../routes/MailRepositoriesRoutesTest.java  | 108 +++
 1 file changed, 108 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/ea70abcd/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
index 7294348..f17b20c 100644
--- 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
@@ -212,6 +212,20 @@ public class MailRepositoriesRoutesTest {
 }
 
 @Test
+public void getMailRepositoriesShouldDeduplicateAccordingToPath() throws 
Exception {
+mailRepositoryStore.create(URL_MY_REPO);
+mailRepositoryStore.create(URL_MY_REPO_OTHER);
+
+when()
+.get()
+.then()
+.statusCode(HttpStatus.OK_200)
+.body("", hasSize(1))
+.body("[0].repository", is(PATH_MY_REPO.asString()))
+.body("[0].path", is(PATH_ESCAPED_MY_REPO));
+}
+
+@Test
 public void getMailRepositoriesShouldReturnTwoRepositoriesWhenTwo() throws 
Exception {
 mailRepositoryStore.create(URL_MY_REPO);
 
mailRepositoryStore.create(MailRepositoryUrl.from("memory://mySecondRepo"));
@@ -271,6 +285,26 @@ public class MailRepositoriesRoutesTest {
 }
 
 @Test
+public void listingKeysShouldMergeRepositoryContentWhenSamePath() throws 
Exception {
+MailRepository mailRepository1 = 
mailRepositoryStore.create(URL_MY_REPO);
+MailRepository mailRepository2 = 
mailRepositoryStore.create(URL_MY_REPO_OTHER);
+
+mailRepository1.store(FakeMail.builder()
+.name("name1")
+.build());
+mailRepository2.store(FakeMail.builder()
+.name("name2")
+.build());
+
+when()
+.get(MY_REPO_MAILS)
+.then()
+.statusCode(HttpStatus.OK_200)
+.body("", hasSize(2))
+.body("", containsInAnyOrder("name1", "name2"));
+}
+
+@Test
 public void listingKeysShouldApplyLimitAndOffset() throws Exception {
 MailRepository mailRepository = 
mailRepositoryStore.create(URL_MY_REPO);
 
@@ -295,6 +329,31 @@ public class MailRepositoriesRoutesTest {
 .body("", contains("name2"));
 }
 
+@Ignore("Limit and offset are applied on a per repository bases")
+@Test
+public void listingKeysShouldApplyLimitWhenSeveralRepositories() throws 
Exception {
+MailRepository mailRepository1 = 
mailRepositoryStore.create(URL_MY_REPO);
+MailRepository mailRepository2 = 
mailRepositoryStore.create(URL_MY_REPO_OTHER);
+
+mailRepository1.store(FakeMail.builder()
+.name("name1")
+.build());
+mailRepository1.store(FakeMail.builder()
+.name("name2")
+.build());
+mailRepository2.store(FakeMail.builder()
+.name("name3")
+.build());
+
+given()
+.param("limit", "1")
+.when()
+.get(MY_REPO_MAILS)
+.then()
+.statusCode(HttpStatus.OK_200)
+.body("", hasSize(1));
+}
+
 @Test
 public void listingKeysShouldReturnErrorOnInvalidOffset() {
 given()
@@ -344,6 +403,34 @@ public class MailRepositoriesRoutesTest {
 .body("", hasSize(0));
 }
 
+@Ignore("Offset is applied on a per mail repository basis")
+@Test
+public void offsetShouldBeAplliedOnTheMergedViewOfMailRepositories() 
throws Exception {
+MailRepository mailRepository1 = 
mailRepositoryStore.create(URL_MY_REPO);
+MailRepository mailRepository2 = 
mailRepositoryStore.create(URL_MY_REPO_OTHER);
+
+mailRepositor

[03/14] james-project git commit: JAMES-2556 Add a test proving reprocessing failure upon concurent mail deletion

2018-10-10 Thread btellier
JAMES-2556 Add a test proving reprocessing failure upon concurent mail deletion


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/9d1d697f
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/9d1d697f
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/9d1d697f

Branch: refs/heads/master
Commit: 9d1d697f6a14b1b1245a846a141787c835a09271
Parents: 0d8c68a
Author: Benoit Tellier 
Authored: Wed Oct 10 10:32:11 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:06 2018 +0700

--
 .../service/ReprocessingServiceTest.java| 196 +++
 1 file changed, 196 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/9d1d697f/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/service/ReprocessingServiceTest.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/service/ReprocessingServiceTest.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/service/ReprocessingServiceTest.java
new file mode 100644
index 000..c1d3688
--- /dev/null
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/service/ReprocessingServiceTest.java
@@ -0,0 +1,196 @@
+/
+ * Licensed to the Apache Software Foundation (ASF) under one   *
+ * or more contributor license agreements.  See the NOTICE file *
+ * distributed with this work for additional information*
+ * regarding copyright ownership.  The ASF licenses this file   *
+ * to you under the Apache License, Version 2.0 (the*
+ * "License"); you may not use this file except in compliance   *
+ * with the License.  You may obtain a copy of the License at   *
+ *  *
+ *   http://www.apache.org/licenses/LICENSE-2.0 *
+ *  *
+ * Unless required by applicable law or agreed to in writing,   *
+ * software distributed under the License is distributed on an  *
+ * "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY   *
+ * KIND, either express or implied.  See the License for the*
+ * specific language governing permissions and limitations  *
+ * under the License.   *
+ /
+
+package org.apache.james.webadmin.service;
+
+import static org.assertj.core.api.Assertions.assertThat;
+
+import java.util.Optional;
+import java.util.concurrent.atomic.AtomicBoolean;
+import java.util.function.Consumer;
+
+import org.apache.james.filesystem.api.FileSystem;
+import org.apache.james.mailrepository.api.MailKey;
+import org.apache.james.mailrepository.api.MailRepository;
+import org.apache.james.mailrepository.api.MailRepositoryPath;
+import org.apache.james.mailrepository.api.MailRepositoryUrl;
+import org.apache.james.mailrepository.memory.MemoryMailRepositoryProvider;
+import org.apache.james.mailrepository.memory.MemoryMailRepositoryStore;
+import org.apache.james.mailrepository.memory.MemoryMailRepositoryUrlStore;
+import org.apache.james.queue.api.MailQueueFactory;
+import org.apache.james.queue.api.ManageableMailQueue;
+import org.apache.james.queue.api.RawMailQueueItemDecoratorFactory;
+import org.apache.james.queue.memory.MemoryMailQueueFactory;
+import org.apache.james.server.core.configuration.Configuration;
+import org.apache.james.server.core.configuration.FileConfigurationProvider;
+import org.apache.james.server.core.filesystem.FileSystemImpl;
+import org.apache.mailet.base.test.FakeMail;
+import org.junit.Before;
+import org.junit.Ignore;
+import org.junit.Test;
+
+import com.github.fge.lambdas.Throwing;
+import com.github.fge.lambdas.consumers.ConsumerChainer;
+import com.google.common.collect.ImmutableList;
+import com.google.common.collect.Sets;
+
+public class ReprocessingServiceTest {
+private static final String MEMORY_PROTOCOL = "memory";
+private static final MailRepositoryPath PATH = 
MailRepositoryPath.from("path");
+private static final String NAME_1 = "key-1";
+private static final String NAME_2 = "key-2";
+private static final String NAME_3 = "key-3";
+private static final MailKey KEY_1 = new MailKey(NAME_1);
+private static final MailKey KEY_2 = new MailKey(NAME_2);
+private static final MailKey KEY_3 = new MailKey(NAME_3);
+private static final String SPOOL = "spool";
+private static final Consumer NOOP_CONSUMER = key -> {};
+private static final Optional NO_TAR

[02/14] james-project git commit: JAMES-2557 DLP should ignore null sender

2018-10-10 Thread btellier
JAMES-2557 DLP should ignore null sender


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/0d8c68ab
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/0d8c68ab
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/0d8c68ab

Branch: refs/heads/master
Commit: 0d8c68ab7860f69eb1c376fc89ab04f0926e223c
Parents: 0135291
Author: Benoit Tellier 
Authored: Tue Oct 9 11:44:27 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:45:50 2018 +0700

--
 .../org/apache/james/transport/matchers/dlp/Dlp.java   |  4 ++--
 .../james/transport/matchers/dlp/DlpDomainRules.java   |  4 +++-
 .../apache/james/transport/matchers/dlp/DlpTest.java   | 13 +
 3 files changed, 18 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/0d8c68ab/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/Dlp.java
--
diff --git 
a/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/Dlp.java
 
b/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/Dlp.java
index 1f65082..a82dc06 100644
--- 
a/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/Dlp.java
+++ 
b/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/Dlp.java
@@ -23,7 +23,6 @@ import java.util.Collection;
 import java.util.Optional;
 
 import javax.inject.Inject;
-import javax.mail.MessagingException;
 
 import org.apache.james.core.MailAddress;
 import org.apache.james.dlp.api.DLPConfigurationItem;
@@ -51,7 +50,7 @@ public class Dlp extends GenericMatcher {
 }
 
 @Override
-public Collection match(Mail mail) throws MessagingException {
+public Collection match(Mail mail) {
 Optional firstMatchingRuleId = 
findFirstMatchingRule(mail);
 
 if (firstMatchingRuleId.isPresent()) {
@@ -69,6 +68,7 @@ public class Dlp extends GenericMatcher {
 private Optional findFirstMatchingRule(Mail mail) 
{
 return Optional
 .ofNullable(mail.getSender())
+.filter(sender -> !sender.isNullSender())
 .flatMap(sender -> matchingRule(sender, mail));
 }
 

http://git-wip-us.apache.org/repos/asf/james-project/blob/0d8c68ab/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/DlpDomainRules.java
--
diff --git 
a/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/DlpDomainRules.java
 
b/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/DlpDomainRules.java
index b954801..031f306 100644
--- 
a/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/DlpDomainRules.java
+++ 
b/server/mailet/mailets/src/main/java/org/apache/james/transport/matchers/dlp/DlpDomainRules.java
@@ -169,7 +169,9 @@ public class DlpDomainRules {
 }
 
 private Stream listEnvelopSender(Mail mail) {
-return 
StreamUtils.ofNullables(mail.getSender()).map(MailAddress::asString);
+return StreamUtils.ofNullables(mail.getSender())
+.filter(sender -> !sender.isNullSender())
+.map(MailAddress::asString);
 }
 
 private Stream listFromHeaders(Mail mail) throws 
MessagingException {

http://git-wip-us.apache.org/repos/asf/james-project/blob/0d8c68ab/server/mailet/mailets/src/test/java/org/apache/james/transport/matchers/dlp/DlpTest.java
--
diff --git 
a/server/mailet/mailets/src/test/java/org/apache/james/transport/matchers/dlp/DlpTest.java
 
b/server/mailet/mailets/src/test/java/org/apache/james/transport/matchers/dlp/DlpTest.java
index 12aa394..88266d6 100644
--- 
a/server/mailet/mailets/src/test/java/org/apache/james/transport/matchers/dlp/DlpTest.java
+++ 
b/server/mailet/mailets/src/test/java/org/apache/james/transport/matchers/dlp/DlpTest.java
@@ -34,6 +34,7 @@ import java.util.Optional;
 import java.util.regex.Pattern;
 
 import org.apache.james.core.Domain;
+import org.apache.james.core.MailAddress;
 import org.apache.james.core.builder.MimeMessageBuilder;
 import org.apache.james.dlp.api.DLPConfigurationItem.Id;
 import org.apache.mailet.base.test.FakeMail;
@@ -103,6 +104,18 @@ class DlpTest {
 }
 
 @Test
+void nullSenderShouldBeIgnored() throws Exception {
+Dlp dlp = new Dlp(
+asRulesLoaderFor(
+JAMES_APACHE_ORG_DOMAIN,
+DlpDomainRules.builder().recipientRule(Id.of("match all 
recipient"), Pattern.compile(".*")).build()));
+
+FakeMail mail = 
FakeMail.builde

[01/14] james-project git commit: JAMES-2544 Use Feign retry mechanism to retry calls on RabbitMQ management API

2018-10-10 Thread btellier
Repository: james-project
Updated Branches:
  refs/heads/master eec12864d -> 34f102990


JAMES-2544 Use Feign retry mechanism to retry calls on RabbitMQ management API


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/01352919
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/01352919
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/01352919

Branch: refs/heads/master
Commit: 01352919615cb49d70934e03759e885cb69dc7ad
Parents: eec1286
Author: Benoit Tellier 
Authored: Wed Oct 10 09:12:16 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 09:15:24 2018 +0700

--
 .../james/queue/rabbitmq/RabbitMQManagementApi.java| 13 +
 1 file changed, 13 insertions(+)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/01352919/server/queue/queue-rabbitmq/src/main/java/org/apache/james/queue/rabbitmq/RabbitMQManagementApi.java
--
diff --git 
a/server/queue/queue-rabbitmq/src/main/java/org/apache/james/queue/rabbitmq/RabbitMQManagementApi.java
 
b/server/queue/queue-rabbitmq/src/main/java/org/apache/james/queue/rabbitmq/RabbitMQManagementApi.java
index 8d39b24..cbeea3a 100644
--- 
a/server/queue/queue-rabbitmq/src/main/java/org/apache/james/queue/rabbitmq/RabbitMQManagementApi.java
+++ 
b/server/queue/queue-rabbitmq/src/main/java/org/apache/james/queue/rabbitmq/RabbitMQManagementApi.java
@@ -19,6 +19,7 @@
 
 package org.apache.james.queue.rabbitmq;
 
+import java.util.Date;
 import java.util.List;
 import java.util.stream.Stream;
 
@@ -30,13 +31,23 @@ import com.fasterxml.jackson.annotation.JsonProperty;
 import feign.Feign;
 import feign.Logger;
 import feign.RequestLine;
+import feign.RetryableException;
+import feign.Retryer;
 import feign.auth.BasicAuthRequestInterceptor;
+import feign.codec.ErrorDecoder;
 import feign.jackson.JacksonDecoder;
 import feign.jackson.JacksonEncoder;
 import feign.slf4j.Slf4jLogger;
 
 class RabbitMQManagementApi {
 
+private static final ErrorDecoder RETRY_500 = (methodKey, response) -> {
+if (response.status() == 500) {
+throw new RetryableException("Error encountered, scheduling 
retry", response.request().httpMethod(), new Date());
+}
+throw new RuntimeException("Non recoverable exception status: " + 
response.status());
+};
+
 public interface Api {
 
 class MessageQueue {
@@ -58,6 +69,8 @@ class RabbitMQManagementApi {
 .logLevel(Logger.Level.FULL)
 .encoder(new JacksonEncoder())
 .decoder(new JacksonDecoder())
+.retryer(new Retryer.Default())
+.errorDecoder(RETRY_500)
 .target(Api.class, configuration.getManagementUri().toString());
 
 }


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[07/14] james-project git commit: JAMES-2556 Remove unused MailRepositoryStoreService::getRepository

2018-10-10 Thread btellier
JAMES-2556 Remove unused MailRepositoryStoreService::getRepository


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/95b4ead2
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/95b4ead2
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/95b4ead2

Branch: refs/heads/master
Commit: 95b4ead22d4fd4d934f848c10579b258ab0d4899
Parents: d68a63e
Author: Benoit Tellier 
Authored: Wed Oct 10 10:22:48 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:19 2018 +0700

--
 .../james/webadmin/service/MailRepositoryStoreService.java  | 9 -
 1 file changed, 9 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/95b4ead2/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
index a626655..987b443 100644
--- 
a/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
@@ -105,15 +105,6 @@ public class MailRepositoryStoreService {
 return new 
ClearMailRepositoryTask(getRepositories(path).collect(Guavate.toImmutableList()),
 path);
 }
 
-public MailRepository getRepository(MailRepositoryUrl url) throws 
MailRepositoryStore.MailRepositoryStoreException {
-return mailRepositoryStore.get(url)
-.orElseThrow(() -> ErrorResponder.builder()
-.statusCode(HttpStatus.NOT_FOUND_404)
-.type(ErrorResponder.ErrorType.NOT_FOUND)
-.message(url.asString() + " does not exist")
-.haltError());
-}
-
 public Stream getRepositories(MailRepositoryPath path) 
throws MailRepositoryStore.MailRepositoryStoreException {
 Stream byPath = mailRepositoryStore.getByPath(path);
 List repositories = 
byPath.collect(Collectors.toList());


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[05/14] james-project git commit: JAMES-2556 Fix reprocessing upon concurent mail deletion

2018-10-10 Thread btellier
JAMES-2556 Fix reprocessing upon concurent mail deletion

Single key reprocessing should fail on missing key while full reprocessing 
should not.

Because the argument list keeps growing adding a 'Reprocessor' object helps 
readability.


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/f15655b2
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/f15655b2
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/f15655b2

Branch: refs/heads/master
Commit: f15655b20e3e0e3b35707773ba70ddded2563403
Parents: 95b4ead
Author: Benoit Tellier 
Authored: Wed Oct 10 10:41:29 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:19 2018 +0700

--
 .../webadmin/service/ReprocessingService.java   | 47 +++-
 .../service/ReprocessingServiceTest.java|  2 -
 2 files changed, 36 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/f15655b2/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
index f02bd3f..c316b95 100644
--- 
a/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/ReprocessingService.java
@@ -31,12 +31,38 @@ import 
org.apache.james.mailrepository.api.MailRepositoryPath;
 import org.apache.james.mailrepository.api.MailRepositoryStore;
 import org.apache.james.queue.api.MailQueue;
 import org.apache.james.queue.api.MailQueueFactory;
+import org.apache.james.util.OptionalUtils;
 import org.apache.james.util.streams.Iterators;
 import org.apache.mailet.Mail;
 
 import com.github.fge.lambdas.Throwing;
 
 public class ReprocessingService {
+public static class MissingKeyException extends RuntimeException {
+MissingKeyException(MailKey key) {
+super(key.asString() + " can not be found");
+}
+}
+
+static class Reprocessor {
+private final MailQueue mailQueue;
+private final Optional targetProcessor;
+
+Reprocessor(MailQueue mailQueue, Optional targetProcessor) {
+this.mailQueue = mailQueue;
+this.targetProcessor = targetProcessor;
+}
+
+private void reprocess(MailRepository repository, Mail mail) {
+try {
+targetProcessor.ifPresent(mail::setState);
+mailQueue.enQueue(mail);
+repository.remove(mail);
+} catch (Exception e) {
+throw new RuntimeException("Error encountered while 
reprocessing mail " + mail.getName(), e);
+}
+}
+}
 
 private final MailQueueFactory mailQueueFactory;
 private final MailRepositoryStoreService mailRepositoryStoreService;
@@ -49,29 +75,28 @@ public class ReprocessingService {
 }
 
 public void reprocessAll(MailRepositoryPath path, Optional 
targetProcessor, String targetQueue, Consumer keyListener) throws 
MailRepositoryStore.MailRepositoryStoreException, MessagingException {
-MailQueue mailQueue = getMailQueue(targetQueue);
+Reprocessor reprocessor = new Reprocessor(getMailQueue(targetQueue), 
targetProcessor);
 
 mailRepositoryStoreService
 .getRepositories(path)
 .forEach(Throwing.consumer((MailRepository repository) ->
 Iterators.toStream(repository.list())
 .peek(keyListener)
-.forEach(Throwing.consumer(key -> reprocess(repository, 
mailQueue, key, targetProcessor.sneakyThrow());
+.map(Throwing.function(key -> 
Optional.ofNullable(repository.retrieve(key
+.flatMap(OptionalUtils::toStream)
+.forEach(mail -> reprocessor.reprocess(repository, 
mail;
 }
 
 public void reprocess(MailRepositoryPath path, MailKey key, 
Optional targetProcessor, String targetQueue) throws 
MailRepositoryStore.MailRepositoryStoreException, MessagingException {
-MailQueue mailQueue = getMailQueue(targetQueue);
+Reprocessor reprocessor = new Reprocessor(getMailQueue(targetQueue), 
targetProcessor);
 
 mailRepositoryStoreService
 .getRepositories(path)
-.forEach(Throwing.consumer((MailRepository repository) -> 
reprocess(repository, mailQueue, key, targetProcessor)).sneak

[09/14] james-project git commit: JAMES-2556 Fix limit and offset behaviour in multi-repository context

2018-10-10 Thread btellier
JAMES-2556 Fix limit and offset behaviour in multi-repository context


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/d68a63ef
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/d68a63ef
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/d68a63ef

Branch: refs/heads/master
Commit: d68a63efda3734eb0b34bc2c7abde327762a02c5
Parents: ea70abc
Author: Benoit Tellier 
Authored: Wed Oct 10 10:22:00 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:19 2018 +0700

--
 .../service/MailRepositoryStoreService.java   | 18 +++---
 .../routes/MailRepositoriesRoutesTest.java|  2 --
 2 files changed, 7 insertions(+), 13 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/d68a63ef/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
index ff4ee76..a626655 100644
--- 
a/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/main/java/org/apache/james/webadmin/service/MailRepositoryStoreService.java
@@ -48,8 +48,8 @@ import org.apache.mailet.Mail;
 import org.eclipse.jetty.http.HttpStatus;
 
 import com.github.fge.lambdas.Throwing;
-import com.github.fge.lambdas.functions.ThrowingFunction;
 import com.github.steveash.guavate.Guavate;
+import com.google.common.collect.ImmutableList;
 
 public class MailRepositoryStoreService {
 private final MailRepositoryStore mailRepositoryStore;
@@ -70,17 +70,13 @@ public class MailRepositoryStoreService {
 }
 
 public Optional> listMails(MailRepositoryPath path, 
Offset offset, Limit limit) throws 
MailRepositoryStore.MailRepositoryStoreException, MessagingException {
-ThrowingFunction> list = repository 
-> list(repository, offset, limit);
-return Optional.of(getRepositories(path)
-.flatMap(Throwing.function(list).sneakyThrow())
-.collect(Guavate.toImmutableList()));
-}
+Optional> maybeMails = 
Optional.of(getRepositories(path)
+.flatMap(Throwing.function((MailRepository repository) -> 
Iterators.toStream(repository.list())).sneakyThrow())
+.map(MailKeyDTO::new)
+.skip(offset.getOffset()));
 
-private Stream list(MailRepository mailRepository, Offset 
offset, Limit limit) throws MessagingException {
-return limit.applyOnStream(
-Iterators.toStream(mailRepository.list())
-.skip(offset.getOffset()))
-.map(MailKeyDTO::new);
+return maybeMails.map(limit::applyOnStream)
+.map(stream -> stream.collect(ImmutableList.toImmutableList()));
 }
 
 public Optional size(MailRepositoryPath path) throws 
MailRepositoryStore.MailRepositoryStoreException {

http://git-wip-us.apache.org/repos/asf/james-project/blob/d68a63ef/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
--
diff --git 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
index f17b20c..ea78e4c 100644
--- 
a/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
+++ 
b/server/protocols/webadmin/webadmin-mailrepository/src/test/java/org/apache/james/webadmin/routes/MailRepositoriesRoutesTest.java
@@ -329,7 +329,6 @@ public class MailRepositoriesRoutesTest {
 .body("", contains("name2"));
 }
 
-@Ignore("Limit and offset are applied on a per repository bases")
 @Test
 public void listingKeysShouldApplyLimitWhenSeveralRepositories() throws 
Exception {
 MailRepository mailRepository1 = 
mailRepositoryStore.create(URL_MY_REPO);
@@ -403,7 +402,6 @@ public class MailRepositoriesRoutesTest {
 .body("", hasSize(0));
 }
 
-@Ignore("Offset is applied on a per mail repository basis")
 @Test
 public void offsetShouldBeAplliedOnTheMergedViewOfMailRepositories() 
throws Exception {
 MailRepository mailRep

[12/14] james-project git commit: JAMES-2334 Use resilient channel for queue declaration

2018-10-10 Thread btellier
JAMES-2334 Use resilient channel for queue declaration


Project: http://git-wip-us.apache.org/repos/asf/james-project/repo
Commit: http://git-wip-us.apache.org/repos/asf/james-project/commit/de7d0e0a
Tree: http://git-wip-us.apache.org/repos/asf/james-project/tree/de7d0e0a
Diff: http://git-wip-us.apache.org/repos/asf/james-project/diff/de7d0e0a

Branch: refs/heads/master
Commit: de7d0e0aaf711a6da6105788d2bbecacdbb2d967
Parents: 9d52d56
Author: Benoit Tellier 
Authored: Mon Oct 8 10:31:27 2018 +0700
Committer: Benoit Tellier 
Committed: Wed Oct 10 15:46:36 2018 +0700

--
 .../org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
--


http://git-wip-us.apache.org/repos/asf/james-project/blob/de7d0e0a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
--
diff --git 
a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
 
b/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
index 6486a84..081f0d1 100644
--- 
a/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
+++ 
b/backends-common/rabbitmq/src/test/java/org/apache/james/backend/rabbitmq/RabbitMQClusterTest.java
@@ -247,9 +247,9 @@ class RabbitMQClusterTest {
 
 @Test
 void nodeKillingWhenConsuming(DockerRabbitMQCluster cluster) throws 
Exception {
-node2Channel.exchangeDeclare(EXCHANGE_NAME, DIRECT, DURABLE);
-node2Channel.queueDeclare(QUEUE, DURABLE, !EXCLUSIVE, 
!AUTO_DELETE, ImmutableMap.of()).getQueue();
-node2Channel.queueBind(QUEUE, EXCHANGE_NAME, ROUTING_KEY);
+resilientChannel.exchangeDeclare(EXCHANGE_NAME, DIRECT, DURABLE);
+resilientChannel.queueDeclare(QUEUE, DURABLE, !EXCLUSIVE, 
!AUTO_DELETE, ImmutableMap.of()).getQueue();
+resilientChannel.queueBind(QUEUE, EXCHANGE_NAME, ROUTING_KEY);
 
 int nbMessages = 10;
 IntStream.range(0, nbMessages)


-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Updated] (JAMES-2558) Use SpamAssassin version 3.4.2 in our test suite

2018-10-10 Thread Tellier Benoit (JIRA)


 [ 
https://issues.apache.org/jira/browse/JAMES-2558?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Tellier Benoit updated JAMES-2558:
--
Priority: Minor  (was: Major)

> Use SpamAssassin version 3.4.2 in our test suite
> 
>
> Key: JAMES-2558
> URL: https://issues.apache.org/jira/browse/JAMES-2558
> Project: James Server
>  Issue Type: Improvement
>  Components: mailbox, Mailet Contributions
>Affects Versions: master
>Reporter: Tellier Benoit
>Priority: Minor
>  Labels: security
> Fix For: 3.2.0
>
>
> Version 3.4.1 is vulnerable to CVE-2017-15705, CVE-2016-1238, CVE-2018-11780 
> & CVE-2018-11781
> Causing denial of service.
> JAMES only enforces SpamAssassin 3.4.1 use as part of its test suite, and 
> SpamAssassin components are currently undocumented. 
> Thus we just need to update SpamAssassin images used in tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org



[jira] [Created] (JAMES-2558) Use SpamAssassin version 3.4.2 in our test suite

2018-10-10 Thread Tellier Benoit (JIRA)
Tellier Benoit created JAMES-2558:
-

 Summary: Use SpamAssassin version 3.4.2 in our test suite
 Key: JAMES-2558
 URL: https://issues.apache.org/jira/browse/JAMES-2558
 Project: James Server
  Issue Type: Improvement
  Components: mailbox, Mailet Contributions
Affects Versions: master
Reporter: Tellier Benoit
 Fix For: 3.2.0


Version 3.4.1 is vulnerable to CVE-2017-15705, CVE-2016-1238, CVE-2018-11780 & 
CVE-2018-11781

Causing denial of service.

JAMES only enforces SpamAssassin 3.4.1 use as part of its test suite, and 
SpamAssassin components are currently undocumented. 

Thus we just need to update SpamAssassin images used in tests.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

-
To unsubscribe, e-mail: server-dev-unsubscr...@james.apache.org
For additional commands, e-mail: server-dev-h...@james.apache.org