[
https://issues.apache.org/jira/browse/CAMEL-12316?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16385688#comment-16385688
]
ASF GitHub Bot commented on CAMEL-12316:
----------------------------------------
oscerd closed pull request #2243: CAMEL-12316 : MongoDB - Add allowDiskUse
option to aggregate operation
URL: https://github.com/apache/camel/pull/2243
This is a PR merged from a forked repository.
As GitHub hides the original diff on merge, it is displayed below for
the sake of provenance:
As this is a foreign pull request (from a fork), the diff is supplied
below (as it won't show otherwise due to GitHub magic):
diff --git a/components/camel-mongodb/src/main/docs/mongodb-component.adoc
b/components/camel-mongodb/src/main/docs/mongodb-component.adoc
index dbbff25a4ff..e3dd18c321d 100644
--- a/components/camel-mongodb/src/main/docs/mongodb-component.adoc
+++ b/components/camel-mongodb/src/main/docs/mongodb-component.adoc
@@ -528,9 +528,9 @@ containing the number of records deleted (copied from
*Available as of Camel 2.14*
Perform a aggregation with the given pipeline contained in the
-body. *Aggregations could be long and heavy operations. Use with care.*
+body.
+*Aggregations could be long and heavy operations. Use with care.*
-
[source,java]
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
@@ -541,15 +541,18 @@ from("direct:aggregate")
.to("mock:resultAggregate");
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
-Efficient retrieval is supported via outputType=DBCursor and the following
header :
+Supports the following IN message headers:
[width="100%",cols="10%,10%,10%,70%",options="header",]
|=======================================================================
|Header key |Quick constant |Description (extracted from MongoDB API doc)
|Expected type
|`CamelMongoDbBatchSize` |`MongoDbConstants.BATCH_SIZE` | Sets the number of
documents to return per batch. |int/Integer
+|`CamelMongoDbAllowDiskUse` |`MongoDbConstants.ALLOW_DISK_USE` | Enable
aggregation pipeline stages to write data to temporary files. |boolean/Boolean
|=======================================================================
+Efficient retrieval is supported via outputType=DBCursor.
+
You can also "stream" the documents returned from the server into your route
by including outputType=DBCursor (Camel 2.21+) as an endpoint option
which may prove simpler than setting the above headers. This hands your
Exchange the DBCursor from the Mongo driver, just as if you were executing
the aggregate() within the Mongo shell, allowing your route to iterate over
the results. By default and without this option, this component will load
diff --git
a/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbConstants.java
b/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbConstants.java
index 88cf6724ad3..ba3e71bf4be 100644
---
a/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbConstants.java
+++
b/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbConstants.java
@@ -39,6 +39,7 @@
public static final String WRITERESULT = "CamelMongoWriteResult";
public static final String OID = "CamelMongoOid";
public static final String DISTINCT_QUERY_FIELD =
"CamelMongoDbDistinctQueryField";
+ public static final String ALLOW_DISK_USE = "CamelMongoDbAllowDiskUse";
private MongoDbConstants() {
}
diff --git
a/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbProducer.java
b/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbProducer.java
index f0192d938cf..63568e7a1ed 100644
---
a/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbProducer.java
+++
b/components/camel-mongodb/src/main/java/org/apache/camel/component/mongodb/MongoDbProducer.java
@@ -510,7 +510,10 @@ private Object getMultiInsertBody(Exchange exchange) {
if (batchSize != null) {
aggregationResult.batchSize(batchSize);
}
-
+
+ Boolean allowDiskUse =
exchange.getIn().getHeader(MongoDbConstants.ALLOW_DISK_USE, Boolean.FALSE,
Boolean.class);
+ aggregationResult.allowDiskUse(allowDiskUse);
+
Iterable<BasicDBObject> result;
if
(!MongoDbOutputType.DBCursor.equals(endpoint.getOutputType())) {
try {
diff --git
a/components/camel-mongodb/src/test/java/org/apache/camel/component/mongodb/MongoDbAggregateOperationTest.java
b/components/camel-mongodb/src/test/java/org/apache/camel/component/mongodb/MongoDbAggregateOperationTest.java
index b547d237f98..4f37a673d6a 100644
---
a/components/camel-mongodb/src/test/java/org/apache/camel/component/mongodb/MongoDbAggregateOperationTest.java
+++
b/components/camel-mongodb/src/test/java/org/apache/camel/component/mongodb/MongoDbAggregateOperationTest.java
@@ -16,7 +16,9 @@
*/
package org.apache.camel.component.mongodb;
+import java.util.HashMap;
import java.util.List;
+import java.util.Map;
import com.mongodb.BasicDBObject;
import com.mongodb.DBObject;
@@ -77,14 +79,19 @@ public void testAggregateDBCursor() {
}
@Test
- public void testAggregateDBCursorBatchSize() {
+ public void testAggregateWithOptions() {
// Test that the collection has 0 documents in it
assertEquals(0, testCollection.count());
pumpDataIntoTestCollection();
+ Map<String, Object> options = new HashMap<>();
+ options.put(MongoDbConstants.BATCH_SIZE, 10);
+ options.put(MongoDbConstants.ALLOW_DISK_USE, true);
+
Object result = template
- .requestBodyAndHeader("direct:aggregateDBCursor",
- "[{ $match : {$or : [{\"scientist\" :
\"Darwin\"},{\"scientist\" : \"Einstein\"}]}}]", MongoDbConstants.BATCH_SIZE,
10);
+ .requestBodyAndHeaders("direct:aggregateDBCursor",
+ "[{ $match : {$or : [{\"scientist\" :
\"Darwin\"},{\"scientist\" : \"Einstein\"}]}}]", options);
+
assertTrue("Result is not of type DBCursor", result instanceof
MongoIterable);
diff --git a/components/camel-mongodb3/src/main/docs/mongodb3-component.adoc
b/components/camel-mongodb3/src/main/docs/mongodb3-component.adoc
index aef5bf03403..335134008ad 100644
--- a/components/camel-mongodb3/src/main/docs/mongodb3-component.adoc
+++ b/components/camel-mongodb3/src/main/docs/mongodb3-component.adoc
@@ -574,9 +574,9 @@ containing the number of records deleted (copied from
===== aggregate
Perform a aggregation with the given pipeline contained in the
-body. *Aggregations could be long and heavy operations. Use with care.*
+body.
+*Aggregations could be long and heavy operations. Use with care.*
-
[source,java]
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
@@ -590,15 +590,18 @@ from("direct:aggregate")
----------------------------------------------------------------------------------------------------------------------------------------------------------------------
-Efficient retrieval is supported via outputType=MongoIterable and the
following header :
+Supports the following IN message headers:
[width="100%",cols="10%,10%,10%,70%",options="header",]
|=======================================================================
|Header key |Quick constant |Description (extracted from MongoDB API doc)
|Expected type
|`CamelMongoDbBatchSize` |`MongoDbConstants.BATCH_SIZE` | Sets the number of
documents to return per batch. |int/Integer
+|`CamelMongoDbAllowDiskUse` |`MongoDbConstants.ALLOW_DISK_USE` | Enable
aggregation pipeline stages to write data to temporary files. |boolean/Boolean
|=======================================================================
+Efficient retrieval is supported via outputType=MongoIterable.
+
You can also "stream" the documents returned from the server into your route
by including outputType=DBCursor (Camel 2.21+) as an endpoint option
which may prove simpler than setting the above headers. This hands your
Exchange the DBCursor from the Mongo driver, just as if you were executing
the aggregate() within the Mongo shell, allowing your route to iterate over
the results. By default and without this option, this component will load
diff --git
a/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbConstants.java
b/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbConstants.java
index 7196a9897c3..70fe95c60ac 100644
---
a/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbConstants.java
+++
b/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbConstants.java
@@ -40,6 +40,7 @@
public static final String WRITERESULT = "CamelMongoWriteResult";
public static final String OID = "CamelMongoOid";
public static final String DISTINCT_QUERY_FIELD =
"CamelMongoDbDistinctQueryField";
+ public static final String ALLOW_DISK_USE = "CamelMongoDbAllowDiskUse";
public static final String MONGO_ID = "_id"; // default id field
diff --git
a/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbProducer.java
b/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbProducer.java
index 17cd024a8a4..ad558a852e4 100644
---
a/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbProducer.java
+++
b/components/camel-mongodb3/src/main/java/org/apache/camel/component/mongodb3/MongoDbProducer.java
@@ -47,6 +47,7 @@
import org.slf4j.LoggerFactory;
import static com.mongodb.client.model.Filters.eq;
+import static
org.apache.camel.component.mongodb3.MongoDbConstants.ALLOW_DISK_USE;
import static org.apache.camel.component.mongodb3.MongoDbConstants.BATCH_SIZE;
import static org.apache.camel.component.mongodb3.MongoDbConstants.COLLECTION;
import static
org.apache.camel.component.mongodb3.MongoDbConstants.COLLECTION_INDEX;
@@ -530,8 +531,11 @@ private void processAndTransferResult(Object result,
Exchange exchange, MongoDbO
if (batchSize != null) {
aggregationResult.batchSize(batchSize);
}
-
- Iterable<Document> result;
+
+ Boolean allowDiskUse =
exchange.getIn().getHeader(MongoDbConstants.ALLOW_DISK_USE, Boolean.FALSE,
Boolean.class);
+ aggregationResult.allowDiskUse(allowDiskUse);
+
+ Iterable<Document> result;
if
(!MongoDbOutputType.MongoIterable.equals(endpoint.getOutputType())) {
try {
result = new ArrayList<>();
diff --git
a/components/camel-mongodb3/src/test/java/org/apache/camel/component/mongodb3/MongoDbAggregateOperationTest.java
b/components/camel-mongodb3/src/test/java/org/apache/camel/component/mongodb3/MongoDbAggregateOperationTest.java
index 9241567d405..be849bc8f75 100644
---
a/components/camel-mongodb3/src/test/java/org/apache/camel/component/mongodb3/MongoDbAggregateOperationTest.java
+++
b/components/camel-mongodb3/src/test/java/org/apache/camel/component/mongodb3/MongoDbAggregateOperationTest.java
@@ -16,7 +16,9 @@
*/
package org.apache.camel.component.mongodb3;
+import java.util.HashMap;
import java.util.List;
+import java.util.Map;
import com.mongodb.client.MongoIterable;
@@ -76,14 +78,18 @@ public void testAggregateDBCursor() {
}
@Test
- public void testAggregateDBCursorBatchSize() {
+ public void testAggregateWithOptions() {
// Test that the collection has 0 documents in it
assertEquals(0, testCollection.count());
pumpDataIntoTestCollection();
+ Map<String, Object> options = new HashMap<>();
+ options.put(MongoDbConstants.BATCH_SIZE, 10);
+ options.put(MongoDbConstants.ALLOW_DISK_USE, true);
+
Object result = template
- .requestBodyAndHeader("direct:aggregateDBCursor",
- "[{ $match : {$or : [{\"scientist\" :
\"Darwin\"},{\"scientist\" : \"Einstein\"}]}}]", MongoDbConstants.BATCH_SIZE,
10);
+ .requestBodyAndHeaders("direct:aggregateDBCursor",
+ "[{ $match : {$or : [{\"scientist\" :
\"Darwin\"},{\"scientist\" : \"Einstein\"}]}}]", options);
assertTrue("Result is not of type DBCursor", result instanceof
MongoIterable);
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
> mongodb : Add allowDiskUse option to aggregate operation
> --------------------------------------------------------
>
> Key: CAMEL-12316
> URL: https://issues.apache.org/jira/browse/CAMEL-12316
> Project: Camel
> Issue Type: New Feature
> Components: camel-mongodb, camel-mongodb3
> Reporter: Farès Hassak
> Assignee: Andrea Cosentino
> Priority: Major
> Fix For: 2.21.0
>
>
> Hello,
> We need the possibility to pass allowDiskUse option to aggregate operation
> in header message with false as default value.
> https://docs.mongodb.com/manual/reference/command/aggregate/
> https://docs.mongodb.com/manual/reference/method/db.collection.aggregate/#example-aggregate-method-external-sort
> Farès
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
