This is an automated email from the ASF dual-hosted git repository.
davsclaus pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/camel.git
The following commit(s) were added to refs/heads/master by this push:
new 2ec6ff9 CAMEL-13108: camel-core - Move dataset into camel-dataset
component.
2ec6ff9 is described below
commit 2ec6ff90eb58e33ae2ac31de030cf24328fa8b8e
Author: Claus Ibsen <[email protected]>
AuthorDate: Sun Apr 7 09:39:40 2019 +0200
CAMEL-13108: camel-core - Move dataset into camel-dataset component.
---
docs/components/modules/ROOT/nav.adoc | 2 +
.../modules/ROOT/pages/dataset-component.adoc | 248 +++++++++++++++++++++
.../modules/ROOT/pages/dataset-test-component.adoc | 152 +++++++++++++
.../camel-spring-boot-dependencies/pom.xml | 10 +
4 files changed, 412 insertions(+)
diff --git a/docs/components/modules/ROOT/nav.adoc
b/docs/components/modules/ROOT/nav.adoc
index c91fd7c..5fe4287 100644
--- a/docs/components/modules/ROOT/nav.adoc
+++ b/docs/components/modules/ROOT/nav.adoc
@@ -74,6 +74,8 @@
* xref:cxf-component.adoc[CXF Component]
* xref:cxfrs-component.adoc[CXF-RS Component]
* xref:dataformat-component.adoc[Data Format Component]
+* xref:dataset-component.adoc[Dataset Component]
+* xref:dataset-test-component.adoc[DataSet Test Component]
* xref:digitalocean-component.adoc[DigitalOcean Component]
* xref:direct-component.adoc[Direct Component]
* xref:direct-vm-component.adoc[Direct VM Component]
diff --git a/docs/components/modules/ROOT/pages/dataset-component.adoc
b/docs/components/modules/ROOT/pages/dataset-component.adoc
new file mode 100644
index 0000000..4d12d6f
--- /dev/null
+++ b/docs/components/modules/ROOT/pages/dataset-component.adoc
@@ -0,0 +1,248 @@
+[[dataset-component]]
+== Dataset Component
+
+*Available as of Camel version 1.3*
+
+Testing of distributed and asynchronous processing is
+notoriously difficult. The <<mock-component,Mock>>, <<mock-component,Test>>
+and <<dataset-component,DataSet>> endpoints work great with the
+Camel Testing Framework to simplify your unit and
+integration testing using
+link:enterprise-integration-patterns.html[Enterprise Integration
+Patterns] and Camel's large range of Components
+together with the powerful Bean Integration.
+
+The DataSet component provides a mechanism to easily perform load & soak
+testing of your system. It works by allowing you to create
+http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/component/dataset/DataSet.html[DataSet
+instances] both as a source of messages and as a way to assert that the
+data set is received.
+
+Camel will use the <<log-component,throughput logger>> when sending
+dataset's.
+
+=== URI format
+
+[source]
+----
+dataset:name[?options]
+----
+
+Where *name* is used to find the
+http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/component/dataset/DataSet.html[DataSet
+instance] in the Registry
+
+Camel ships with a support implementation of
+`org.apache.camel.component.dataset.DataSet`, the
+`org.apache.camel.component.dataset.DataSetSupport` class, that can be
+used as a base for implementing your own DataSet. Camel also ships with
+some implementations that can be used for testing:
+ `org.apache.camel.component.dataset.SimpleDataSet`,
`org.apache.camel.component.dataset.ListDataSet`
+and `org.apache.camel.component.dataset.FileDataSet`, all of which
+extend `DataSetSupport`.
+
+=== Options
+
+
+// component options: START
+The Dataset component has no options.
+// component options: END
+
+
+
+// endpoint options: START
+The Dataset endpoint is configured using URI syntax:
+
+----
+dataset:name
+----
+
+with the following path and query parameters:
+
+==== Path Parameters (1 parameters):
+
+
+[width="100%",cols="2,5,^1,2",options="header"]
+|===
+| Name | Description | Default | Type
+| *name* | *Required* Name of DataSet to lookup in the registry | | DataSet
+|===
+
+
+==== Query Parameters (16 parameters):
+
+
+[width="100%",cols="2,5,^1,2",options="header"]
+|===
+| Name | Description | Default | Type
+| *dataSetIndex* (common) | Controls the behaviour of the CamelDataSetIndex
header. For Consumers: - off = the header will not be set - strict/lenient =
the header will be set For Producers: - off = the header value will not be
verified, and will not be set if it is not present = strict = the header value
must be present and will be verified = lenient = the header value will be
verified if it is present, and will be set if it is not present | lenient |
String
+| *initialDelay* (consumer) | Time period in millis to wait before starting
sending messages. | 1000 | long
+| *minRate* (consumer) | Wait until the DataSet contains at least this number
of messages | 0 | int
+| *preloadSize* (consumer) | Sets how many messages should be preloaded (sent)
before the route completes its initialization | 0 | long
+| *produceDelay* (consumer) | Allows a delay to be specified which causes a
delay when a message is sent by the consumer (to simulate slow processing) | 3
| long
+| *consumeDelay* (producer) | Allows a delay to be specified which causes a
delay when a message is consumed by the producer (to simulate slow processing)
| 0 | long
+| *assertPeriod* (producer) | Sets a grace period after which the mock
endpoint will re-assert to ensure the preliminary assertion is still valid.
This is used for example to assert that exactly a number of messages arrives.
For example if expectedMessageCount(int) was set to 5, then the assertion is
satisfied when 5 or more message arrives. To ensure that exactly 5 messages
arrives, then you would need to wait a little period to ensure no further
message arrives. This is what you can us [...]
+| *expectedCount* (producer) | Specifies the expected number of message
exchanges that should be received by this endpoint. Beware: If you want to
expect that 0 messages, then take extra care, as 0 matches when the tests
starts, so you need to set a assert period time to let the test run for a while
to make sure there are still no messages arrived; for that use
setAssertPeriod(long). An alternative is to use NotifyBuilder, and use the
notifier to know when Camel is done routing some mess [...]
+| *reportGroup* (producer) | A number that is used to turn on throughput
logging based on groups of the size. | | int
+| *resultMinimumWaitTime* (producer) | Sets the minimum expected amount of
time (in millis) the assertIsSatisfied() will wait on a latch until it is
satisfied | 0 | long
+| *resultWaitTime* (producer) | Sets the maximum amount of time (in millis)
the assertIsSatisfied() will wait on a latch until it is satisfied | 0 | long
+| *retainFirst* (producer) | Specifies to only retain the first n'th number of
received Exchanges. This is used when testing with big data, to reduce memory
consumption by not storing copies of every Exchange this mock endpoint
receives. Important: When using this limitation, then the getReceivedCounter()
will still return the actual number of received Exchanges. For example if we
have received 5000 Exchanges, and have configured to only retain the first 10
Exchanges, then the getReceive [...]
+| *retainLast* (producer) | Specifies to only retain the last n'th number of
received Exchanges. This is used when testing with big data, to reduce memory
consumption by not storing copies of every Exchange this mock endpoint
receives. Important: When using this limitation, then the getReceivedCounter()
will still return the actual number of received Exchanges. For example if we
have received 5000 Exchanges, and have configured to only retain the last 20
Exchanges, then the getReceivedCo [...]
+| *sleepForEmptyTest* (producer) | Allows a sleep to be specified to wait to
check that this endpoint really is empty when expectedMessageCount(int) is
called with zero | 0 | long
+| *copyOnExchange* (producer) | Sets whether to make a deep copy of the
incoming Exchange when received at this mock endpoint. Is by default true. |
true | boolean
+| *synchronous* (advanced) | Sets whether synchronous processing should be
strictly used, or Camel is allowed to use asynchronous processing (if
supported). | false | boolean
+|===
+// endpoint options: END
+// spring-boot-auto-configure options: START
+=== Spring Boot Auto-Configuration
+
+When using Spring Boot make sure to use the following Maven dependency to have
support for auto configuration:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-dataset-starter</artifactId>
+ <version>x.x.x</version>
+ <!-- use the same version as your Camel core version -->
+</dependency>
+----
+
+
+The component supports 2 options, which are listed below.
+
+
+
+[width="100%",cols="2,5,^1,2",options="header"]
+|===
+| Name | Description | Default | Type
+| *camel.component.dataset.enabled* | Whether to enable auto configuration of
the dataset component. This is enabled by default. | | Boolean
+| *camel.component.dataset.resolve-property-placeholders* | Whether the
component should resolve property placeholders on itself when starting. Only
properties which are of String type can use property placeholders. | true |
Boolean
+|===
+// spring-boot-auto-configure options: END
+
+You can append query options to the URI in the following format,
+`?option=value&option=value&...`
+
+=== Configuring DataSet
+
+Camel will lookup in the Registry for a bean
+implementing the DataSet interface. So you can register your own DataSet
+as:
+
+[source,xml]
+----
+<bean id="myDataSet" class="com.mycompany.MyDataSet">
+ <property name="size" value="100"/>
+</bean>
+----
+
+=== Example
+
+For example, to test that a set of messages are sent to a queue and then
+consumed from the queue without losing any messages:
+
+[source,java]
+----
+// send the dataset to a queue
+from("dataset:foo").to("activemq:SomeQueue");
+
+// now lets test that the messages are consumed correctly
+from("activemq:SomeQueue").to("dataset:foo");
+----
+
+The above would look in the Registry to find the
+*foo* DataSet instance which is used to create the messages.
+
+Then you create a DataSet implementation, such as using the
+`SimpleDataSet` as described below, configuring things like how big the
+data set is and what the messages look like etc.
+
+
+=== DataSetSupport (abstract class)
+
+The DataSetSupport abstract class is a nice starting point for new
+DataSets, and provides some useful features to derived classes.
+
+==== Properties on DataSetSupport
+
+[width="100%",cols="10%,10%,10%,70%",options="header",]
+|===
+|Property |Type |Default |Description
+
+|`defaultHeaders` |`Map<String,Object>` |`null` |Specifies the default message
body. For SimpleDataSet it is a constant
+payload; though if you want to create custom payloads per message,
+create your own derivation of `DataSetSupport`.
+
+|`outputTransformer` |`org.apache.camel.Processor` |null |
+
+|`size` |`long` |`10` |Specifies how many messages to send/consume.
+
+|`reportCount` |`long` |`-1` |Specifies the number of messages to be received
before reporting
+progress. Useful for showing progress of a large load test. If < 0, then
+`size` / 5, if is 0 then `size`, else set to `reportCount` value.
+|===
+
+=== SimpleDataSet
+
+The `SimpleDataSet` extends `DataSetSupport`, and adds a default body.
+
+==== Additional Properties on SimpleDataSet
+
+[width="100%",cols="10%,10%,10%,70%",options="header",]
+|===
+|Property |Type |Default |Description
+
+|`defaultBody` |`Object` |`<hello>world!</hello>` |Specifies the default
message body. By default, the `SimpleDataSet`
+produces the same constant payload for each exchange. If you want to
+customize the payload for each exchange, create a Camel `Processor` and
+configure the `SimpleDataSet` to use it by setting the
+`outputTransformer` property.
+|===
+
+=== ListDataSet
+
+*Available since Camel 2.17*
+
+The List`DataSet` extends `DataSetSupport`, and adds a list of default
+bodies.
+
+==== Additional Properties on ListDataSet
+
+[width="100%",cols="10%,10%,10%,70%",options="header",]
+|===
+|Property |Type |Default |Description
+
+|`defaultBodies` |`List<Object>` |`empty LinkedList<Object>` |Specifies the
default message body. By default, the `ListDataSet`
+selects a constant payload from the list of `defaultBodies` using the
+`CamelDataSetIndex`. If you want to customize the payload, create a
+Camel `Processor` and configure the `ListDataSet` to use it by setting
+the `outputTransformer` property.
+
+|`size` |`long` |the size of the defaultBodies list |Specifies how many
messages to send/consume. This value can be different
+from the size of the `defaultBodies` list. If the value is less than the
+size of the `defaultBodies` list, some of the list elements will not be
+used. If the value is greater than the size of the `defaultBodies` list,
+the payload for the exchange will be selected using the modulus of the
+`CamelDataSetIndex` and the size of the `defaultBodies` list (i.e.
+`CamelDataSetIndex % defaultBodies.size()` )
+|===
+
+=== FileDataSet
+
+*Available since Camel 2.17*
+
+The `FileDataSet` extends `ListDataSet`, and adds support for loading
+the bodies from a file.
+
+==== Additional Properties on FileDataSet
+
+[width="100%",cols="10%,10%,10%,70%",options="header",]
+|===
+|Property |Type |Default |Description
+
+|`sourceFile` |`File` |null |Specifies the source file for payloads
+
+|`delimiter` |`String` |\z |Specifies the delimiter pattern used by a
`java.util.Scanner` to split
+the file into multiple payloads.
+|===
diff --git a/docs/components/modules/ROOT/pages/dataset-test-component.adoc
b/docs/components/modules/ROOT/pages/dataset-test-component.adoc
new file mode 100644
index 0000000..d4bf677
--- /dev/null
+++ b/docs/components/modules/ROOT/pages/dataset-test-component.adoc
@@ -0,0 +1,152 @@
+[[dataset-test-component]]
+== DataSet Test Component
+*Available as of Camel version 1.3*
+*Available as of Camel version 1.3*
+
+
+
+Testing of distributed and asynchronous processing is
+notoriously difficult. The <<mock-component,Mock>>, <<mock-component,Test>>
+and <<dataset-component,DataSet>> endpoints work great with the
+Camel Testing Framework to simplify your unit and
+integration testing using
+link:enterprise-integration-patterns.html[Enterprise Integration
+Patterns] and Camel's large range of Components
+together with the powerful Bean Integration.
+
+The *dataset-test* component extends the <<mock-component,Mock>> component to
+support pulling messages from another endpoint on startup to set the
+expected message bodies on the underlying <<mock-component,Mock>> endpoint.
+That is, you use the test endpoint in a route and messages arriving on
+it will be implicitly compared to some expected messages extracted from
+some other location.
+
+So you can use, for example, an expected set of message bodies as files.
+This will then set up a properly configured <<mock-component,Mock>>
+endpoint, which is only valid if the received messages match the number
+of expected messages and their message payloads are equal.
+
+Maven users will need to add the following dependency to their `pom.xml`
+for this component when using *Camel 2.8* or older:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-spring</artifactId>
+ <version>x.x.x</version>
+ <!-- use the same version as your Camel core version -->
+</dependency>
+----
+
+From Camel 2.9 onwards the <<test-component,Test>> component is provided
+directly in the camel-core.
+
+=== URI format
+
+[source]
+----
+dataset-test:expectedMessagesEndpointUri
+----
+
+Where *expectedMessagesEndpointUri* refers to some other
+Component URI that the expected message bodies are
+pulled from before starting the test.
+
+=== URI Options
+
+// component options: START
+The DataSet Test component has no options.
+// component options: END
+
+
+// endpoint options: START
+The DataSet Test endpoint is configured using URI syntax:
+
+----
+dataset-test:name
+----
+
+with the following path and query parameters:
+
+==== Path Parameters (1 parameters):
+
+
+[width="100%",cols="2,5,^1,2",options="header"]
+|===
+| Name | Description | Default | Type
+| *name* | *Required* Name of endpoint to lookup in the registry to use for
polling messages used for testing | | String
+|===
+
+
+==== Query Parameters (14 parameters):
+
+
+[width="100%",cols="2,5,^1,2",options="header"]
+|===
+| Name | Description | Default | Type
+| *anyOrder* (producer) | Whether the expected messages should arrive in the
same order or can be in any order. | false | boolean
+| *delimiter* (producer) | The split delimiter to use when split is enabled.
By default the delimiter is new line based. The delimiter can be a regular
expression. | | String
+| *split* (producer) | If enabled the messages loaded from the test endpoint
will be split using new line delimiters so each line is an expected message.
For example to use a file endpoint to load a file where each line is an
expected message. | false | boolean
+| *timeout* (producer) | The timeout to use when polling for message bodies
from the URI | 2000 | long
+| *assertPeriod* (producer) | Sets a grace period after which the mock
endpoint will re-assert to ensure the preliminary assertion is still valid.
This is used for example to assert that exactly a number of messages arrives.
For example if expectedMessageCount(int) was set to 5, then the assertion is
satisfied when 5 or more message arrives. To ensure that exactly 5 messages
arrives, then you would need to wait a little period to ensure no further
message arrives. This is what you can us [...]
+| *expectedCount* (producer) | Specifies the expected number of message
exchanges that should be received by this endpoint. Beware: If you want to
expect that 0 messages, then take extra care, as 0 matches when the tests
starts, so you need to set a assert period time to let the test run for a while
to make sure there are still no messages arrived; for that use
setAssertPeriod(long). An alternative is to use NotifyBuilder, and use the
notifier to know when Camel is done routing some mess [...]
+| *reportGroup* (producer) | A number that is used to turn on throughput
logging based on groups of the size. | | int
+| *resultMinimumWaitTime* (producer) | Sets the minimum expected amount of
time (in millis) the assertIsSatisfied() will wait on a latch until it is
satisfied | 0 | long
+| *resultWaitTime* (producer) | Sets the maximum amount of time (in millis)
the assertIsSatisfied() will wait on a latch until it is satisfied | 0 | long
+| *retainFirst* (producer) | Specifies to only retain the first n'th number of
received Exchanges. This is used when testing with big data, to reduce memory
consumption by not storing copies of every Exchange this mock endpoint
receives. Important: When using this limitation, then the getReceivedCounter()
will still return the actual number of received Exchanges. For example if we
have received 5000 Exchanges, and have configured to only retain the first 10
Exchanges, then the getReceive [...]
+| *retainLast* (producer) | Specifies to only retain the last n'th number of
received Exchanges. This is used when testing with big data, to reduce memory
consumption by not storing copies of every Exchange this mock endpoint
receives. Important: When using this limitation, then the getReceivedCounter()
will still return the actual number of received Exchanges. For example if we
have received 5000 Exchanges, and have configured to only retain the last 20
Exchanges, then the getReceivedCo [...]
+| *sleepForEmptyTest* (producer) | Allows a sleep to be specified to wait to
check that this endpoint really is empty when expectedMessageCount(int) is
called with zero | 0 | long
+| *copyOnExchange* (producer) | Sets whether to make a deep copy of the
incoming Exchange when received at this mock endpoint. Is by default true. |
true | boolean
+| *synchronous* (advanced) | Sets whether synchronous processing should be
strictly used, or Camel is allowed to use asynchronous processing (if
supported). | false | boolean
+|===
+// endpoint options: END
+// spring-boot-auto-configure options: START
+=== Spring Boot Auto-Configuration
+
+When using Spring Boot make sure to use the following Maven dependency to have
support for auto configuration:
+
+[source,xml]
+----
+<dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-dataset-starter</artifactId>
+ <version>x.x.x</version>
+ <!-- use the same version as your Camel core version -->
+</dependency>
+----
+
+
+The component supports 2 options, which are listed below.
+
+
+
+[width="100%",cols="2,5,^1,2",options="header"]
+|===
+| Name | Description | Default | Type
+| *camel.component.dataset-test.enabled* | Whether to enable auto
configuration of the dataset-test component. This is enabled by default. | |
Boolean
+| *camel.component.dataset-test.resolve-property-placeholders* | Whether the
component should resolve property placeholders on itself when starting. Only
properties which are of String type can use property placeholders. | true |
Boolean
+|===
+// spring-boot-auto-configure options: END
+
+
+=== Example
+
+For example, you could write a test case as follows:
+
+[source,java]
+----
+from("seda:someEndpoint").
+ to("dataset-test:file://data/expectedOutput?noop=true");
+----
+
+If your test then invokes the
+http://camel.apache.org/maven/current/camel-core/apidocs/org/apache/camel/component/mock/MockEndpoint.html#assertIsSatisfied(org.apache.camel.CamelContext)[MockEndpoint.assertIsSatisfied(camelContext)
+method], your test case will perform the necessary assertions.
+
+To see how you can set other expectations on the test endpoint, see the
+<<mock-component,Mock>> component.
+
+=== See Also
+
+* Spring Testing
\ No newline at end of file
diff --git
a/platforms/spring-boot/spring-boot-dm/camel-spring-boot-dependencies/pom.xml
b/platforms/spring-boot/spring-boot-dm/camel-spring-boot-dependencies/pom.xml
index 2a6ab61..5a22a52 100644
---
a/platforms/spring-boot/spring-boot-dm/camel-spring-boot-dependencies/pom.xml
+++
b/platforms/spring-boot/spring-boot-dm/camel-spring-boot-dependencies/pom.xml
@@ -949,6 +949,16 @@
</dependency>
<dependency>
<groupId>org.apache.camel</groupId>
+ <artifactId>camel-dataset</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.camel</groupId>
+ <artifactId>camel-dataset-starter</artifactId>
+ <version>${project.version}</version>
+ </dependency>
+ <dependency>
+ <groupId>org.apache.camel</groupId>
<artifactId>camel-digitalocean</artifactId>
<version>${project.version}</version>
</dependency>
