This is an automated email from the ASF dual-hosted git repository.
szaszm pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/nifi-minifi-cpp.git
The following commit(s) were added to refs/heads/main by this push:
new 0f73c02 MINIFICPP-1358 Add SendBody property functionality to
InvokeHTTP
0f73c02 is described below
commit 0f73c02c82a4f6959159526be9585969a3c898db
Author: Gabor Gyimesi <[email protected]>
AuthorDate: Wed Apr 14 15:06:25 2021 +0200
MINIFICPP-1358 Add SendBody property functionality to InvokeHTTP
This closes #1041
Signed-off-by: Marton Szasz <[email protected]>
---
PROCESSORS.md | 9 +++++---
docker/DockerVerify.sh | 1 +
docker/test/integration/features/http.feature | 17 +++++++++++++-
extensions/http-curl/processors/InvokeHTTP.cpp | 26 +++++++++++++++++-----
extensions/http-curl/processors/InvokeHTTP.h | 2 ++
libminifi/test/resources/TestInvokeHTTPPost.yml | 2 +-
.../test/resources/TestInvokeHTTPPostSecure.yml | 4 ++--
7 files changed, 48 insertions(+), 13 deletions(-)
diff --git a/PROCESSORS.md b/PROCESSORS.md
index 0dccd22..56f0be9 100644
--- a/PROCESSORS.md
+++ b/PROCESSORS.md
@@ -685,15 +685,18 @@ In the list below, the names of required properties
appear in bold. Any other pr
|Follow Redirects|true||Follow HTTP redirects issued by remote server.|
|HTTP Method|GET||HTTP request method (GET, POST, PUT, PATCH, DELETE, HEAD,
OPTIONS). Arbitrary methods are also supported. Methods other than POST, PUT
and PATCH will be sent without a message body.|
|Include Date Header|true||Include an RFC-2616 Date header in the request.|
+|invokehttp-proxy-password|||Password to set when authenticating against proxy|
+|invokehttp-proxy-username|||Username to set when authenticating against proxy|
+|Penalize on "No Retry"|false||Enabling this property will penalize FlowFiles
that are routed to the "No Retry" relationship.|
|Proxy Host|||The fully qualified hostname or IP address of the proxy server|
|Proxy Port|||The port of the proxy server|
+|Put Response Body in Attribute|||If set, the response body received back will
be put into an attribute of the original FlowFile instead of a separate
FlowFile. The attribute key to put to is determined by evaluating value of this
property.|
|Read Timeout|15 secs||Max wait time for response from remote service.|
|Remote URL|||Remote URL which will be connected to, including scheme, host,
port, path.<br/>**Supports Expression Language: true**|
+|send-message-body|true||DEPRECATED. Only kept for backwards compatibility, no
functionality is included.|
+|Send Message Body|true||If true, sends the HTTP message body on
POST/PUT/PATCH requests (default). If false, suppresses the message body and
content-type header for these requests.|
|SSL Context Service|||The SSL Context Service used to provide client
certificate information for TLS/SSL (https) connections.|
|Use Chunked Encoding|false||When POST'ing, PUT'ing or PATCH'ing content set
this property to true in order to not pass the 'Content-length' header and
instead send 'Transfer-Encoding' with a value of 'chunked'. This will enable
the data transfer mechanism which was introduced in HTTP 1.1 to pass data of
unknown lengths in chunks.|
-|invokehttp-proxy-password|||Password to set when authenticating against proxy|
-|invokehttp-proxy-username|||Username to set when authenticating against proxy|
-|send-message-body|true||If true, sends the HTTP message body on
POST/PUT/PATCH requests (default). If false, suppresses the message body and
content-type header for these requests.|
### Relationships
| Name | Description |
diff --git a/docker/DockerVerify.sh b/docker/DockerVerify.sh
index 4ccc98f..7361a2e 100755
--- a/docker/DockerVerify.sh
+++ b/docker/DockerVerify.sh
@@ -84,6 +84,7 @@ exec
behave "${BEHAVE_OPTS[@]}" "features/http.feature" -n "A MiNiFi instance
transfers data to another MiNiFi instance" &&
behave "${BEHAVE_OPTS[@]}" "features/http.feature" -n "A MiNiFi instance
sends data through a HTTP proxy and another one listens" &&
behave "${BEHAVE_OPTS[@]}" "features/http.feature" -n "A MiNiFi instance and
transfers hashed data to another MiNiFi instance" &&
+ behave "${BEHAVE_OPTS[@]}" "features/http.feature" -n "A MiNiFi instance
transfers data to another MiNiFi instance without message body" &&
behave "${BEHAVE_OPTS[@]}" "features/kafka.feature" -n "A MiNiFi instance
transfers data to a kafka broker" &&
behave "${BEHAVE_OPTS[@]}" "features/kafka.feature" -n "PublishKafka sends
flowfiles to failure when the broker is not available" &&
behave "${BEHAVE_OPTS[@]}" "features/kafka.feature" -n "PublishKafka sends
can use SSL connect" &&
diff --git a/docker/test/integration/features/http.feature
b/docker/test/integration/features/http.feature
index 1f5e89d..2e5b479 100644
--- a/docker/test/integration/features/http.feature
+++ b/docker/test/integration/features/http.feature
@@ -33,7 +33,7 @@ Feature: Sending data using InvokeHTTP to a receiver using
ListenHTTP
| InvokeHTTP | invokehttp-proxy-password | test101 |
And the "success" relationship of the GetFile processor is connected to
the InvokeHTTP
- And a http proxy server "http-proxy" is set up accordingly
+ And a http proxy server "http-proxy" is set up accordingly
And a ListenHTTP processor with the "Listening Port" property set to
"8080" in a "minifi-listen" flow
And a PutFile processor with the "Directory" property set to "/tmp/output"
in the "minifi-listen" flow
@@ -58,3 +58,18 @@ Feature: Sending data using InvokeHTTP to a receiver using
ListenHTTP
When both instances start up
Then a flowfile with the content "test" is placed in the monitored
directory in less than 30 seconds
+
+ Scenario: A MiNiFi instance transfers data to another MiNiFi instance
without message body
+ Given a GetFile processor with the "Input Directory" property set to
"/tmp/input"
+ And a file with the content "test" is present in "/tmp/input"
+ And a InvokeHTTP processor with the "Remote URL" property set to
"http://secondary:8080/contentListener";
+ And the "HTTP Method" of the InvokeHTTP processor is set to "POST"
+ And the "Send Message Body" of the InvokeHTTP processor is set to "false"
+ And the "success" relationship of the GetFile processor is connected to
the InvokeHTTP
+
+ And a ListenHTTP processor with the "Listening Port" property set to
"8080" in a "secondary" flow
+ And a PutFile processor with the "Directory" property set to "/tmp/output"
in the "secondary" flow
+ And the "success" relationship of the ListenHTTP processor is connected to
the PutFile
+
+ When both instances start up
+ Then at least one empty flowfile is placed in the monitored directory in
less than 30 seconds
diff --git a/extensions/http-curl/processors/InvokeHTTP.cpp
b/extensions/http-curl/processors/InvokeHTTP.cpp
index d6fa2f8..42f2933 100644
--- a/extensions/http-curl/processors/InvokeHTTP.cpp
+++ b/extensions/http-curl/processors/InvokeHTTP.cpp
@@ -99,9 +99,17 @@ core::Property InvokeHTTP::ContentType("Content-type", "The
Content-Type to spec
"POST or PATCH. In the case of an empty
value after evaluating an expression language expression, "
"Content-Type defaults to",
"application/octet-stream");
-core::Property InvokeHTTP::SendBody("send-message-body", "If true, sends the
HTTP message body on POST/PUT/PATCH requests (default). "
- "If false, suppresses the message body and
content-type header for these requests.",
- "true");
+core::Property InvokeHTTP::SendBody(
+ core::PropertyBuilder::createProperty("send-message-body", "Send Body")
+ ->withDescription("DEPRECATED. Only kept for backwards compatibility, no
functionality is included.")
+ ->withDefaultValue<bool>(true)
+ ->build());
+core::Property InvokeHTTP::SendMessageBody(
+ core::PropertyBuilder::createProperty("Send Message Body")
+ ->withDescription("If true, sends the HTTP message body on
POST/PUT/PATCH requests (default). "
+ "If false, suppresses the message body and
content-type header for these requests.")
+ ->withDefaultValue<bool>(true)
+ ->build());
core::Property InvokeHTTP::UseChunkedEncoding("Use Chunked Encoding", "When
POST'ing, PUT'ing or PATCH'ing content set this property to true in order to
not pass the 'Content-length' header"
" and instead send
'Transfer-Encoding' with a value of 'chunked'. This will enable the data
transfer mechanism which was introduced in HTTP 1.1 "
"to pass data of unknown lengths
in chunks.",
@@ -156,6 +164,7 @@ void InvokeHTTP::initialize() {
properties.insert(UseChunkedEncoding);
properties.insert(ContentType);
properties.insert(SendBody);
+ properties.insert(SendMessageBody);
properties.insert(DisablePeerVerification);
properties.insert(AlwaysOutputResponse);
properties.insert(FollowRedirects);
@@ -264,6 +273,7 @@ void InvokeHTTP::onSchedule(const
std::shared_ptr<core::ProcessContext> &context
context->getProperty(ProxyUsername.getName(), proxy_.username);
context->getProperty(ProxyPassword.getName(), proxy_.password);
context->getProperty(FollowRedirects.getName(), follow_redirects_);
+ context->getProperty(SendMessageBody.getName(), send_body_);
}
InvokeHTTP::~InvokeHTTP() = default;
@@ -311,7 +321,7 @@ void InvokeHTTP::onTrigger(const
std::shared_ptr<core::ProcessContext> &context,
client.setReadTimeout(read_timeout_ms_);
client.setFollowRedirects(follow_redirects_);
- if (!content_type_.empty()) {
+ if (send_body_ && !content_type_.empty()) {
client.setContentType(content_type_);
}
@@ -331,12 +341,16 @@ void InvokeHTTP::onTrigger(const
std::shared_ptr<core::ProcessContext> &context,
std::shared_ptr<ResourceClaim> claim = flowFile->getResourceClaim();
if (claim) {
callback = std::unique_ptr<utils::ByteInputCallBack>(new
utils::ByteInputCallBack());
- session->read(flowFile, callback.get());
+ if (send_body_) {
+ session->read(flowFile, callback.get());
+ }
callbackObj = std::unique_ptr<utils::HTTPUploadCallback>(new
utils::HTTPUploadCallback);
callbackObj->ptr = callback.get();
callbackObj->pos = 0;
logger_->log_trace("InvokeHTTP -- Setting callback, size is %d",
callback->getBufferSize());
- if (!use_chunked_encoding_) {
+ if (!send_body_) {
+ client.appendHeader("Content-Length", "0");
+ } else if (!use_chunked_encoding_) {
client.appendHeader("Content-Length",
std::to_string(flowFile->getSize()));
}
client.setUploadCallback(callbackObj.get());
diff --git a/extensions/http-curl/processors/InvokeHTTP.h
b/extensions/http-curl/processors/InvokeHTTP.h
index 34b9a70..9a0fca8 100644
--- a/extensions/http-curl/processors/InvokeHTTP.h
+++ b/extensions/http-curl/processors/InvokeHTTP.h
@@ -72,6 +72,7 @@ class InvokeHTTP : public core::Processor {
static core::Property ProxyPassword;
static core::Property ContentType;
static core::Property SendBody;
+ static core::Property SendMessageBody;
static core::Property UseChunkedEncoding;
static core::Property DisablePeerVerification;
static core::Property PropPutOutputAttributes;
@@ -159,6 +160,7 @@ class InvokeHTTP : public core::Processor {
bool disable_peer_verification_{false};
utils::HTTPProxy proxy_;
bool follow_redirects_{true};
+ bool send_body_{true};
private:
std::shared_ptr<logging::Logger>
logger_{logging::LoggerFactory<InvokeHTTP>::getLogger()};
diff --git a/libminifi/test/resources/TestInvokeHTTPPost.yml
b/libminifi/test/resources/TestInvokeHTTPPost.yml
index cb8eb8e..e1e87cd 100644
--- a/libminifi/test/resources/TestInvokeHTTPPost.yml
+++ b/libminifi/test/resources/TestInvokeHTTPPost.yml
@@ -78,7 +78,7 @@ Processors:
Read Timeout: 1 s
Remote URL: http://localhost:0/minifi
Use Chunked Encoding: 'false'
- send-message-body: 'true'
+ Send Message Body: 'true'
- id: 8543b2e2-a373-4d8d-8710-ac314666696b
name: LogAttribute
class: org.apache.nifi.minifi.processors.LogAttribute
diff --git a/libminifi/test/resources/TestInvokeHTTPPostSecure.yml
b/libminifi/test/resources/TestInvokeHTTPPostSecure.yml
index 6e9bd9a..5f92372 100644
--- a/libminifi/test/resources/TestInvokeHTTPPostSecure.yml
+++ b/libminifi/test/resources/TestInvokeHTTPPostSecure.yml
@@ -77,7 +77,7 @@ Processors:
Read Timeout: 1 s
Remote URL: https://localhost:0/minifi
Use Chunked Encoding: 'false'
- send-message-body: 'true'
+ Send Message Body: 'true'
SSL Context Service: SSLContextService
- id: 8543b2e2-a373-4d8d-8710-ac314666696b
name: LogAttribute
@@ -157,7 +157,7 @@ Connections:
max work queue data size: 10000 B
flowfile expiration: 0 seconds
queue prioritizer class: ''
-
+
Controller Services:
- name: SSLContextService
id: 2438e3c8-015a-1000-79ca-83af40ec1994
