[
https://issues.apache.org/jira/browse/NIFI-1767?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15310706#comment-15310706
]
ASF GitHub Bot commented on NIFI-1767:
--------------------------------------
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi/pull/349#discussion_r65405495
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/iot/util/MqttWebSocketAsyncClient.java
---
@@ -0,0 +1,113 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.iot.util;
+
+import java.net.URI;
+import java.util.concurrent.LinkedBlockingQueue;
+
+import org.apache.nifi.logging.ProcessorLog;
+import org.eclipse.paho.client.mqttv3.MqttAsyncClient;
+import org.eclipse.paho.client.mqttv3.MqttCallback;
+import org.eclipse.paho.client.mqttv3.MqttException;
+import org.eclipse.paho.client.mqttv3.TimerPingSender;
+import org.eclipse.paho.client.mqttv3.MqttConnectOptions;
+import org.eclipse.paho.client.mqttv3.MqttSecurityException;
+import org.eclipse.paho.client.mqttv3.IMqttDeliveryToken;
+import org.eclipse.paho.client.mqttv3.MqttMessage;
+import org.eclipse.paho.client.mqttv3.internal.NetworkModule;
+import org.eclipse.paho.client.mqttv3.persist.MemoryPersistence;
+
+public class MqttWebSocketAsyncClient extends MqttAsyncClient implements
MqttCallback {
+
+ protected volatile LinkedBlockingQueue<IoTMessage>
awsQueuedMqttMessages = new LinkedBlockingQueue<IoTMessage>();
+ protected final ProcessorLog logger;
+ protected final String serverURI;
+
+ protected static String createDummyURI(String original) {
+ if (!original.startsWith("ws:") && !original.startsWith("wss:")) {
+ return original;
+ }
+ final URI uri = URI.create(original);
+ return "tcp://DUMMY-" + uri.getHost() + ":"
+ + (uri.getPort() > 0 ? uri.getPort() : 80);
+ }
+
+ protected static boolean isDummyURI(String uri) {
+ return uri.startsWith("tcp://DUMMY-");
+ }
+
+ public MqttWebSocketAsyncClient(String serverURI, String clientId,
+ ProcessorLog logger) throws
MqttException {
+ super(createDummyURI(serverURI), clientId, new
MemoryPersistence(), new TimerPingSender());
+ this.serverURI = serverURI;
+ this.logger = logger;
+ this.setCallback(this);
+ }
+
+ @Override
+ protected NetworkModule[] createNetworkModules(String address,
+ MqttConnectOptions
options) throws MqttException{
+ String[] serverURIs = options.getServerURIs();
+ String[] array = serverURIs == null ? new String[] { address } :
+ serverURIs.length == 0 ? new String[] { address }: serverURIs;
+
+ NetworkModule[] networkModules = new NetworkModule[array.length];
+ for (int i = 0; i < array.length; i++) {
+ networkModules[i] = createNetworkModule(array[i], options);
+ }
+ return networkModules;
+ }
+
+ protected NetworkModule createNetworkModule(String input,
+ MqttConnectOptions
options) throws MqttException,
+ MqttSecurityException {
+ final String address = isDummyURI(input) ? this.serverURI : input;
+ if (!address.startsWith("ws:") && !address.startsWith("wss:")) {
+ return super.createNetworkModules(address, options)[0];
+ }
+
+ final String subProtocol = (options.getMqttVersion() ==
MqttConnectOptions.MQTT_VERSION_3_1) ? "mqttv3.1" : "mqtt";
+ return newWebSocketNetworkModule(URI.create(address), subProtocol,
options);
+ }
+
+ protected NetworkModule newWebSocketNetworkModule(URI uri,
+ String subProtocol,
MqttConnectOptions options) {
+ final WebSocketNetworkModule netModule = new
WebSocketNetworkModule(
+ uri, subProtocol, getClientId());
+ netModule.setConnectTimeout(options.getConnectionTimeout());
+ return netModule;
+ }
+
+ public LinkedBlockingQueue<IoTMessage> getAwsQueuedMqttMessages() {
+ return awsQueuedMqttMessages;
+ }
+
+ @Override
+ public void connectionLost(Throwable t) {
+ logger.error("Connection to " + this.getServerURI() + " lost with
cause: " + t.getMessage());
+ }
+
+ @Override
+ public void deliveryComplete(IMqttDeliveryToken token) {
+ }
+
+ @Override
+ public void messageArrived(String topic, MqttMessage message) throws
Exception {
+ logger.info("Message arrived from topic: " + topic);
--- End diff --
This came up in my review as well (I did this at first), we shouldn't set
the messageArrived message to INFO. The rate at which MQTT message potentially
arrive would fill up the logs very quickly. Check out how I handle it in
ConsumeMQTT[1].
[1]
https://github.com/apache/nifi/blob/f47af1ce8336c9305916f00738976f3505b01b0b/nifi-nar-bundles/nifi-mqtt-bundle/nifi-mqtt-processors/src/main/java/org/apache/nifi/processors/mqtt/ConsumeMQTT.java#L310
> AWS IoT processors
> ------------------
>
> Key: NIFI-1767
> URL: https://issues.apache.org/jira/browse/NIFI-1767
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
> Reporter: Kay Lerch
> Attachments: 20160413_apache-nifi-aws-iot-pull-request_lerchkay.pdf
>
>
> Four new processors to communicate with Amazon’s managed device gateway
> service AWS IoT.
> h5.Use cases
> * Consume reported states from a fleet of things managed and secured on
> Amazon’s gateway service
> * Propagate desired states to a fleet of things managed and secured on
> Amazon’s gateway service
> * Intercept M2M communication
> * Hybrid IoT solutions: brings together a managed device gateway in the cloud
> and onpremise data-consumers and -providers.
> h4.GetIOTMqtt:
> Opens up a connection to an AWS-account-specific websocket endpoint in order
> to subscribe to any of the MQTT topics belonging to a registered thing in AWS
> IoT.
> h4.PutIOTMqtt
> Opens up a connection to an AWS-account-specific websocket endpoint in order
> to publish messages to any of the MQTT topics belonging to a registered thing
> in AWS IoT.
> h4.GetIOTShadow
> In AWS IoT a physical thing is represented with its last reported state by
> the so-called thing shadow. This processor reads out the current state of a
> shadow (persisted as JSON) by requesting the managed API of AWS IoT.
> h4.PutIOTShadow
> In AWS IoT a physical thing is represented with its last reported state by
> the so-called thing shadow. This processor updates the current state of a
> shadow (persisted as JSON) by requesting the managed API of AWS IoT. An
> update to a shadow lets AWS IoT propagate changes to the MQTT topics of the
> thing.
> h5.Known issues:
> * It was hard for me to write appropriate integration tests since the MQTT
> processors work with durable websocket-connections which are kind of tough to
> test. With your help I would love to do a better job on testing and hand it
> in later on. All of the processors were tested in a live-scenario which ran
> over a longer period of time. Didn’t observe any issue.
> * I got rid of all the properties for the deprecated
> AWSCredentialProviderService and only made use of
> AWSCredentialsProviderControllerService. If both are still necessary for
> backward-compatibilities sake I would add the deprecated feature.
> Refers to Pull Request 349: https://github.com/apache/nifi/pull/349
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
