[GitHub] nifi issue #239: Nifi 1540 - AWS Kinesis Get and Put Processors
Github user simplesteph commented on the issue: https://github.com/apache/nifi/pull/239 +1 on separate nar for now. In the short term that's how users (like me) may benefit from it! Should be a quick win as well --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1136: [NIFI-2900] fixes timestamps to be ISO 8601 compliant, as ...
Github user simplesteph commented on the issue: https://github.com/apache/nifi/pull/1136 Sorry I haven't had the time to work on it just yet... Agree with all the above --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1136: [NIFI-2900] fixes timestamps to be ISO 8601 compliant, as ...
Github user simplesteph commented on the issue: https://github.com/apache/nifi/pull/1136 That could be a solution, but I believe it's a more complicated one than just keeping the date-time annotation and make the API compliant with the OpenAPI specs. If you create a "Time" class then it could work. I looked and it seems that yes, Swagger doesn't take JAXB adapters into account. I'm still wondering if it wouldn't be simpler and better to just keep it as date-time and update the UI to re-format it if need be. Which clients would be broken by this change? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1136: [NIFI-2900] fixes timestamps to be ISO 8601 compliant, as ...
Github user simplesteph commented on the issue: https://github.com/apache/nifi/pull/1136 hi @mcgilman , pain commit but it should be good. the resulting `swagger.json` doesn't have any datetime, which makes the api compliant in that regard. Not sure when 2.0 is planned, and not sure how you track todos against it, but converting these fields back to compliant datetime is a must for the next version of the api --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #239: Nifi 1540 - AWS Kinesis Put Processor
Github user simplesteph commented on the issue: https://github.com/apache/nifi/pull/239 Amazing guys congrats! Quick question, are there plans to have a GetKinesis processor too? or should we open another JIRA for that? Kind regards, Stephane [image: Simple Machines] *Stephane Maarek* | Developer +61 416 575 980 steph...@simplemachines.com.au simplemachines.com.au Level 2, 145 William Street, Sydney NSW 2010 On 12 October 2016 at 3:04:31 PM, mans2singh (notificati...@github.com) wrote: @jvwing <https://github.com/jvwing> - Thanks for your review/advice/guidance. Mans â You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <https://github.com/apache/nifi/pull/239#issuecomment-253113649>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AT4raUcLBnkIUM3QuknAgNrQe6rAFe79ks5qzFxPgaJpZM4HfAGh> . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1135: [NIFI-2899] Updated swagger-maven-plugin to 3.0.1
Github user simplesteph commented on the issue: https://github.com/apache/nifi/pull/1135 good stuff. Quick question, what do you think of NiFi automating the build and release of API clients in various language, vs me just maintaining it in my own repos? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1135: [NIFI-2899] Updated swagger-maven-plugin to 3.0.1
Github user simplesteph commented on the issue: https://github.com/apache/nifi/pull/1135 No worries, Iâll build it on my own in my separate repos, see if people have any usage for it, and based on adoption thinking of merging this into the base. What I really like though is that the documentation generated for the API is stellar. On 14 October 2016 at 11:26:08 AM, Andy LoPresto (notificati...@github.com) wrote: I don't think that's something we should do at this time. The NiFi build is already long enough (10 - 20 minutes on commodity hardware depending on multithreading, tests, and contrib-check) and I don't think many people need this functionality, much less in multiple languages, and I don't know how we would determine the default languages -- maybe just Java to start as that's what the codebase is written in? If you want to submit a PR for a custom profile that builds the API clients, I wouldn't object to that as long as it was disabled by default. Not sure what others' feelings are. â You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <https://github.com/apache/nifi/pull/1135#issuecomment-253677513>, or mute the thread <https://github.com/notifications/unsubscribe-auth/AT4rabcK5wAo8dv5KKU__y0sHge8-dKGks5qzswfgaJpZM4KWfwS> . --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #1136: [NIFI-2900] fixes timestamps to be ISO 8601 complia...
GitHub user simplesteph opened a pull request: https://github.com/apache/nifi/pull/1136 [NIFI-2900] fixes timestamps to be ISO 8601 compliant, as it should b⦠â¦e because of OpenAPI specs You can merge this pull request into a Git repository by running: $ git pull https://github.com/simplesteph/nifi NIFI-2900 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1136.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1136 commit 9d0f9a9625cbd4cd9b42e9695b8929087a81 Author: simplesteph <stephane.maa...@gmail.com> Date: 2016-10-14T06:57:41Z [NIFI-2900] fixes timestamps to be ISO 8601 compliant, as it should be because of OpenAPI specs --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #239: Nifi 1540 - AWS Kinesis Put Processor
Github user simplesteph commented on a diff in the pull request:
https://github.com/apache/nifi/pull/239#discussion_r82724343
--- Diff:
nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/kinesis/stream/PutKinesis.java
---
@@ -0,0 +1,174 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.processors.aws.kinesis.stream;
+
+import java.io.ByteArrayOutputStream;
+import java.nio.ByteBuffer;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.List;
+import java.util.Map;
+import java.util.Random;
+
+import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.InputRequirement.Requirement;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.DataUnit;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.util.StandardValidators;
+
+import com.amazonaws.services.kinesis.AmazonKinesisClient;
+import com.amazonaws.services.kinesis.model.PutRecordsRequest;
+import com.amazonaws.services.kinesis.model.PutRecordsRequestEntry;
+import com.amazonaws.services.kinesis.model.PutRecordsResult;
+import com.amazonaws.services.kinesis.model.PutRecordsResultEntry;
+
+@SupportsBatching
+@InputRequirement(Requirement.INPUT_REQUIRED)
+@Tags({"amazon", "aws", "kinesis", "put", "stream"})
+@CapabilityDescription("Sends the contents to a specified Amazon Kinesis. "
++ "In order to send data to Kinesis, the stream name has to be
specified.")
+@WritesAttributes({
+@WritesAttribute(attribute = "aws.kinesis.error.message", description
= "Error message on posting message to AWS Kinesis"),
+@WritesAttribute(attribute = "aws.kinesis.error.code", description =
"Error code for the message when posting to AWS Kinesis"),
+@WritesAttribute(attribute = "aws.kinesis.sequence.number",
description = "Sequence number for the message when posting to AWS Kinesis"),
+@WritesAttribute(attribute = "aws.kinesis.shard.id", description =
"Shard id of the message posted to AWS Kinesis")})
+public class PutKinesis extends AbstractKinesisProcessor {
+
+/**
+ * Kinesis put record response error code
+ */
+public static final String AWS_KINESIS_ERROR_CODE =
"aws.kinesis.error.code";
+
+public static final String AWS_KINESIS_SHARD_ID =
"aws.kinesis.shard.id";
+
+public static final String AWS_KINESIS_SEQUENCE_NUMBER =
"aws.kinesis.sequence.number";
+
+public static final PropertyDescriptor KINESIS_PARTITION_KEY = new
PropertyDescriptor.Builder()
+.displayName("Amazon Kinesis Stream Partition Key")
+.name("amazon-kinesis-stream-partition-key")
+.description("The partition key attribute. If it is not set, a
random value is used")
+.expressionLanguageSupported(true)
+.defaultValue("${kinesis.partition.key}")
+.required(false)
+
.addValidator(StandardValidators.ATTRIBUTE_EXPRESSION_LANGUAGE_VALIDATOR).build();
+
+public static final List properties =
Collections.unmodifiableList(
+Arrays.asList(KINESIS_STREAM_NAM
[GitHub] nifi issue #1136: [NIFI-2900] fixes timestamps to be ISO 8601 compliant, as ...
Github user simplesteph commented on the issue: https://github.com/apache/nifi/pull/1136 I think then we need to add `dataType = "string"` in the annotation to every field `DateTime` that uses any `TimeAdapter` or `DateTimeAdapter`. Hopefully this way the format won't be shown as "date-time". Somehow my maven is acting up and I can't test it... Let me know your thoughts --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
