[jira] [Created] (NIFI-4331) Add "NotFound" route to QueryElasticsearchHttp
Phil Rogers created NIFI-4331: - Summary: Add "NotFound" route to QueryElasticsearchHttp Key: NIFI-4331 URL: https://issues.apache.org/jira/browse/NIFI-4331 Project: Apache NiFi Issue Type: New Feature Components: Extensions Affects Versions: 1.3.0 Environment: Linux Centos 7 Reporter: Phil Rogers Priority: Minor Fix For: 1.3.0 I have a need to check whether a record exists in Elasticsearch to decide what to do next. The attribute I need to check isn't a key in elasticsearch so I am using the QueryElasticsearchHttp to do the query. If the QueryElasticsearchHttp process experiences a not found on the query result the processor drops any further processing of that queued entry. It would be great if the QueryElasticsearchHttp processor could have a "not found" route added to it that way smarter logic could be applied Thanks in advance -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-2528) Update ListenHTTP to honor SSLContextService Protocols
[ https://issues.apache.org/jira/browse/NIFI-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146288#comment-16146288 ] ASF GitHub Bot commented on NIFI-2528: -- Github user m-hogue commented on the issue: https://github.com/apache/nifi/pull/1986 @alopresto : All of your recommendations have been implemented. Please let me know if you'd like any more changes. > Update ListenHTTP to honor SSLContextService Protocols > -- > > Key: NIFI-2528 > URL: https://issues.apache.org/jira/browse/NIFI-2528 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0, 0.8.0, 0.7.1 >Reporter: Joe Skora >Assignee: Michael Hogue > > Update ListenHTTP to honor SSLContextService Protocols as [NIFI-1688] did for > PostHTTP. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1986: NIFI-2528: added support for SSLContextService protocols i...
Github user m-hogue commented on the issue: https://github.com/apache/nifi/pull/1986 @alopresto : All of your recommendations have been implemented. Please let me know if you'd like any more changes. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2528) Update ListenHTTP to honor SSLContextService Protocols
[
https://issues.apache.org/jira/browse/NIFI-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146012#comment-16146012
]
ASF GitHub Bot commented on NIFI-2528:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135896956
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/SSLContextServiceUtils.java
---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.components.AllowableValue;
+
+public class SSLContextServiceUtils {
--- End diff --
Mike, I did read the SO link you had posted but I also read [Java 8 Default
and Static
Methods](https://www.intertech.com/Blog/java-8-tutorial-default-and-static-methods-guide/)
and I think as long as the definitions are on the interfaces, it will be ok.
> Update ListenHTTP to honor SSLContextService Protocols
> --
>
> Key: NIFI-2528
> URL: https://issues.apache.org/jira/browse/NIFI-2528
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.8.0, 0.7.1
>Reporter: Joe Skora
>Assignee: Michael Hogue
>
> Update ListenHTTP to honor SSLContextService Protocols as [NIFI-1688] did for
> PostHTTP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #1986: NIFI-2528: added support for SSLContextService prot...
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135896956
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/SSLContextServiceUtils.java
---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.components.AllowableValue;
+
+public class SSLContextServiceUtils {
--- End diff --
Mike, I did read the SO link you had posted but I also read [Java 8 Default
and Static
Methods](https://www.intertech.com/Blog/java-8-tutorial-default-and-static-methods-guide/)
and I think as long as the definitions are on the interfaces, it will be ok.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-2528) Update ListenHTTP to honor SSLContextService Protocols
[
https://issues.apache.org/jira/browse/NIFI-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146007#comment-16146007
]
ASF GitHub Bot commented on NIFI-2528:
--
Github user m-hogue commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135896059
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/SSLContextServiceUtils.java
---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.components.AllowableValue;
+
+public class SSLContextServiceUtils {
--- End diff --
You can indeed have static methods in interfaces, but they cannot be
overridden by child classes. They 'belong' to the interface. There were also
default methods added in Java 8, which can be overridden by child classes, but
they aren't static meaning that they can't be used in statically initialized
variables. For these reasons, i elected to put the methods in a util class -
but i totally understand the concern here.
See this SO article:
https://stackoverflow.com/questions/27833168/difference-between-static-and-default-methods-in-interface
> Update ListenHTTP to honor SSLContextService Protocols
> --
>
> Key: NIFI-2528
> URL: https://issues.apache.org/jira/browse/NIFI-2528
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.8.0, 0.7.1
>Reporter: Joe Skora
>Assignee: Michael Hogue
>
> Update ListenHTTP to honor SSLContextService Protocols as [NIFI-1688] did for
> PostHTTP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #1986: NIFI-2528: added support for SSLContextService prot...
Github user m-hogue commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135896059
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/SSLContextServiceUtils.java
---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.components.AllowableValue;
+
+public class SSLContextServiceUtils {
--- End diff --
You can indeed have static methods in interfaces, but they cannot be
overridden by child classes. They 'belong' to the interface. There were also
default methods added in Java 8, which can be overridden by child classes, but
they aren't static meaning that they can't be used in statically initialized
variables. For these reasons, i elected to put the methods in a util class -
but i totally understand the concern here.
See this SO article:
https://stackoverflow.com/questions/27833168/difference-between-static-and-default-methods-in-interface
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-2528) Update ListenHTTP to honor SSLContextService Protocols
[
https://issues.apache.org/jira/browse/NIFI-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16146002#comment-16146002
]
ASF GitHub Bot commented on NIFI-2528:
--
Github user m-hogue commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135895369
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/StandardRestrictedSSLContextService.java
---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+
+/**
+ * This class is functionally the same as {@link
StandardSSLContextService}, but it restricts the allowable
+ * values that can be selected for SSL protocols.
+ */
+@Tags({"ssl", "secure", "certificate", "keystore", "truststore", "jks",
"p12", "pkcs12", "pkcs"})
+@CapabilityDescription("Restricted implementation of the
SSLContextService. Provides the ability to configure "
++ "keystore and/or truststore properties once and reuse that
configuration throughout the application, "
++ "but only allows a restricted set of SSL protocols to be chosen.
The set of protocols selectable will "
++ "evolve over time as new protocols emerge and older protocols
are deprecated. This service is recommended "
++ "over StandardSSLContextService if a component doesn't expect to
communicate with legacy systems since it's "
++ "unlikely that legacy systems will support these protocols.")
+public class StandardRestrictedSSLContextService extends
StandardSSLContextService implements RestrictedSSLContextService {
+
+public static final PropertyDescriptor RESTRICTED_SSL_ALGORITHM = new
PropertyDescriptor.Builder()
+.name("SSL Protocol")
+.defaultValue("TLSv1.2")
--- End diff --
Gotcha. I can make this change.
> Update ListenHTTP to honor SSLContextService Protocols
> --
>
> Key: NIFI-2528
> URL: https://issues.apache.org/jira/browse/NIFI-2528
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.8.0, 0.7.1
>Reporter: Joe Skora
>Assignee: Michael Hogue
>
> Update ListenHTTP to honor SSLContextService Protocols as [NIFI-1688] did for
> PostHTTP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #1986: NIFI-2528: added support for SSLContextService prot...
Github user m-hogue commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135895369
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/StandardRestrictedSSLContextService.java
---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+
+/**
+ * This class is functionally the same as {@link
StandardSSLContextService}, but it restricts the allowable
+ * values that can be selected for SSL protocols.
+ */
+@Tags({"ssl", "secure", "certificate", "keystore", "truststore", "jks",
"p12", "pkcs12", "pkcs"})
+@CapabilityDescription("Restricted implementation of the
SSLContextService. Provides the ability to configure "
++ "keystore and/or truststore properties once and reuse that
configuration throughout the application, "
++ "but only allows a restricted set of SSL protocols to be chosen.
The set of protocols selectable will "
++ "evolve over time as new protocols emerge and older protocols
are deprecated. This service is recommended "
++ "over StandardSSLContextService if a component doesn't expect to
communicate with legacy systems since it's "
++ "unlikely that legacy systems will support these protocols.")
+public class StandardRestrictedSSLContextService extends
StandardSSLContextService implements RestrictedSSLContextService {
+
+public static final PropertyDescriptor RESTRICTED_SSL_ALGORITHM = new
PropertyDescriptor.Builder()
+.name("SSL Protocol")
+.defaultValue("TLSv1.2")
--- End diff --
Gotcha. I can make this change.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-2528) Update ListenHTTP to honor SSLContextService Protocols
[
https://issues.apache.org/jira/browse/NIFI-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145942#comment-16145942
]
ASF GitHub Bot commented on NIFI-2528:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135886286
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/StandardRestrictedSSLContextService.java
---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+
+/**
+ * This class is functionally the same as {@link
StandardSSLContextService}, but it restricts the allowable
+ * values that can be selected for SSL protocols.
+ */
+@Tags({"ssl", "secure", "certificate", "keystore", "truststore", "jks",
"p12", "pkcs12", "pkcs"})
+@CapabilityDescription("Restricted implementation of the
SSLContextService. Provides the ability to configure "
++ "keystore and/or truststore properties once and reuse that
configuration throughout the application, "
++ "but only allows a restricted set of SSL protocols to be chosen.
The set of protocols selectable will "
++ "evolve over time as new protocols emerge and older protocols
are deprecated. This service is recommended "
++ "over StandardSSLContextService if a component doesn't expect to
communicate with legacy systems since it's "
++ "unlikely that legacy systems will support these protocols.")
+public class StandardRestrictedSSLContextService extends
StandardSSLContextService implements RestrictedSSLContextService {
+
+public static final PropertyDescriptor RESTRICTED_SSL_ALGORITHM = new
PropertyDescriptor.Builder()
+.name("SSL Protocol")
+.defaultValue("TLSv1.2")
--- End diff --
No, I left `.name()` the same to be backward compatible (changing the name
means that the value stored in the flow will not be retrieved on load). This is
the whole reason `.displayName()` was introduced -- it provides a human-facing
value that isn't used for value resolution.
> Update ListenHTTP to honor SSLContextService Protocols
> --
>
> Key: NIFI-2528
> URL: https://issues.apache.org/jira/browse/NIFI-2528
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.8.0, 0.7.1
>Reporter: Joe Skora
>Assignee: Michael Hogue
>
> Update ListenHTTP to honor SSLContextService Protocols as [NIFI-1688] did for
> PostHTTP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #1986: NIFI-2528: added support for SSLContextService prot...
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135886286
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/StandardRestrictedSSLContextService.java
---
@@ -0,0 +1,81 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.util.ArrayList;
+import java.util.Collections;
+import java.util.List;
+
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.processor.util.StandardValidators;
+
+/**
+ * This class is functionally the same as {@link
StandardSSLContextService}, but it restricts the allowable
+ * values that can be selected for SSL protocols.
+ */
+@Tags({"ssl", "secure", "certificate", "keystore", "truststore", "jks",
"p12", "pkcs12", "pkcs"})
+@CapabilityDescription("Restricted implementation of the
SSLContextService. Provides the ability to configure "
++ "keystore and/or truststore properties once and reuse that
configuration throughout the application, "
++ "but only allows a restricted set of SSL protocols to be chosen.
The set of protocols selectable will "
++ "evolve over time as new protocols emerge and older protocols
are deprecated. This service is recommended "
++ "over StandardSSLContextService if a component doesn't expect to
communicate with legacy systems since it's "
++ "unlikely that legacy systems will support these protocols.")
+public class StandardRestrictedSSLContextService extends
StandardSSLContextService implements RestrictedSSLContextService {
+
+public static final PropertyDescriptor RESTRICTED_SSL_ALGORITHM = new
PropertyDescriptor.Builder()
+.name("SSL Protocol")
+.defaultValue("TLSv1.2")
--- End diff --
No, I left `.name()` the same to be backward compatible (changing the name
means that the value stored in the flow will not be retrieved on load). This is
the whole reason `.displayName()` was introduced -- it provides a human-facing
value that isn't used for value resolution.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-2528) Update ListenHTTP to honor SSLContextService Protocols
[
https://issues.apache.org/jira/browse/NIFI-2528?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145939#comment-16145939
]
ASF GitHub Bot commented on NIFI-2528:
--
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135885847
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/SSLContextServiceUtils.java
---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.components.AllowableValue;
+
+public class SSLContextServiceUtils {
--- End diff --
With Java 8, you can provide a `static` method implementation on an
`interface`. I'd define a static method `buildAllowableValues()` in
`SSLContextService` and implement it with the complete list, and "override"
(define a method with the same signature but it's not technically an override
in this case) in `RestrictedSSLContextService` which returns the limited list.
Am I missing something here? I haven't tried it myself but I think this
should work.
> Update ListenHTTP to honor SSLContextService Protocols
> --
>
> Key: NIFI-2528
> URL: https://issues.apache.org/jira/browse/NIFI-2528
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.8.0, 0.7.1
>Reporter: Joe Skora
>Assignee: Michael Hogue
>
> Update ListenHTTP to honor SSLContextService Protocols as [NIFI-1688] did for
> PostHTTP.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi pull request #1986: NIFI-2528: added support for SSLContextService prot...
Github user alopresto commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1986#discussion_r135885847
--- Diff:
nifi-nar-bundles/nifi-standard-services/nifi-ssl-context-bundle/nifi-ssl-context-service/src/main/java/org/apache/nifi/ssl/SSLContextServiceUtils.java
---
@@ -0,0 +1,77 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+package org.apache.nifi.ssl;
+
+import java.security.NoSuchAlgorithmException;
+import java.util.ArrayList;
+import java.util.Arrays;
+import java.util.Collections;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Set;
+
+import javax.net.ssl.SSLContext;
+
+import org.apache.nifi.components.AllowableValue;
+
+public class SSLContextServiceUtils {
--- End diff --
With Java 8, you can provide a `static` method implementation on an
`interface`. I'd define a static method `buildAllowableValues()` in
`SSLContextService` and implement it with the complete list, and "override"
(define a method with the same signature but it's not technically an override
in this case) in `RestrictedSSLContextService` which returns the limited list.
Am I missing something here? I haven't tried it myself but I think this
should work.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-3332) Bug in ListXXX causes matching timestamps to be ignored on later runs
[ https://issues.apache.org/jira/browse/NIFI-3332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145684#comment-16145684 ] ASF GitHub Bot commented on NIFI-3332: -- Github user jskora commented on the issue: https://github.com/apache/nifi/pull/1975 @ijokarumawak @bbende Sorry I haven't kept up with this recently. I'm happy to say that I was able to do some preliminary testing and this appears to solve the problem! **_Thanks!_** 磊 > Bug in ListXXX causes matching timestamps to be ignored on later runs > - > > Key: NIFI-3332 > URL: https://issues.apache.org/jira/browse/NIFI-3332 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.7.1, 1.1.1 >Reporter: Joe Skora >Assignee: Koji Kawamura >Priority: Critical > Fix For: 1.4.0 > > Attachments: listfiles.png, Test-showing-ListFile-timestamp-bug.log, > Test-showing-ListFile-timestamp-bug.patch > > > The new state implementation for the ListXXX processors based on > AbstractListProcessor creates a race conditions when processor runs occur > while a batch of files is being written with the same timestamp. > The changes to state management dropped tracking of the files processed for a > given timestamp. Without the record of files processed, the remainder of the > batch is ignored on the next processor run since their timestamp is not > greater than the one timestamp stored in processor state. With the file > tracking it was possible to process files that matched the timestamp exactly > and exclude the previously processed files. > A basic time goes as follows. > T0 - system creates or receives batch of files with Tx timestamp where Tx > is more than the current timestamp in processor state. > T1 - system writes 1st half of Tx batch to the ListFile source directory. > T2 - ListFile runs picking up 1st half of Tx batch and stores Tx timestamp > in processor state. > T3 - system writes 2nd half of Tx batch to ListFile source directory. > T4 - ListFile runs ignoring any files with T <= Tx, eliminating 2nd half Tx > timestamp batch. > I've attached a patch[1] for TestListFile.java that adds an instrumented unit > test demonstrates the problem and a log[2] of the output from one such run. > The test writes 3 files each in two batches with processor runs after each > batch. Batch 2 writes files with timestamps older than, equal to, and newer > than the timestamp stored when batch 1 was processed, but only the newer file > is picked up. The older file is correctly ignored but file with the matchin > timestamp file should have been processed. > [1] Test-showing-ListFile-timestamp-bug.patch > [2] Test-showing-ListFile-timestamp-bug.log -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[GitHub] nifi issue #1975: NIFI-3332: ListXXX to not miss files with the latest proce...
Github user jskora commented on the issue: https://github.com/apache/nifi/pull/1975 @ijokarumawak @bbende Sorry I haven't kept up with this recently. I'm happy to say that I was able to do some preliminary testing and this appears to solve the problem! **_Thanks!_** ð¥ --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #2063: NIFI-4269 Added the ability to serialize Mongo documents t...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/2063 Will take a look this week, working towards a hard deadline at the moment :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-4269) GetMongo should be able to provide pure JSON as well as "extended JSON"
[
https://issues.apache.org/jira/browse/NIFI-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145666#comment-16145666
]
ASF GitHub Bot commented on NIFI-4269:
--
Github user mattyb149 commented on the issue:
https://github.com/apache/nifi/pull/2063
Will take a look this week, working towards a hard deadline at the moment :)
> GetMongo should be able to provide pure JSON as well as "extended JSON"
> ---
>
> Key: NIFI-4269
> URL: https://issues.apache.org/jira/browse/NIFI-4269
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Mike Thomsen
>
> GetMongo currently only supports "extended JSON." That results in ugly JSON
> like this:
> {
> "timestamp": {
> "$date": LONG_TIMESTAMP
> }
> }
> instead of something cleaner like this:
> {
> "timestamp": "2017-08-07T00:00:00Z"
> }
> GetMongo should be refactored to support both modes of JSON serialization.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[jira] [Commented] (NIFI-4318) Processor cannot be stopped when Kerberos authentication default to prompt
[
https://issues.apache.org/jira/browse/NIFI-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145337#comment-16145337
]
Matt Burgess commented on NIFI-4318:
If you have a JAAS config you can try setting doNotPrompt to true:
http://www.docjar.com/docs/api/com/sun/security/auth/module/Krb5LoginModule.html
> Processor cannot be stopped when Kerberos authentication default to prompt
> --
>
> Key: NIFI-4318
> URL: https://issues.apache.org/jira/browse/NIFI-4318
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.3.0
> Environment: 3-nodes cluster
>Reporter: Pierre Villard
> Attachments: image001.png, thread-2.txt, thread.txt
>
>
> I was unable to stop a PutHiveQL processor (it was showing running threads
> and remained in this state at least half an hour). I had to restart NiFi to
> solve the situation. It looks like the Kerberos authentication mechanism is
> falling back to manual user input and wait for some input (see below
> promptForName):
> {noformat}
> "Timer-Driven Process Thread-2" Id=139 RUNNABLE (in native code)
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:255)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> - waiting on java.io.BufferedInputStream@2e2d3f92
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> - waiting on java.io.InputStreamReader@64628fdf
> at java.io.InputStreamReader.read(InputStreamReader.java:184)
> at java.io.BufferedReader.fill(BufferedReader.java:161)
> at java.io.BufferedReader.readLine(BufferedReader.java:324)
> - waiting on java.io.InputStreamReader@64628fdf
> at java.io.BufferedReader.readLine(BufferedReader.java:389)
> at
> com.sun.security.auth.callback.TextCallbackHandler.readLine(TextCallbackHandler.java:153)
> at
> com.sun.security.auth.callback.TextCallbackHandler.handle(TextCallbackHandler.java:120)
> at
> com.sun.security.auth.module.Krb5LoginModule.promptForName(Krb5LoginModule.java:858)
> at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:704)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.GeneratedMethodAccessor597.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at sun.security.jgss.GSSUtil.login(GSSUtil.java:258)
> at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
> at
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
> at
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
> at
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> at
> org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:183)
> at
> org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:151)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
>
[jira] [Commented] (NIFI-1490) Add multipart request support to ListenHTTP Processor
[ https://issues.apache.org/jira/browse/NIFI-1490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145310#comment-16145310 ] Michael Moser commented on NIFI-1490: - +1 to the separate processor to handle multipart/form-data, so that both ListenHTTP and HandleHttpRequest (see NIFI-3469) can be used to receive data. > Add multipart request support to ListenHTTP Processor > - > > Key: NIFI-1490 > URL: https://issues.apache.org/jira/browse/NIFI-1490 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andrew Serff > > The current ListenHTTP processor does not seem to support multipart requests > that are encoded with multipart/form-data. When a multipart request is > received, the ListenHTTPServlet just copies the Request InputStream to the > FlowFiles content which leaves the form encoding wrapper in the content and > in turn makes the file invalid. > Specifically, we want to be able to support file uploads in a multipart > request. > See this thread in the mailing list for more info: > http://mail-archives.apache.org/mod_mbox/nifi-users/201602.mbox/%3C6DE9CEEF-2A37-480F-8D3C-5028C590FD9E%40acesinc.net%3E -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Comment Edited] (NIFI-3248) GetSolr can miss recently updated documents
[
https://issues.apache.org/jira/browse/NIFI-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145306#comment-16145306
]
Johannes Peter edited comment on NIFI-3248 at 8/29/17 1:46 PM:
---
Have you considered to use the Solr field \_version\_ yet? It can be treated
like a timestamp. It also can be transformed to a timestamp. E. g. sorting for
"\_version\_ desc" sorts documents depending on their time of indexing.
was (Author: jope):
Have you considered to use the Solr field \_version\_ yet? It can be treated
like a timestamp. It also can be transformed to a timestamp. E. g. sorting for
"_version_ desc" sorts documents depending on their time of indexing.
> GetSolr can miss recently updated documents
> ---
>
> Key: NIFI-3248
> URL: https://issues.apache.org/jira/browse/NIFI-3248
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0, 0.5.0, 0.6.0, 0.5.1, 0.7.0, 0.6.1, 1.1.0, 0.7.1,
> 1.0.1
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Attachments: nifi-flow.png, query-result-with-curly-bracket.png,
> query-result-with-square-bracket.png
>
>
> GetSolr holds the last query timestamp so that it only fetches documents
> those have been added or updated since the last query.
> However, GetSolr misses some of those updated documents, and once the
> documents date field value becomes older than last query timestamp, the
> document won't be able to be queried by GetSolr any more.
> This JIRA is for tracking the process of investigating this behavior, and
> discussion on them.
> Here are things that can be a cause of this behavior:
> |#|Short description|Should we address it?|
> |1|Timestamp range filter, curly or square bracket?|No|
> |2|Timezone difference between update and query|Additional docs might be
> helpful|
> |3|Lag comes from NearRealTIme nature of Solr|Should be documented at least,
> add 'commit lag-time'?|
> h2. 1. Timestamp range filter, curly or square bracket?
> At the first glance, using curly and square bracket in mix looked strange
> ([source
> code|https://github.com/apache/nifi/blob/support/nifi-0.5.x/nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/GetSolr.java#L202]).
> But these difference has a meaning.
> The square bracket on the range query is inclusive and the curly bracket is
> exclusive. If we use inclusive on both sides and a document has a time stamp
> exactly on the boundary then it could be returned in two consecutive
> executions, and we only want it in one.
> This is intentional, and it should be as it is.
> h2. 2. Timezone difference between update and query
> Solr treats date fields as [UTC
> representation|https://cwiki.apache.org/confluence/display/solr/Working+with+Dates|].
> If date field String value of an updated document represents time without
> timezone, and NiFi is running on an environment using timezone other than
> UTC, GetSolr can't perform date range query as users expect.
> Let's say NiFi is running with JST(UTC+9). A process added a document to Solr
> at 15:00 JST. But the date field doesn't have timezone. So, Solr indexed it
> as 15:00 UTC. Then GetSolr performs range query at 15:10 JST, targeting any
> documents updated from 15:00 to 15:10 JST. GetSolr formatted dates using UTC,
> i.e. 6:00 to 6:10 UTC. The updated document won't be matched with the date
> range filter.
> To avoid this, updated documents must have proper timezone in date field
> string representation.
> If one uses NiFi expression language to set current timestamp to that date
> field, following NiFi expression can be used:
> {code}
> ${now():format("-MM-dd'T'HH:mm:ss.SSSZ")}
> {code}
> It will produce a result like:
> {code}
> 2016-12-27T15:30:04.895+0900
> {code}
> Then it will be indexed in Solr with UTC and will be queried by GetSolr as
> expected.
> h2. 3. Lag comes from NearRealTIme nature of Solr
> Solr provides Near Real Time search capability, that means, the recently
> updated documents can be queried in Near Real Time, but it's not real time.
> This latency can be controlled by either on client side which requests the
> update operation by specifying "commitWithin" parameter, or on the Solr
> server side, "autoCommit" and "autoSoftCommit" in
> [solrconfig.xml|https://cwiki.apache.org/confluence/display/solr/UpdateHandlers+in+SolrConfig#UpdateHandlersinSolrConfig-Commits].
> Since commit and updating index can be costly, it's recommended to set this
> interval long enough up to the maximum tolerable latency.
> However, this can be problematic with GetSolr. For instance, as shown in the
> simple NiFi flow below, GetSolr can miss updated documents:
> {code}
> t1: GetSolr queried
> t2: GenerateFlowFile set date = t2
[jira] [Comment Edited] (NIFI-3248) GetSolr can miss recently updated documents
[
https://issues.apache.org/jira/browse/NIFI-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145306#comment-16145306
]
Johannes Peter edited comment on NIFI-3248 at 8/29/17 1:46 PM:
---
Have you considered to use the Solr field \_version\_ yet? It can be treated
like a timestamp. It also can be transformed to a timestamp. E. g. sorting for
"_version_ desc" sorts documents depending on their time of indexing.
was (Author: jope):
Have you considered to use the Solr field "__version__" yet? It can be treated
like a timestamp. It also can be transformed to a timestamp. E. g. sorting for
"_version_ desc" sorts documents depending on their time of indexing.
> GetSolr can miss recently updated documents
> ---
>
> Key: NIFI-3248
> URL: https://issues.apache.org/jira/browse/NIFI-3248
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0, 0.5.0, 0.6.0, 0.5.1, 0.7.0, 0.6.1, 1.1.0, 0.7.1,
> 1.0.1
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Attachments: nifi-flow.png, query-result-with-curly-bracket.png,
> query-result-with-square-bracket.png
>
>
> GetSolr holds the last query timestamp so that it only fetches documents
> those have been added or updated since the last query.
> However, GetSolr misses some of those updated documents, and once the
> documents date field value becomes older than last query timestamp, the
> document won't be able to be queried by GetSolr any more.
> This JIRA is for tracking the process of investigating this behavior, and
> discussion on them.
> Here are things that can be a cause of this behavior:
> |#|Short description|Should we address it?|
> |1|Timestamp range filter, curly or square bracket?|No|
> |2|Timezone difference between update and query|Additional docs might be
> helpful|
> |3|Lag comes from NearRealTIme nature of Solr|Should be documented at least,
> add 'commit lag-time'?|
> h2. 1. Timestamp range filter, curly or square bracket?
> At the first glance, using curly and square bracket in mix looked strange
> ([source
> code|https://github.com/apache/nifi/blob/support/nifi-0.5.x/nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/GetSolr.java#L202]).
> But these difference has a meaning.
> The square bracket on the range query is inclusive and the curly bracket is
> exclusive. If we use inclusive on both sides and a document has a time stamp
> exactly on the boundary then it could be returned in two consecutive
> executions, and we only want it in one.
> This is intentional, and it should be as it is.
> h2. 2. Timezone difference between update and query
> Solr treats date fields as [UTC
> representation|https://cwiki.apache.org/confluence/display/solr/Working+with+Dates|].
> If date field String value of an updated document represents time without
> timezone, and NiFi is running on an environment using timezone other than
> UTC, GetSolr can't perform date range query as users expect.
> Let's say NiFi is running with JST(UTC+9). A process added a document to Solr
> at 15:00 JST. But the date field doesn't have timezone. So, Solr indexed it
> as 15:00 UTC. Then GetSolr performs range query at 15:10 JST, targeting any
> documents updated from 15:00 to 15:10 JST. GetSolr formatted dates using UTC,
> i.e. 6:00 to 6:10 UTC. The updated document won't be matched with the date
> range filter.
> To avoid this, updated documents must have proper timezone in date field
> string representation.
> If one uses NiFi expression language to set current timestamp to that date
> field, following NiFi expression can be used:
> {code}
> ${now():format("-MM-dd'T'HH:mm:ss.SSSZ")}
> {code}
> It will produce a result like:
> {code}
> 2016-12-27T15:30:04.895+0900
> {code}
> Then it will be indexed in Solr with UTC and will be queried by GetSolr as
> expected.
> h2. 3. Lag comes from NearRealTIme nature of Solr
> Solr provides Near Real Time search capability, that means, the recently
> updated documents can be queried in Near Real Time, but it's not real time.
> This latency can be controlled by either on client side which requests the
> update operation by specifying "commitWithin" parameter, or on the Solr
> server side, "autoCommit" and "autoSoftCommit" in
> [solrconfig.xml|https://cwiki.apache.org/confluence/display/solr/UpdateHandlers+in+SolrConfig#UpdateHandlersinSolrConfig-Commits].
> Since commit and updating index can be costly, it's recommended to set this
> interval long enough up to the maximum tolerable latency.
> However, this can be problematic with GetSolr. For instance, as shown in the
> simple NiFi flow below, GetSolr can miss updated documents:
> {code}
> t1: GetSolr queried
> t2: GenerateFlowFile set date = t2
[jira] [Comment Edited] (NIFI-3248) GetSolr can miss recently updated documents
[
https://issues.apache.org/jira/browse/NIFI-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145306#comment-16145306
]
Johannes Peter edited comment on NIFI-3248 at 8/29/17 1:45 PM:
---
Have you considered to use the Solr field "__version__" yet? It can be treated
like a timestamp. It also can be transformed to a timestamp. E. g. sorting for
"_version_ desc" sorts documents depending on their time of indexing.
was (Author: jope):
Have you considered to use the Solr field "_version_" yet? It can be treated
like a timestamp. It also can be transformed to a timestamp. E. g. sorting for
"_version_ desc" sorts documents depending on their time of indexing.
> GetSolr can miss recently updated documents
> ---
>
> Key: NIFI-3248
> URL: https://issues.apache.org/jira/browse/NIFI-3248
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0, 0.5.0, 0.6.0, 0.5.1, 0.7.0, 0.6.1, 1.1.0, 0.7.1,
> 1.0.1
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Attachments: nifi-flow.png, query-result-with-curly-bracket.png,
> query-result-with-square-bracket.png
>
>
> GetSolr holds the last query timestamp so that it only fetches documents
> those have been added or updated since the last query.
> However, GetSolr misses some of those updated documents, and once the
> documents date field value becomes older than last query timestamp, the
> document won't be able to be queried by GetSolr any more.
> This JIRA is for tracking the process of investigating this behavior, and
> discussion on them.
> Here are things that can be a cause of this behavior:
> |#|Short description|Should we address it?|
> |1|Timestamp range filter, curly or square bracket?|No|
> |2|Timezone difference between update and query|Additional docs might be
> helpful|
> |3|Lag comes from NearRealTIme nature of Solr|Should be documented at least,
> add 'commit lag-time'?|
> h2. 1. Timestamp range filter, curly or square bracket?
> At the first glance, using curly and square bracket in mix looked strange
> ([source
> code|https://github.com/apache/nifi/blob/support/nifi-0.5.x/nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/GetSolr.java#L202]).
> But these difference has a meaning.
> The square bracket on the range query is inclusive and the curly bracket is
> exclusive. If we use inclusive on both sides and a document has a time stamp
> exactly on the boundary then it could be returned in two consecutive
> executions, and we only want it in one.
> This is intentional, and it should be as it is.
> h2. 2. Timezone difference between update and query
> Solr treats date fields as [UTC
> representation|https://cwiki.apache.org/confluence/display/solr/Working+with+Dates|].
> If date field String value of an updated document represents time without
> timezone, and NiFi is running on an environment using timezone other than
> UTC, GetSolr can't perform date range query as users expect.
> Let's say NiFi is running with JST(UTC+9). A process added a document to Solr
> at 15:00 JST. But the date field doesn't have timezone. So, Solr indexed it
> as 15:00 UTC. Then GetSolr performs range query at 15:10 JST, targeting any
> documents updated from 15:00 to 15:10 JST. GetSolr formatted dates using UTC,
> i.e. 6:00 to 6:10 UTC. The updated document won't be matched with the date
> range filter.
> To avoid this, updated documents must have proper timezone in date field
> string representation.
> If one uses NiFi expression language to set current timestamp to that date
> field, following NiFi expression can be used:
> {code}
> ${now():format("-MM-dd'T'HH:mm:ss.SSSZ")}
> {code}
> It will produce a result like:
> {code}
> 2016-12-27T15:30:04.895+0900
> {code}
> Then it will be indexed in Solr with UTC and will be queried by GetSolr as
> expected.
> h2. 3. Lag comes from NearRealTIme nature of Solr
> Solr provides Near Real Time search capability, that means, the recently
> updated documents can be queried in Near Real Time, but it's not real time.
> This latency can be controlled by either on client side which requests the
> update operation by specifying "commitWithin" parameter, or on the Solr
> server side, "autoCommit" and "autoSoftCommit" in
> [solrconfig.xml|https://cwiki.apache.org/confluence/display/solr/UpdateHandlers+in+SolrConfig#UpdateHandlersinSolrConfig-Commits].
> Since commit and updating index can be costly, it's recommended to set this
> interval long enough up to the maximum tolerable latency.
> However, this can be problematic with GetSolr. For instance, as shown in the
> simple NiFi flow below, GetSolr can miss updated documents:
> {code}
> t1: GetSolr queried
> t2: GenerateFlowFile set date = t2
[jira] [Commented] (NIFI-3248) GetSolr can miss recently updated documents
[
https://issues.apache.org/jira/browse/NIFI-3248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145306#comment-16145306
]
Johannes Peter commented on NIFI-3248:
--
Have you considered to use the Solr field "_version_" yet? It can be treated
like a timestamp. It also can be transformed to a timestamp. E. g. sorting for
"_version_ desc" sorts documents depending on their time of indexing.
> GetSolr can miss recently updated documents
> ---
>
> Key: NIFI-3248
> URL: https://issues.apache.org/jira/browse/NIFI-3248
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0, 0.5.0, 0.6.0, 0.5.1, 0.7.0, 0.6.1, 1.1.0, 0.7.1,
> 1.0.1
>Reporter: Koji Kawamura
>Assignee: Koji Kawamura
> Attachments: nifi-flow.png, query-result-with-curly-bracket.png,
> query-result-with-square-bracket.png
>
>
> GetSolr holds the last query timestamp so that it only fetches documents
> those have been added or updated since the last query.
> However, GetSolr misses some of those updated documents, and once the
> documents date field value becomes older than last query timestamp, the
> document won't be able to be queried by GetSolr any more.
> This JIRA is for tracking the process of investigating this behavior, and
> discussion on them.
> Here are things that can be a cause of this behavior:
> |#|Short description|Should we address it?|
> |1|Timestamp range filter, curly or square bracket?|No|
> |2|Timezone difference between update and query|Additional docs might be
> helpful|
> |3|Lag comes from NearRealTIme nature of Solr|Should be documented at least,
> add 'commit lag-time'?|
> h2. 1. Timestamp range filter, curly or square bracket?
> At the first glance, using curly and square bracket in mix looked strange
> ([source
> code|https://github.com/apache/nifi/blob/support/nifi-0.5.x/nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/GetSolr.java#L202]).
> But these difference has a meaning.
> The square bracket on the range query is inclusive and the curly bracket is
> exclusive. If we use inclusive on both sides and a document has a time stamp
> exactly on the boundary then it could be returned in two consecutive
> executions, and we only want it in one.
> This is intentional, and it should be as it is.
> h2. 2. Timezone difference between update and query
> Solr treats date fields as [UTC
> representation|https://cwiki.apache.org/confluence/display/solr/Working+with+Dates|].
> If date field String value of an updated document represents time without
> timezone, and NiFi is running on an environment using timezone other than
> UTC, GetSolr can't perform date range query as users expect.
> Let's say NiFi is running with JST(UTC+9). A process added a document to Solr
> at 15:00 JST. But the date field doesn't have timezone. So, Solr indexed it
> as 15:00 UTC. Then GetSolr performs range query at 15:10 JST, targeting any
> documents updated from 15:00 to 15:10 JST. GetSolr formatted dates using UTC,
> i.e. 6:00 to 6:10 UTC. The updated document won't be matched with the date
> range filter.
> To avoid this, updated documents must have proper timezone in date field
> string representation.
> If one uses NiFi expression language to set current timestamp to that date
> field, following NiFi expression can be used:
> {code}
> ${now():format("-MM-dd'T'HH:mm:ss.SSSZ")}
> {code}
> It will produce a result like:
> {code}
> 2016-12-27T15:30:04.895+0900
> {code}
> Then it will be indexed in Solr with UTC and will be queried by GetSolr as
> expected.
> h2. 3. Lag comes from NearRealTIme nature of Solr
> Solr provides Near Real Time search capability, that means, the recently
> updated documents can be queried in Near Real Time, but it's not real time.
> This latency can be controlled by either on client side which requests the
> update operation by specifying "commitWithin" parameter, or on the Solr
> server side, "autoCommit" and "autoSoftCommit" in
> [solrconfig.xml|https://cwiki.apache.org/confluence/display/solr/UpdateHandlers+in+SolrConfig#UpdateHandlersinSolrConfig-Commits].
> Since commit and updating index can be costly, it's recommended to set this
> interval long enough up to the maximum tolerable latency.
> However, this can be problematic with GetSolr. For instance, as shown in the
> simple NiFi flow below, GetSolr can miss updated documents:
> {code}
> t1: GetSolr queried
> t2: GenerateFlowFile set date = t2
> t3: PutSolrContentStream stored new doc
> t4: GetSolr queried again, from t1 to t4, but the new doc hasn't been indexed
> t5: Solr completed index
> t6: GetSolr queried again, from t4 to t6, the doc didn't match query
> {code}
> This behavior should be at least documented.
> Plus, it would be
[jira] [Updated] (NIFI-4329) ParseEVTX fails on files containing ForwardedEvents
[ https://issues.apache.org/jira/browse/NIFI-4329?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Colin Dix updated NIFI-4329: Description: When processing EVTX files from the Windows ForwardedEvents channel, the ParseEVTX processor fails with "Bad Chunk". EVTX files from other "normal" channels parse correctly. Most recently tested with 1.2.0.3.0.1.0-43. was:When processing EVTX files from the Windows ForwardedEvents channel, the ParseEVTX processor fails with "Bad Chunk". EVTX files from other "normal" channels parse correctly. > ParseEVTX fails on files containing ForwardedEvents > --- > > Key: NIFI-4329 > URL: https://issues.apache.org/jira/browse/NIFI-4329 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.0, 1.2.0 > Environment: Windows Server 2012, Redhat Linux 6 >Reporter: Colin Dix > Labels: newbie > > When processing EVTX files from the Windows ForwardedEvents channel, the > ParseEVTX processor fails with "Bad Chunk". EVTX files from other "normal" > channels parse correctly. > Most recently tested with 1.2.0.3.0.1.0-43. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4330) ConsumeKafka* throw NullPointerException if Kafka message has a null value
Mark Payne created NIFI-4330: Summary: ConsumeKafka* throw NullPointerException if Kafka message has a null value Key: NIFI-4330 URL: https://issues.apache.org/jira/browse/NIFI-4330 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Created] (NIFI-4329) ParseEVTX fails on files containing ForwardedEvents
Colin Dix created NIFI-4329: --- Summary: ParseEVTX fails on files containing ForwardedEvents Key: NIFI-4329 URL: https://issues.apache.org/jira/browse/NIFI-4329 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.2.0, 1.1.0 Environment: Windows Server 2012, Redhat Linux 6 Reporter: Colin Dix When processing EVTX files from the Windows ForwardedEvents channel, the ParseEVTX processor fails with "Bad Chunk". EVTX files from other "normal" channels parse correctly. -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-4318) Processor cannot be stopped when Kerberos authentication default to prompt
[
https://issues.apache.org/jira/browse/NIFI-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145179#comment-16145179
]
Pierre Villard commented on NIFI-4318:
--
Facing the same issue today with a ListHDFS processor. I updated the JIRA to
reflect the more general issue.
> Processor cannot be stopped when Kerberos authentication default to prompt
> --
>
> Key: NIFI-4318
> URL: https://issues.apache.org/jira/browse/NIFI-4318
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.3.0
> Environment: 3-nodes cluster
>Reporter: Pierre Villard
> Attachments: image001.png, thread-2.txt, thread.txt
>
>
> I was unable to stop a PutHiveQL processor (it was showing running threads
> and remained in this state at least half an hour). I had to restart NiFi to
> solve the situation. It looks like the Kerberos authentication mechanism is
> falling back to manual user input and wait for some input (see below
> promptForName):
> {noformat}
> "Timer-Driven Process Thread-2" Id=139 RUNNABLE (in native code)
> at java.io.FileInputStream.readBytes(Native Method)
> at java.io.FileInputStream.read(FileInputStream.java:255)
> at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
> at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
> - waiting on java.io.BufferedInputStream@2e2d3f92
> at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
> at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
> at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
> - waiting on java.io.InputStreamReader@64628fdf
> at java.io.InputStreamReader.read(InputStreamReader.java:184)
> at java.io.BufferedReader.fill(BufferedReader.java:161)
> at java.io.BufferedReader.readLine(BufferedReader.java:324)
> - waiting on java.io.InputStreamReader@64628fdf
> at java.io.BufferedReader.readLine(BufferedReader.java:389)
> at
> com.sun.security.auth.callback.TextCallbackHandler.readLine(TextCallbackHandler.java:153)
> at
> com.sun.security.auth.callback.TextCallbackHandler.handle(TextCallbackHandler.java:120)
> at
> com.sun.security.auth.module.Krb5LoginModule.promptForName(Krb5LoginModule.java:858)
> at
> com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:704)
> at
> com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
> at sun.reflect.GeneratedMethodAccessor597.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
> at
> javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
> at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
> at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
> at sun.security.jgss.GSSUtil.login(GSSUtil.java:258)
> at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
> at
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
> at
> sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
> at java.security.AccessController.doPrivileged(Native Method)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
> at
> sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
> at
> sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
> at
> sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
> at
> sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
> at
> org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:183)
> at
> org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:151)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
> at
[jira] [Updated] (NIFI-4318) Processor cannot be stopped when Kerberos authentication default to prompt
[
https://issues.apache.org/jira/browse/NIFI-4318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-4318:
-
Description:
I was unable to stop a PutHiveQL processor (it was showing running threads and
remained in this state at least half an hour). I had to restart NiFi to solve
the situation. It looks like the Kerberos authentication mechanism is falling
back to manual user input and wait for some input (see below promptForName):
{noformat}
"Timer-Driven Process Thread-2" Id=139 RUNNABLE (in native code)
at java.io.FileInputStream.readBytes(Native Method)
at java.io.FileInputStream.read(FileInputStream.java:255)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:284)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
- waiting on java.io.BufferedInputStream@2e2d3f92
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
- waiting on java.io.InputStreamReader@64628fdf
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.readLine(BufferedReader.java:324)
- waiting on java.io.InputStreamReader@64628fdf
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at
com.sun.security.auth.callback.TextCallbackHandler.readLine(TextCallbackHandler.java:153)
at
com.sun.security.auth.callback.TextCallbackHandler.handle(TextCallbackHandler.java:120)
at
com.sun.security.auth.module.Krb5LoginModule.promptForName(Krb5LoginModule.java:858)
at
com.sun.security.auth.module.Krb5LoginModule.attemptAuthentication(Krb5LoginModule.java:704)
at
com.sun.security.auth.module.Krb5LoginModule.login(Krb5LoginModule.java:617)
at sun.reflect.GeneratedMethodAccessor597.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at javax.security.auth.login.LoginContext.invoke(LoginContext.java:755)
at
javax.security.auth.login.LoginContext.access$000(LoginContext.java:195)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:682)
at javax.security.auth.login.LoginContext$4.run(LoginContext.java:680)
at java.security.AccessController.doPrivileged(Native Method)
at
javax.security.auth.login.LoginContext.invokePriv(LoginContext.java:680)
at javax.security.auth.login.LoginContext.login(LoginContext.java:587)
at sun.security.jgss.GSSUtil.login(GSSUtil.java:258)
at sun.security.jgss.krb5.Krb5Util.getTicket(Krb5Util.java:158)
at
sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:335)
at
sun.security.jgss.krb5.Krb5InitCredential$1.run(Krb5InitCredential.java:331)
at java.security.AccessController.doPrivileged(Native Method)
at
sun.security.jgss.krb5.Krb5InitCredential.getTgt(Krb5InitCredential.java:330)
at
sun.security.jgss.krb5.Krb5InitCredential.getInstance(Krb5InitCredential.java:145)
at
sun.security.jgss.krb5.Krb5MechFactory.getCredentialElement(Krb5MechFactory.java:122)
at
sun.security.jgss.krb5.Krb5MechFactory.getMechanismContext(Krb5MechFactory.java:187)
at
sun.security.jgss.GSSManagerImpl.getMechanismContext(GSSManagerImpl.java:224)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:212)
at
sun.security.jgss.GSSContextImpl.initSecContext(GSSContextImpl.java:179)
at
org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:183)
at
org.apache.hive.service.auth.HttpAuthUtils$HttpKerberosClientAction.run(HttpAuthUtils.java:151)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:422)
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1866)
at
org.apache.hive.service.auth.HttpAuthUtils.getKerberosServiceTicket(HttpAuthUtils.java:83)
at
org.apache.hive.jdbc.HttpKerberosRequestInterceptor.addHttpAuthHeader(HttpKerberosRequestInterceptor.java:62)
at
org.apache.hive.jdbc.HttpRequestInterceptorBase.process(HttpRequestInterceptorBase.java:74)
at
org.apache.http.protocol.ImmutableHttpProcessor.process(ImmutableHttpProcessor.java:132)
at
org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:183)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at
org.apache.http.impl.execchain.ServiceUnavailableRetryExec.execute(ServiceUnavailableRetryExec.java:85)
at
[jira] [Commented] (NIFI-4269) GetMongo should be able to provide pure JSON as well as "extended JSON"
[
https://issues.apache.org/jira/browse/NIFI-4269?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145118#comment-16145118
]
ASF GitHub Bot commented on NIFI-4269:
--
Github user MikeThomsen commented on the issue:
https://github.com/apache/nifi/pull/2063
@mattyb149 Can you take a look? Thanks.
> GetMongo should be able to provide pure JSON as well as "extended JSON"
> ---
>
> Key: NIFI-4269
> URL: https://issues.apache.org/jira/browse/NIFI-4269
> Project: Apache NiFi
> Issue Type: Improvement
>Reporter: Mike Thomsen
>
> GetMongo currently only supports "extended JSON." That results in ugly JSON
> like this:
> {
> "timestamp": {
> "$date": LONG_TIMESTAMP
> }
> }
> instead of something cleaner like this:
> {
> "timestamp": "2017-08-07T00:00:00Z"
> }
> GetMongo should be refactored to support both modes of JSON serialization.
--
This message was sent by Atlassian JIRA
(v6.4.14#64029)
[GitHub] nifi issue #2063: NIFI-4269 Added the ability to serialize Mongo documents t...
Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/2063 @mattyb149 Can you take a look? Thanks. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
