[jira] [Updated] (NIFI-8628) Variable Registry - Variable count doesn't reset when opening the variable dialog
[ https://issues.apache.org/jira/browse/NIFI-8628?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hsin-Ying Lee updated NIFI-8628: Status: Patch Available (was: In Progress) > Variable Registry - Variable count doesn't reset when opening the variable > dialog > - > > Key: NIFI-8628 > URL: https://issues.apache.org/jira/browse/NIFI-8628 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Hsin-Ying Lee >Assignee: Hsin-Ying Lee >Priority: Minor > Time Spent: 10m > Remaining Estimate: 0h > > `variablesCount` doesn't reset when opening the variable dialogs, > `variablesCount` will grow each time. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[
https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351419#comment-17351419
]
KevinSky edited comment on NIFI-8625 at 5/26/21, 1:35 AM:
--
hi [#Otto Fowler], you can refer that :
https://issues.apache.org/jira/browse/NIFI-8461?focusedCommentId=17349069=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17349069
Here is a better solution:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java
{code:java}
package org.apache.nifi.script.impl;import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.processors.script.ScriptEngineConfigurator;
import org.python.core.PyString;import javax.script.Compilable;
import javax.script.CompiledScript;
import javax.script.ScriptContext;
import javax.script.ScriptEngine;
import javax.script.ScriptException;
import java.net.URL;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicReference;
import java.util.stream.Collectors;/**
* A helper class to configure the Jython engine with any specific requirements
*/
public class JythonScriptEngineConfigurator implements ScriptEngineConfigurator
{private final Map compiledScriptMap = new
ConcurrentHashMap<>();private String prefix = null;@Override
public String getScriptEngineName() {
return "python";
}@Override
public URL[] getModuleURLsForClasspath(String[] modulePaths, ComponentLog
log) {
// We don't need to add the module paths to the classpath, they will be
added via sys.path.append
return new URL[0];
}@Override
public Object init(ScriptEngine engine, String scriptBody, String[]
modulePaths) throws ScriptException {
if (engine == null) {
return null;
}CompiledScript compiled = compiledScriptMap.get(engine);
if (compiled != null) {
return compiled;
}// Add prefix for import sys and all jython modules
if (prefix == null) {
prefix = "import sys\n"
+ Arrays.stream(modulePaths).map((modulePath) ->
"sys.path.append(" + PyString.encode_UnicodeEscape(modulePath, true) + ")")
.collect(Collectors.joining("\n"));
}compiled = ((Compilable) engine).compile(prefix + scriptBody);
compiledScriptMap.put(engine, compiled);
return compiled;
}@Override
public Object eval(ScriptEngine engine, String scriptBody, String[]
modulePaths) throws ScriptException {
if (engine == null) {
return null;
}CompiledScript compiled = compiledScriptMap.get(engine);
if (compiled == null) {
throw new ScriptException("Jython script has not been compiled, the
processor must be restarted.");
}return
compiled.eval(engine.getBindings(ScriptContext.ENGINE_SCOPE));
}
}{code}
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java
{code:java}
public void stop() {
if (engineQ != null) {
engineQ.clear();
}
scriptEngineConfiguratorMap.clear();
}{code}
was (Author: kevinsky1093):
hi [#Otto Fowler], you can refer that :
https://issues.apache.org/jira/browse/NIFI-8461?focusedCommentId=17349069=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17349069
Here is a better solution:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java
{code:java}
package org.apache.nifi.script.impl;import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.processors.script.ScriptEngineConfigurator;
import org.python.core.PyString;import javax.script.Compilable;
import javax.script.CompiledScript;
import javax.script.ScriptContext;
import javax.script.ScriptEngine;
import javax.script.ScriptException;
import java.net.URL;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicReference;
import java.util.stream.Collectors;/**
* A helper class to configure the Jython engine with any specific requirements
*/
public class JythonScriptEngineConfigurator implements ScriptEngineConfigurator
{private final Map compiledScriptMap = new
ConcurrentHashMap<>();private String prefix = null;@Override
public String getScriptEngineName() {
return "python";
}@Override
public URL[] getModuleURLsForClasspath(String[] modulePaths, ComponentLog
log) {
// We don't need to add the module paths to the classpath, they will be
added via sys.path.append
return new URL[0];
}
[jira] [Comment Edited] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[
https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351419#comment-17351419
]
KevinSky edited comment on NIFI-8625 at 5/26/21, 1:34 AM:
--
hi [#Otto Fowler], you can refer that :
https://issues.apache.org/jira/browse/NIFI-8461?focusedCommentId=17349069=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17349069
Here is a better solution:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java
{code:java}
package org.apache.nifi.script.impl;import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.processors.script.ScriptEngineConfigurator;
import org.python.core.PyString;import javax.script.Compilable;
import javax.script.CompiledScript;
import javax.script.ScriptContext;
import javax.script.ScriptEngine;
import javax.script.ScriptException;
import java.net.URL;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicReference;
import java.util.stream.Collectors;/**
* A helper class to configure the Jython engine with any specific requirements
*/
public class JythonScriptEngineConfigurator implements ScriptEngineConfigurator
{private final Map compiledScriptMap = new
ConcurrentHashMap<>();private String prefix = null;@Override
public String getScriptEngineName() {
return "python";
}@Override
public URL[] getModuleURLsForClasspath(String[] modulePaths, ComponentLog
log) {
// We don't need to add the module paths to the classpath, they will be
added via sys.path.append
return new URL[0];
}@Override
public Object init(ScriptEngine engine, String scriptBody, String[]
modulePaths) throws ScriptException {
if (engine == null) {
return null;
}CompiledScript compiled = compiledScriptMap.get(engine);
if (compiled != null) {
return compiled;
}// Add prefix for import sys and all jython modules
if (prefix == null) {
prefix = "import sys\n"
+ Arrays.stream(modulePaths).map((modulePath) ->
"sys.path.append(" + PyString.encode_UnicodeEscape(modulePath, true) + ")")
.collect(Collectors.joining("\n"));
}compiled = ((Compilable) engine).compile(prefix + scriptBody);
compiledScriptMap.put(engine, compiled);
return compiled;
}@Override
public Object eval(ScriptEngine engine, String scriptBody, String[]
modulePaths) throws ScriptException {
if (engine == null) {
return null;
}CompiledScript compiled = compiledScriptMap.get(engine);
if (compiled == null) {
throw new ScriptException("Jython script has not been compiled, the
processor must be restarted.");
}return
compiled.eval(engine.getBindings(ScriptContext.ENGINE_SCOPE));
}
}{code}
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java
{code:java}
public void stop() {
if (engineQ != null) {
engineQ.clear();
}scriptEngineConfiguratorMap.clear();
}{code}
was (Author: kevinsky1093):
hi [#Otto Fowler],
Here is a better solution:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java
{code:java}
package org.apache.nifi.script.impl;import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.processors.script.ScriptEngineConfigurator;
import org.python.core.PyString;import javax.script.Compilable;
import javax.script.CompiledScript;
import javax.script.ScriptContext;
import javax.script.ScriptEngine;
import javax.script.ScriptException;
import java.net.URL;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicReference;
import java.util.stream.Collectors;/**
* A helper class to configure the Jython engine with any specific requirements
*/
public class JythonScriptEngineConfigurator implements ScriptEngineConfigurator
{private final Map compiledScriptMap = new
ConcurrentHashMap<>();private String prefix = null;@Override
public String getScriptEngineName() {
return "python";
}@Override
public URL[] getModuleURLsForClasspath(String[] modulePaths, ComponentLog
log) {
// We don't need to add the module paths to the classpath, they will be
added via sys.path.append
return new URL[0];
}@Override
public Object init(ScriptEngine engine, String scriptBody, String[]
modulePaths) throws ScriptException {
if (engine == null) {
return null;
}
[jira] [Comment Edited] (NIFI-8461) Queue reports items but cannot list them
[ https://issues.apache.org/jira/browse/NIFI-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351423#comment-17351423 ] KevinSky edited comment on NIFI-8461 at 5/26/21, 1:33 AM: -- hi [#Matt Bugrgess],here is a better solution: https://issues.apache.org/jira/browse/NIFI-8625?focusedCommentId=17351419=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17351419 was (Author: kevinsky1093): hi [#anchor] https://issues.apache.org/jira/browse/NIFI-8625?focusedCommentId=17351419=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17351419 > Queue reports items but cannot list them > > > Key: NIFI-8461 > URL: https://issues.apache.org/jira/browse/NIFI-8461 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 > Environment: Docker image apache/nifi :1.13.2, Windows 10 Pro, WSL 2 > (Ubuntu), x86_64 >Reporter: Kevin Aagaard >Assignee: Matt Burgess >Priority: Major > Labels: queue > Attachments: ExecuteScript_Queue_Issue.txt, NIFI-8461.zip, > image-2021-04-22-08-48-58-117.png, image-2021-04-22-08-49-40-904.png, > image-2021-04-27-14-23-07-849.png, image-2021-04-27-14-23-24-477.png, > image-2021-05-21-16-02-22-044.png > > > The following segment of workflow demonstrates the issue. The queue reports > that there are items within it, but they cannot be listed, even after > stopping the consumer (and producer). Since this is in a Docker Container, it > is unlikely an OS error. > > !image-2021-04-22-08-48-58-117.png! > > !image-2021-04-22-08-49-40-904.png! > I do not currently have a simplified example workflow to recreate the error, > but can work on it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8461) Queue reports items but cannot list them
[ https://issues.apache.org/jira/browse/NIFI-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351423#comment-17351423 ] KevinSky commented on NIFI-8461: https://issues.apache.org/jira/browse/NIFI-8625?focusedCommentId=17351419=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17351419 > Queue reports items but cannot list them > > > Key: NIFI-8461 > URL: https://issues.apache.org/jira/browse/NIFI-8461 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 > Environment: Docker image apache/nifi :1.13.2, Windows 10 Pro, WSL 2 > (Ubuntu), x86_64 >Reporter: Kevin Aagaard >Assignee: Matt Burgess >Priority: Major > Labels: queue > Attachments: ExecuteScript_Queue_Issue.txt, NIFI-8461.zip, > image-2021-04-22-08-48-58-117.png, image-2021-04-22-08-49-40-904.png, > image-2021-04-27-14-23-07-849.png, image-2021-04-27-14-23-24-477.png, > image-2021-05-21-16-02-22-044.png > > > The following segment of workflow demonstrates the issue. The queue reports > that there are items within it, but they cannot be listed, even after > stopping the consumer (and producer). Since this is in a Docker Container, it > is unlikely an OS error. > > !image-2021-04-22-08-48-58-117.png! > > !image-2021-04-22-08-49-40-904.png! > I do not currently have a simplified example workflow to recreate the error, > but can work on it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8461) Queue reports items but cannot list them
[ https://issues.apache.org/jira/browse/NIFI-8461?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351423#comment-17351423 ] KevinSky edited comment on NIFI-8461 at 5/26/21, 1:32 AM: -- hi [#anchor] https://issues.apache.org/jira/browse/NIFI-8625?focusedCommentId=17351419=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17351419 was (Author: kevinsky1093): https://issues.apache.org/jira/browse/NIFI-8625?focusedCommentId=17351419=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-17351419 > Queue reports items but cannot list them > > > Key: NIFI-8461 > URL: https://issues.apache.org/jira/browse/NIFI-8461 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 > Environment: Docker image apache/nifi :1.13.2, Windows 10 Pro, WSL 2 > (Ubuntu), x86_64 >Reporter: Kevin Aagaard >Assignee: Matt Burgess >Priority: Major > Labels: queue > Attachments: ExecuteScript_Queue_Issue.txt, NIFI-8461.zip, > image-2021-04-22-08-48-58-117.png, image-2021-04-22-08-49-40-904.png, > image-2021-04-27-14-23-07-849.png, image-2021-04-27-14-23-24-477.png, > image-2021-05-21-16-02-22-044.png > > > The following segment of workflow demonstrates the issue. The queue reports > that there are items within it, but they cannot be listed, even after > stopping the consumer (and producer). Since this is in a Docker Container, it > is unlikely an OS error. > > !image-2021-04-22-08-48-58-117.png! > > !image-2021-04-22-08-49-40-904.png! > I do not currently have a simplified example workflow to recreate the error, > but can work on it. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[
https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351419#comment-17351419
]
KevinSky edited comment on NIFI-8625 at 5/26/21, 1:30 AM:
--
hi [#Otto Fowler],
Here is a better solution:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java
{code:java}
package org.apache.nifi.script.impl;import org.apache.nifi.logging.ComponentLog;
import org.apache.nifi.processors.script.ScriptEngineConfigurator;
import org.python.core.PyString;import javax.script.Compilable;
import javax.script.CompiledScript;
import javax.script.ScriptContext;
import javax.script.ScriptEngine;
import javax.script.ScriptException;
import java.net.URL;
import java.util.Arrays;
import java.util.Map;
import java.util.concurrent.ConcurrentHashMap;
import java.util.concurrent.atomic.AtomicReference;
import java.util.stream.Collectors;/**
* A helper class to configure the Jython engine with any specific requirements
*/
public class JythonScriptEngineConfigurator implements ScriptEngineConfigurator
{private final Map compiledScriptMap = new
ConcurrentHashMap<>();private String prefix = null;@Override
public String getScriptEngineName() {
return "python";
}@Override
public URL[] getModuleURLsForClasspath(String[] modulePaths, ComponentLog
log) {
// We don't need to add the module paths to the classpath, they will be
added via sys.path.append
return new URL[0];
}@Override
public Object init(ScriptEngine engine, String scriptBody, String[]
modulePaths) throws ScriptException {
if (engine == null) {
return null;
}CompiledScript compiled = compiledScriptMap.get(engine);
if (compiled != null) {
return compiled;
}// Add prefix for import sys and all jython modules
if (prefix == null) {
prefix = "import sys\n"
+ Arrays.stream(modulePaths).map((modulePath) ->
"sys.path.append(" + PyString.encode_UnicodeEscape(modulePath, true) + ")")
.collect(Collectors.joining("\n"));
}compiled = ((Compilable) engine).compile(prefix + scriptBody);
compiledScriptMap.put(engine, compiled);
return compiled;
}@Override
public Object eval(ScriptEngine engine, String scriptBody, String[]
modulePaths) throws ScriptException {
if (engine == null) {
return null;
}CompiledScript compiled = compiledScriptMap.get(engine);
if (compiled == null) {
throw new ScriptException("Jython script has not been compiled, the
processor must be restarted.");
}return
compiled.eval(engine.getBindings(ScriptContext.ENGINE_SCOPE));
}
}{code}
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java
{code:java}
public void stop() {
if (engineQ != null) {
engineQ.clear();
}scriptEngineConfiguratorMap.clear();
}{code}
was (Author: kevinsky1093):
hi [#Otto Fowler],
Here is a better solution:
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java
!image-2021-05-26-09-22-59-199.png|width=556,height=293!
!image-2021-05-26-09-23-13-561.png|width=565,height=506!
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java
!image-2021-05-26-09-23-35-938.png|width=569,height=133!
> ExecuteScript processor always stuck after restart or multi thread
> --
>
> Key: NIFI-8625
> URL: https://issues.apache.org/jira/browse/NIFI-8625
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: KevinSky
>Priority: Critical
> Labels: ExecuteScript,stcuk
> Attachments: executeScript_nifi_8625.xml,
> image-2021-05-21-16-22-34-775.png
>
>
> In single thread, executeScript just stop and start, flow file always stuck
> in connections.
> In multi thread.executeScript always throw exception like is already marked
> for transfer in or is not known in this session.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351419#comment-17351419 ] KevinSky edited comment on NIFI-8625 at 5/26/21, 1:25 AM: -- hi [#Otto Fowler], Here is a better solution: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java !image-2021-05-26-09-22-59-199.png|width=556,height=293! !image-2021-05-26-09-23-13-561.png|width=565,height=506! nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java !image-2021-05-26-09-23-35-938.png|width=569,height=133! was (Author: kevinsky1093): hi [#Otto Fowler], Here is a better solution: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java !image-2021-05-26-09-13-01-784.png|width=965,height=298! !image-2021-05-26-09-13-25-543.png! nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java !image-2021-05-26-09-13-55-074.png|width=814,height=104! > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Priority: Critical > Labels: ExecuteScript,stcuk > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Comment Edited] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351419#comment-17351419 ] KevinSky edited comment on NIFI-8625 at 5/26/21, 1:20 AM: -- hi [#Otto Fowler], Here is a better solution: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java !image-2021-05-26-09-13-01-784.png|width=965,height=298! !image-2021-05-26-09-13-25-543.png! nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java !image-2021-05-26-09-13-55-074.png|width=814,height=104! was (Author: kevinsky1093): hi [#Otto Fowler], Here is a better solution: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java !image-2021-05-26-09-13-01-784.png|width=965,height=298! !image-2021-05-26-09-13-25-543.png! nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java !image-2021-05-26-09-13-55-074.png|width=814,height=104! > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Priority: Critical > Labels: ExecuteScript,stcuk > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351419#comment-17351419 ] KevinSky commented on NIFI-8625: hi [#Otto Fowler], Here is a better solution: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/impl/JythonScriptEngineConfigurator.java !image-2021-05-26-09-13-01-784.png|width=965,height=298! !image-2021-05-26-09-13-25-543.png! nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/script/ScriptingComponentHelper.java !image-2021-05-26-09-13-55-074.png|width=814,height=104! > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Priority: Critical > Labels: ExecuteScript,stcuk > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8614) Null Cluster Node Firewall fails to start on Spring 5
[
https://issues.apache.org/jira/browse/NIFI-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-8614:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Null Cluster Node Firewall fails to start on Spring 5
> -
>
> Key: NIFI-8614
> URL: https://issues.apache.org/jira/browse/NIFI-8614
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Blocker
> Fix For: 1.14.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Following the upgrade to Spring Framework 5 in NIFI-8502, a cluster node can
> fail start due to a null {{ClusterNodeFirewall}} object. The
> {{FileBasedClusterNodeFirewallFactoryBean}} returns null when no
> configuration file exists, which causes ApplicationContext.getBean() to
> return a {{NullBean}} in {{NodeClusterCoordinatorFactoryBean}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8614) Null Cluster Node Firewall fails to start on Spring 5
[
https://issues.apache.org/jira/browse/NIFI-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351351#comment-17351351
]
ASF subversion and git services commented on NIFI-8614:
---
Commit 103aae64cb28de235983880ba2b78f935ca56767 in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=103aae6 ]
NIFI-8614 Adjusted NodeClusterCoordinatorFactoryBean to handle null
ClusterNodeFirewall (#5100)
- Changed FileBasedClusterNodeFirewallFactoryBean to return null when
configuration file is not found
> Null Cluster Node Firewall fails to start on Spring 5
> -
>
> Key: NIFI-8614
> URL: https://issues.apache.org/jira/browse/NIFI-8614
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Blocker
> Fix For: 1.14.0
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> Following the upgrade to Spring Framework 5 in NIFI-8502, a cluster node can
> fail start due to a null {{ClusterNodeFirewall}} object. The
> {{FileBasedClusterNodeFirewallFactoryBean}} returns null when no
> configuration file exists, which causes ApplicationContext.getBean() to
> return a {{NullBean}} in {{NodeClusterCoordinatorFactoryBean}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] markap14 commented on pull request #5100: NIFI-8614 Corrected Cluster Node Firewall for Spring 5
markap14 commented on pull request #5100: URL: https://github.com/apache/nifi/pull/5100#issuecomment-848256894 Thanks @exceptionfactory this looks good. Ran system tests and saw that they are now successful whereas the cluster-based ones previously were failing. +1 merged to main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 merged pull request #5100: NIFI-8614 Corrected Cluster Node Firewall for Spring 5
markap14 merged pull request #5100: URL: https://github.com/apache/nifi/pull/5100 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-8629) Add periodic logging to Stateless that provides component statuses
[ https://issues.apache.org/jira/browse/NIFI-8629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-8629: - Fix Version/s: 1.14.0 Status: Patch Available (was: Open) > Add periodic logging to Stateless that provides component statuses > -- > > Key: NIFI-8629 > URL: https://issues.apache.org/jira/browse/NIFI-8629 > Project: Apache NiFi > Issue Type: Improvement > Components: NiFi Stateless >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.14.0 > > Time Spent: 10m > Remaining Estimate: 0h > > NiFi provides a lot of details about how each component is operating in the > UI. However, with stateless, that is not available. NiFi also provides a > ControllerStatusReportingTask that logs most of the information. However, > that's part of the standard nar currently, which is almost 80 MB, and is not > included by default. We should add something similar to the > ControllerStatusReportingTask to stateless but have it always run (as long as > the log level is sufficiently high) rather than requiring the extra steps of > configuring the Reporting Task. The output will likely be similar but not > identical to ControllerStatusReportingTask, as we will want to make sure that > the output is tailored well to Stateless. > Additionally, the ControllerStatusReportingTask is fairly expensive to run > because, as an extension, it cannot have access to the framework-level > objects such as Counters, Processors, etc. and as a result must build an > expensive ProcessGroupStatus data model to operate on. By implementing this > in the stateless framework, it can be done more efficiently. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] markap14 opened a new pull request #5101: NIFI-8629: Implemented the LogComponentStatuses task that runs period…
markap14 opened a new pull request #5101:
URL: https://github.com/apache/nifi/pull/5101
…ically in stateless. Also noticed a typo in the
ControllerStatusReportingTask and found in comparing outputs that it had a bug
that caused it to log counters generated only by processors at the root level
so fixed that.
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-8614) Null Cluster Node Firewall fails to start on Spring 5
[
https://issues.apache.org/jira/browse/NIFI-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
David Handermann updated NIFI-8614:
---
Status: Patch Available (was: In Progress)
> Null Cluster Node Firewall fails to start on Spring 5
> -
>
> Key: NIFI-8614
> URL: https://issues.apache.org/jira/browse/NIFI-8614
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Blocker
> Fix For: 1.14.0
>
> Time Spent: 50m
> Remaining Estimate: 0h
>
> Following the upgrade to Spring Framework 5 in NIFI-8502, a cluster node can
> fail start due to a null {{ClusterNodeFirewall}} object. The
> {{FileBasedClusterNodeFirewallFactoryBean}} returns null when no
> configuration file exists, which causes ApplicationContext.getBean() to
> return a {{NullBean}} in {{NodeClusterCoordinatorFactoryBean}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] exceptionfactory opened a new pull request #5100: NIFI-8614 Corrected Cluster Node Firewall for Spring 5
exceptionfactory opened a new pull request #5100:
URL: https://github.com/apache/nifi/pull/5100
Description of PR
NIFI-8614 Updates `NodeClusterCoordinatorFactoryBean` to support a `null`
instance of `ClusterNodeFirewall` as expected prior to upgrading from Spring
Framework 4 to Spring Framework 5. Changes include adjusting
`FileBasedClusterNodeFirewallFactoryBean` to return `null` when the firewall
configuration file is not found. These changes are necessary to support
existing system integration tests and clusters running without TLS configured.
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [X] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [X] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [X] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [X] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [X] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [X] Have you written or updated unit tests to verify your changes?
- [X] Have you verified that the full build is successful on JDK 8?
- [X] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-8522) NiFi can duplicate controller services when generating templates
[ https://issues.apache.org/jira/browse/NIFI-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-8522: - Attachment: Screen Shot 2021-05-25 at 2.55.10 PM.png > NiFi can duplicate controller services when generating templates > > > Key: NIFI-8522 > URL: https://issues.apache.org/jira/browse/NIFI-8522 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matthew Clarke >Priority: Major > Attachments: Screen Shot 2021-05-25 at 2.55.10 PM.png, Screen Shot > 2021-05-25 at 2.55.33 PM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > NiFi throws "The specified observer identifier already exists" exception when > trying to instantiate a template to the canvas. Parsing the template xml > reveals multiple controller services with same UUID. > Steps to reproduce: > 1. Create "PG-level1" > 2. within "PG-level1" create two new sub process groups "PG-level2-A" and > "PG-level2-B". > 3. Create a controller service(s) in "PG-level2-A" like AvroReader and > CSVRecordSetWriter. Add a processor like ConvertRecord and configure it to > reference these two controller services. > 4. Right click on the ConvertRecord processor and copy it. Exit > "PG-level2-A" and enter "PG-level2-B" and paste that copy processor. The > processor will show both configured controller services as "Incompatible > Controller Service Configured" and invalid bulletin will reflect the > properties reference controller services that do not exist. > 5. go up to "PG-level1" level and generate a template. Template is created > successfully. > 6. Try to instantiate this template to the canvas and it will only partially > instantiate and throw the "observer identifier already exist exception. > Inspection of the template shows that the controller services scoped to > PG-level2-A were added into the template again at the parent "PG-level1" > scope with same UUID. > The expected behavior is that the controller services do not get duplicated > and the processor in the instantiate template either has no configured > controller services or still reflects that it references controller > service(s) that do not exist. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8522) NiFi can duplicate controller services when generating templates
[ https://issues.apache.org/jira/browse/NIFI-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-8522: - Attachment: Screen Shot 2021-05-25 at 2.55.33 PM.png > NiFi can duplicate controller services when generating templates > > > Key: NIFI-8522 > URL: https://issues.apache.org/jira/browse/NIFI-8522 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matthew Clarke >Priority: Major > Attachments: Screen Shot 2021-05-25 at 2.55.10 PM.png, Screen Shot > 2021-05-25 at 2.55.33 PM.png > > Time Spent: 0.5h > Remaining Estimate: 0h > > NiFi throws "The specified observer identifier already exists" exception when > trying to instantiate a template to the canvas. Parsing the template xml > reveals multiple controller services with same UUID. > Steps to reproduce: > 1. Create "PG-level1" > 2. within "PG-level1" create two new sub process groups "PG-level2-A" and > "PG-level2-B". > 3. Create a controller service(s) in "PG-level2-A" like AvroReader and > CSVRecordSetWriter. Add a processor like ConvertRecord and configure it to > reference these two controller services. > 4. Right click on the ConvertRecord processor and copy it. Exit > "PG-level2-A" and enter "PG-level2-B" and paste that copy processor. The > processor will show both configured controller services as "Incompatible > Controller Service Configured" and invalid bulletin will reflect the > properties reference controller services that do not exist. > 5. go up to "PG-level1" level and generate a template. Template is created > successfully. > 6. Try to instantiate this template to the canvas and it will only partially > instantiate and throw the "observer identifier already exist exception. > Inspection of the template shows that the controller services scoped to > PG-level2-A were added into the template again at the parent "PG-level1" > scope with same UUID. > The expected behavior is that the controller services do not get duplicated > and the processor in the instantiate template either has no configured > controller services or still reflects that it references controller > service(s) that do not exist. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8522) NiFi can duplicate controller services when generating templates
[ https://issues.apache.org/jira/browse/NIFI-8522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351304#comment-17351304 ] Joseph Gresock commented on NIFI-8522: -- You can only reproduce the bug if you select PG-level2-A and PG-level2-B to create the template at Step 5. The bug does not occur if you create a template by selecting PG-level-1. > NiFi can duplicate controller services when generating templates > > > Key: NIFI-8522 > URL: https://issues.apache.org/jira/browse/NIFI-8522 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Matthew Clarke >Priority: Major > Time Spent: 0.5h > Remaining Estimate: 0h > > NiFi throws "The specified observer identifier already exists" exception when > trying to instantiate a template to the canvas. Parsing the template xml > reveals multiple controller services with same UUID. > Steps to reproduce: > 1. Create "PG-level1" > 2. within "PG-level1" create two new sub process groups "PG-level2-A" and > "PG-level2-B". > 3. Create a controller service(s) in "PG-level2-A" like AvroReader and > CSVRecordSetWriter. Add a processor like ConvertRecord and configure it to > reference these two controller services. > 4. Right click on the ConvertRecord processor and copy it. Exit > "PG-level2-A" and enter "PG-level2-B" and paste that copy processor. The > processor will show both configured controller services as "Incompatible > Controller Service Configured" and invalid bulletin will reflect the > properties reference controller services that do not exist. > 5. go up to "PG-level1" level and generate a template. Template is created > successfully. > 6. Try to instantiate this template to the canvas and it will only partially > instantiate and throw the "observer identifier already exist exception. > Inspection of the template shows that the controller services scoped to > PG-level2-A were added into the template again at the parent "PG-level1" > scope with same UUID. > The expected behavior is that the controller services do not get duplicated > and the processor in the instantiate template either has no configured > controller services or still reflects that it references controller > service(s) that do not exist. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-8520) Parameter Contexts - Show the wrong information of referencing components
[ https://issues.apache.org/jira/browse/NIFI-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-8520. -- Fix Version/s: 1.14.0 Resolution: Fixed > Parameter Contexts - Show the wrong information of referencing components > - > > Key: NIFI-8520 > URL: https://issues.apache.org/jira/browse/NIFI-8520 > Project: Apache NiFi > Issue Type: Bug >Reporter: Hsin-Ying Lee >Assignee: Hsin-Ying Lee >Priority: Minor > Fix For: 1.14.0 > > Attachments: image-2021-05-06-14-25-48-594.png, > image-2021-05-06-14-26-24-746.png > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Parameter contexts show the wrong information of referencing components when > we re-open the view page or edit page. > > Steps to Reproduce, > # Create a process group and binding a parameter context > # create 2 components and use the parameter > # open parameter context view page, close, and open again > ## if not happened click "Name" to sort and redo step 3. > > !image-2021-05-06-14-25-48-594.png! > !image-2021-05-06-14-26-24-746.png! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8520) Parameter Contexts - Show the wrong information of referencing components
[ https://issues.apache.org/jira/browse/NIFI-8520?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351241#comment-17351241 ] ASF subversion and git services commented on NIFI-8520: --- Commit beb1d2f4451c2e3311f0edc4ba2effd8ce8f914a in nifi's branch refs/heads/main from s9514171 [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=beb1d2f ] NIFI-8520 - Parameter Contexts - Show the wrong information of referencing components Signed-off-by: Pierre Villard This closes #5060. > Parameter Contexts - Show the wrong information of referencing components > - > > Key: NIFI-8520 > URL: https://issues.apache.org/jira/browse/NIFI-8520 > Project: Apache NiFi > Issue Type: Bug >Reporter: Hsin-Ying Lee >Assignee: Hsin-Ying Lee >Priority: Minor > Attachments: image-2021-05-06-14-25-48-594.png, > image-2021-05-06-14-26-24-746.png > > Time Spent: 1h 40m > Remaining Estimate: 0h > > Parameter contexts show the wrong information of referencing components when > we re-open the view page or edit page. > > Steps to Reproduce, > # Create a process group and binding a parameter context > # create 2 components and use the parameter > # open parameter context view page, close, and open again > ## if not happened click "Name" to sort and redo step 3. > > !image-2021-05-06-14-25-48-594.png! > !image-2021-05-06-14-26-24-746.png! > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #5060: NIFI-8520 - Parameter Contexts - Show the wrong information of referencing compon…
asfgit closed pull request #5060: URL: https://github.com/apache/nifi/pull/5060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351225#comment-17351225 ] Otto Fowler commented on NIFI-8625: --- I'm going to try to get some feedback or tips from [~mburgess149] > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Priority: Critical > Labels: ExecuteScript,stcuk > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[
https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351179#comment-17351179
]
Otto Fowler commented on NIFI-8625:
---
also seeing : e -> {PyBaseExceptionDerived@19457} "AttributeError("'NoneType'
object has no attribute 'getAttribute'",)
> ExecuteScript processor always stuck after restart or multi thread
> --
>
> Key: NIFI-8625
> URL: https://issues.apache.org/jira/browse/NIFI-8625
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: KevinSky
>Priority: Critical
> Labels: ExecuteScript,stcuk
> Attachments: executeScript_nifi_8625.xml,
> image-2021-05-21-16-22-34-775.png
>
>
> In single thread, executeScript just stop and start, flow file always stuck
> in connections.
> In multi thread.executeScript always throw exception like is already marked
> for transfer in or is not known in this session.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351176#comment-17351176 ] Otto Fowler commented on NIFI-8625: --- It looks like once I get the 'not known to session' exception, every new file fails with that and the queue is not being processed > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Priority: Critical > Labels: ExecuteScript,stcuk > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[ https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351171#comment-17351171 ] Otto Fowler commented on NIFI-8625: --- What I'm doing: I have 4 files in the input directory after processing the first four, I do a `cp` like from 1.txt to 5.txt etc to add a file at a time Sometimes it works, sometimes I get the exception above > ExecuteScript processor always stuck after restart or multi thread > -- > > Key: NIFI-8625 > URL: https://issues.apache.org/jira/browse/NIFI-8625 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.13.2 >Reporter: KevinSky >Priority: Critical > Labels: ExecuteScript,stcuk > Attachments: executeScript_nifi_8625.xml, > image-2021-05-21-16-22-34-775.png > > > In single thread, executeScript just stop and start, flow file always stuck > in connections. > In multi thread.executeScript always throw exception like is already marked > for transfer in or is not known in this session. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[
https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351169#comment-17351169
]
Otto Fowler commented on NIFI-8625:
---
also seeing this at times
{noformat}
xecuteScript[id=bb911eff-0bbc-346e-6a34-3875ae197ff0]
org.apache.nifi.processor.exception.FlowFileHandlingException:
StandardFlowFileRecord[uuid=4cc02d23-3763-4b09-99e2-725d67924ba4,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1621958091080-1, container=default,
section=1], offset=43, length=4],offset=0,name=5.txt,size=4] transfer
relationship not specified
at
org.apache.nifi.controller.repository.StandardProcessSession.validateCommitState(StandardProcessSession.java:245)
at
org.apache.nifi.controller.repository.StandardProcessSession.checkpoint(StandardProcessSession.java:259)
at
org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:406)
at
org.apache.nifi.controller.repository.StandardProcessSession.commitAsync(StandardProcessSession.java:362)
at
org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:273)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1180)
at org.apache.nifi.controller.tasks.Con
{noformat}
> ExecuteScript processor always stuck after restart or multi thread
> --
>
> Key: NIFI-8625
> URL: https://issues.apache.org/jira/browse/NIFI-8625
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: KevinSky
>Priority: Critical
> Labels: ExecuteScript,stcuk
> Attachments: executeScript_nifi_8625.xml,
> image-2021-05-21-16-22-34-775.png
>
>
> In single thread, executeScript just stop and start, flow file always stuck
> in connections.
> In multi thread.executeScript always throw exception like is already marked
> for transfer in or is not known in this session.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[
https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351162#comment-17351162
]
Otto Fowler commented on NIFI-8625:
---
ok, i'm seeing mixed results:
newest error detail is
{noformat}
org.apache.nifi.processor.exception.FlowFileHandlingException:
org.apache.nifi.processor.exception.FlowFileHandlingException:
StandardFlowFileRecord[uuid=29345e45-3fc6-4859-9449-0ce12f640e6f,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1621958091080-1, container=default,
section=1], offset=9, length=4],offset=0,name=2.txt,size=4] is already marked
for transfer
{noformat}
> ExecuteScript processor always stuck after restart or multi thread
> --
>
> Key: NIFI-8625
> URL: https://issues.apache.org/jira/browse/NIFI-8625
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: KevinSky
>Priority: Critical
> Labels: ExecuteScript,stcuk
> Attachments: executeScript_nifi_8625.xml,
> image-2021-05-21-16-22-34-775.png
>
>
> In single thread, executeScript just stop and start, flow file always stuck
> in connections.
> In multi thread.executeScript always throw exception like is already marked
> for transfer in or is not known in this session.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1403) Create TagS3Object processor
[ https://issues.apache.org/jira/browse/MINIFICPP-1403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Gyimesi updated MINIFICPP-1403: - Description: Create new processor to tag S3 objects of a bucket with similar functionality defined in Nifi: [https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.13.2/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.9.0/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html] was: Create new processor to tag S3 objects of a bucket with similar functionality defined in Nifi: [https://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.9.0/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html] > Create TagS3Object processor > > > Key: MINIFICPP-1403 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1403 > Project: Apache NiFi MiNiFi C++ > Issue Type: New Feature >Reporter: Gabor Gyimesi >Priority: Minor > > Create new processor to tag S3 objects of a bucket with similar functionality > defined in Nifi: > [https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.13.2/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.9.0/org.apache.nifi.processors.aws.s3.TagS3Object/index.htmlhttps://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html|https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-aws-nar/1.12.1/org.apache.nifi.processors.aws.s3.ListS3/index.html] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8614) Null Cluster Node Firewall fails to start on Spring 5
[
https://issues.apache.org/jira/browse/NIFI-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351147#comment-17351147
]
Mark Payne commented on NIFI-8614:
--
Changing priority to blocker, as all clustered system tests are now broken.
> Null Cluster Node Firewall fails to start on Spring 5
> -
>
> Key: NIFI-8614
> URL: https://issues.apache.org/jira/browse/NIFI-8614
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Blocker
> Fix For: 1.14.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Following the upgrade to Spring Framework 5 in NIFI-8502, a cluster node can
> fail start due to a null {{ClusterNodeFirewall}} object. The
> {{FileBasedClusterNodeFirewallFactoryBean}} returns null when no
> configuration file exists, which causes ApplicationContext.getBean() to
> return a {{NullBean}} in {{NodeClusterCoordinatorFactoryBean}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8614) Null Cluster Node Firewall fails to start on Spring 5
[
https://issues.apache.org/jira/browse/NIFI-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-8614:
-
Priority: Blocker (was: Major)
> Null Cluster Node Firewall fails to start on Spring 5
> -
>
> Key: NIFI-8614
> URL: https://issues.apache.org/jira/browse/NIFI-8614
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Blocker
> Fix For: 1.14.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Following the upgrade to Spring Framework 5 in NIFI-8502, a cluster node can
> fail start due to a null {{ClusterNodeFirewall}} object. The
> {{FileBasedClusterNodeFirewallFactoryBean}} returns null when no
> configuration file exists, which causes ApplicationContext.getBean() to
> return a {{NullBean}} in {{NodeClusterCoordinatorFactoryBean}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8630) PutEmail fails to negotiate TLS1.2 with latest JDK.
[
https://issues.apache.org/jira/browse/NIFI-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351146#comment-17351146
]
David Handermann commented on NIFI-8630:
Thanks for submitting this issue [~branko.peshevski]! The Java Mail reference
implementation is now named [Jakarta Mail|https://eclipse-ee4j.github.io/mail/]
so dependency updates should leverage those libraries.
> PutEmail fails to negotiate TLS1.2 with latest JDK.
>
>
> Key: NIFI-8630
> URL: https://issues.apache.org/jira/browse/NIFI-8630
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Branko Peshevski
>Priority: Critical
>
> On April 20, 2021 with the latest JDK the support for TLS 1.0 and 1.1 was
> removed/disabled. Using the PutEmail processor with the latest OpenJdk and
> TLS set to true is failing with:
> {code:java}
> Failed to send email for
> StandardFlowFileRecord[uuid=7e8d3482-59ac-4993-a9ea-fa9089ced554,claim=,offset=0,name=7e8d3482-59ac-4993-a9ea-fa9089ced554,size=0]:
> Could not convert socket to TLS; routing to failure:
> javax.mail.MessagingException: Could not convert socket to TLS;
> nested exception is:
> javax.net.ssl.SSLHandshakeException: No appropriate protocol
> (protocol is disabled or cipher suites are inappropriate)
> javax.mail.MessagingException: Could not convert socket to TLS
> at com.sun.mail.smtp.SMTPTransport.startTLS(SMTPTransport.java:1907)
> at
> com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:666)
> at javax.mail.Service.connect(Service.java:317)
> at javax.mail.Service.connect(Service.java:176)
> at javax.mail.Service.connect(Service.java:125)
> at javax.mail.Transport.send0(Transport.java:194)
> at javax.mail.Transport.send(Transport.java:124)
> at
> org.apache.nifi.processors.standard.PutEmail.send(PutEmail.java:541)
> at
> org.apache.nifi.processors.standard.PutEmail.onTrigger(PutEmail.java:395)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> {code}
> From my research the javax.mail version is quite old and it is known that
> 1.4.x fails to renegotiate with better tls algorithm. But I think it would be
> better to add new Property Descriptor to the PutEmail processor that will
> default the ssl protocol to TLS1.2.
> * Update the javax.mail dependency.
> * Add PropertyDescriptor for setting the ssl algorithm that will default to
> TLS 1.2
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Reopened] (NIFI-8614) Null Cluster Node Firewall fails to start on Spring 5
[
https://issues.apache.org/jira/browse/NIFI-8614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne reopened NIFI-8614:
--
Re-Opening because this breaks any cluster where communications are not secured
> Null Cluster Node Firewall fails to start on Spring 5
> -
>
> Key: NIFI-8614
> URL: https://issues.apache.org/jira/browse/NIFI-8614
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.14.0
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> Following the upgrade to Spring Framework 5 in NIFI-8502, a cluster node can
> fail start due to a null {{ClusterNodeFirewall}} object. The
> {{FileBasedClusterNodeFirewallFactoryBean}} returns null when no
> configuration file exists, which causes ApplicationContext.getBean() to
> return a {{NullBean}} in {{NodeClusterCoordinatorFactoryBean}}.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1417) Improve FileStream error reporting
[ https://issues.apache.org/jira/browse/MINIFICPP-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1417: Status: Reopened (was: Closed) > Improve FileStream error reporting > -- > > Key: MINIFICPP-1417 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1417 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 40m > Remaining Estimate: 0h > > In some deployments, "Unknown exception" errors were reported, usually after > flow update. It's caused by a narrowing error from FileStream that occurs > because of incorrect error handling. The goal is to fix this bug and improve > error reporting in FileStream so that future errors don't manifest themselves > as "Unknown exception". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-8620) NullPointerException on commit with multiple ProcessSession
[
https://issues.apache.org/jira/browse/NIFI-8620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-8620:
-
Fix Version/s: 1.14.0
Status: Patch Available (was: Open)
> NullPointerException on commit with multiple ProcessSession
> ---
>
> Key: NIFI-8620
> URL: https://issues.apache.org/jira/browse/NIFI-8620
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.0
> Environment: Kubernetes on linux
>Reporter: Stanislas Deneuville
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.14.0
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> {color:#00}For my particular use case I read and process a big file and
> produce smaller results along the way. I wanted to be able to regularly
> commit what has already been done before the end of the whole process.{color}
> {color:#00}So I inspired myself from the BinManager and created a custom
> processor that use 2 ProcessSessions:{color}
> * {color:#00}a first main session for the read input flowfile and
> creating new ones forked from it{color}
> * {color:#00}a second session for committing on the fly{color}
>
> {color:#00}The workflow is something like that:{color}
> {code:java}
> final ProcessSession mainSession = sessionFactory.createSession();
> final ProcessSession secondSession = sessionFactory.createSession();
> FlowFile inputFlowFile = mainSession.get();
> try (InputStream in = mainSession.read(inputFlowFile)) {
> while (stillSomethingToRead) {
> // read and process data
> inputData = in.read(...);
> transformedData = transform(inputData);
> // Create output flowfile
> FlowFile outputflowFile = mainSession.create(inputFlowfile);
> // write transformedData to outputflowFile content
> [...]
> // also put some attributes on outputflowFile
> [...]
> // Output the results without waiting
> mainSession.migrate(secondSession, outputflowFiles);
> secondSession.transfer(outputflowFiles, successRelationship);
> secondSession.commit();
> }
> }
> mainsession.commit()
> {code}
>
> It works well on Nifi Mock, however in a real Nifi environment I get a null
> pointer exception during the commit.
> {noformat}
> [id=9f6342ac-ae78-30f7-22cf-6d7517618f19] Unknown error occurred:
> java.lang.NullPointerException java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.repository.StandardProcessSession.updateEventContentClaims(StandardProcessSession.java:786)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.updateProvenanceRepo(StandardProcessSession.java:600)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:353)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:332){noformat}
>
> {color:#00}Note: I don't do anything related to Data Provenance in my
> code.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1413) Improve exception logging
[ https://issues.apache.org/jira/browse/MINIFICPP-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1413. --- > Improve exception logging > - > > Key: MINIFICPP-1413 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1413 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.9.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > 1. No empty std::exceptions > 2. Add typeid to catch-all logging when possible -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1417) Improve FileStream error reporting
[ https://issues.apache.org/jira/browse/MINIFICPP-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1417. - > Improve FileStream error reporting > -- > > Key: MINIFICPP-1417 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1417 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 40m > Remaining Estimate: 0h > > In some deployments, "Unknown exception" errors were reported, usually after > flow update. It's caused by a narrowing error from FileStream that occurs > because of incorrect error handling. The goal is to fix this bug and improve > error reporting in FileStream so that future errors don't manifest themselves > as "Unknown exception". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1417) Improve FileStream error reporting
[ https://issues.apache.org/jira/browse/MINIFICPP-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1417. --- > Improve FileStream error reporting > -- > > Key: MINIFICPP-1417 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1417 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 40m > Remaining Estimate: 0h > > In some deployments, "Unknown exception" errors were reported, usually after > flow update. It's caused by a narrowing error from FileStream that occurs > because of incorrect error handling. The goal is to fix this bug and improve > error reporting in FileStream so that future errors don't manifest themselves > as "Unknown exception". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1398) EL date functions are not available on Windows
[
https://issues.apache.org/jira/browse/MINIFICPP-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz closed MINIFICPP-1398.
---
> EL date functions are not available on Windows
> --
>
> Key: MINIFICPP-1398
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1398
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Major
> Fix For: 0.9.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> in Expression.h:
> {{#define EXPRESSION_LANGUAGE_USE_DATE}}
> {{// Disable date in EL for incompatible compilers}}
> {{#if \_\_GNUC\_\_ < 5}}
> {{#undef EXPRESSION_LANGUAGE_USE_DATE}}
> {{#endif}}
> I assume \_\_GNUC\_\_ is evaluated as 0 on Windows and that's the cause.
> update: due to problems with the date and timezone libraries on Windows, I'm
> going to add a fallback based on strftime for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] markap14 opened a new pull request #5099: NIFI-8620: Ensure that we provider appropriate error messages if atte…
markap14 opened a new pull request #5099:
URL: https://github.com/apache/nifi/pull/5099
…mpting to migrate FlowFiles from one session to another without including
full hierarchy; added tests to verify behavior
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Closed] (MINIFICPP-1378) Comment S2S credentials
[ https://issues.apache.org/jira/browse/MINIFICPP-1378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1378. --- > Comment S2S credentials > --- > > Key: MINIFICPP-1378 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1378 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1378) Comment S2S credentials
[ https://issues.apache.org/jira/browse/MINIFICPP-1378?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1378: Fix Version/s: 0.9.0 > Comment S2S credentials > --- > > Key: MINIFICPP-1378 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1378 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Fix For: 0.9.0 > > Time Spent: 20m > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1318) Migrate PersistenceTests and TailFileTests repositories to /var/tmp
[ https://issues.apache.org/jira/browse/MINIFICPP-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1318. --- > Migrate PersistenceTests and TailFileTests repositories to /var/tmp > --- > > Key: MINIFICPP-1318 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1318 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Minor > Fix For: 0.9.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > See MINIFICPP-1188 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1370) Clarify README to not guarantee backwards compatiblity in 0.x versions
[ https://issues.apache.org/jira/browse/MINIFICPP-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1370. --- > Clarify README to not guarantee backwards compatiblity in 0.x versions > -- > > Key: MINIFICPP-1370 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1370 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > 0.x versions are development versions with no compatibility guarantees > according to semver. README incorrectly suggests backwards compatibility with > minor versions even in 0.x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1361) Try improving compile times by hiding spdlog in implementation files
[ https://issues.apache.org/jira/browse/MINIFICPP-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1361. --- > Try improving compile times by hiding spdlog in implementation files > > > Key: MINIFICPP-1361 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1361 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.9.0 > > > spdlog and fmt are template-heavy libraries and since they are included in > almost all of our compilation units, they can have a major impact on our > compilation times. > Goal: try moving all spdlog includes to our logging-related .cpp files, > possibly by using forward declarations and extra indirections, and see if it > makes a difference. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1375) Allow users to build MSI with Universal C Runtime DLLs
[ https://issues.apache.org/jira/browse/MINIFICPP-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1375. --- > Allow users to build MSI with Universal C Runtime DLLs > -- > > Key: MINIFICPP-1375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > MiNiFi C++ fails to start on older windows versions that don't have the > Universal C Runtime embedded or the respective update installed. Microsoft > allows for the redistribution of these DLLs and they only take up about 2 MB, > so we should allow users to include them in the MSI for private/internal use. > The resulting MSI can not be redistributed under Apache-2.0, because the UCRT > license is not compatible with Apache-2.0 ([Category > X|https://www.apache.org/legal/resolved.html#category-x]). > [https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/] (see > point 6. near the bottom) > Redistributable files list: > [https://docs.microsoft.com/en-us/legal/windows-sdk/redist] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1257) PublishKafka should not leak messages on failed produce
[ https://issues.apache.org/jira/browse/MINIFICPP-1257?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1257. --- > PublishKafka should not leak messages on failed produce > --- > > Key: MINIFICPP-1257 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1257 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Blocker > Fix For: 0.9.0 > > Time Spent: 50m > Remaining Estimate: 0h > > On failure `rd_kafka_producev` doesn't call > PublishKafka::messageDeliveryCallback, which takes ownership of the messages, > leaking the whole batch if any of the deliveries fail. > > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1362) Windows service control handler code log level should be trace
[ https://issues.apache.org/jira/browse/MINIFICPP-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1362. --- > Windows service control handler code log level should be trace > -- > > Key: MINIFICPP-1362 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1362 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Fix For: 0.9.0 > > Time Spent: 20m > Remaining Estimate: 0h > > https://docs.microsoft.com/en-us/windows/win32/services/writing-a-control-handler-function > MiNiFi is logging the control code as INFO, which is spamming the event log. > It should be TRACE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (NIFI-8620) NullPointerException on commit with multiple ProcessSession
[
https://issues.apache.org/jira/browse/NIFI-8620?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne reassigned NIFI-8620:
Assignee: Mark Payne
> NullPointerException on commit with multiple ProcessSession
> ---
>
> Key: NIFI-8620
> URL: https://issues.apache.org/jira/browse/NIFI-8620
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.0
> Environment: Kubernetes on linux
>Reporter: Stanislas Deneuville
>Assignee: Mark Payne
>Priority: Major
>
> {color:#00}For my particular use case I read and process a big file and
> produce smaller results along the way. I wanted to be able to regularly
> commit what has already been done before the end of the whole process.{color}
> {color:#00}So I inspired myself from the BinManager and created a custom
> processor that use 2 ProcessSessions:{color}
> * {color:#00}a first main session for the read input flowfile and
> creating new ones forked from it{color}
> * {color:#00}a second session for committing on the fly{color}
>
> {color:#00}The workflow is something like that:{color}
> {code:java}
> final ProcessSession mainSession = sessionFactory.createSession();
> final ProcessSession secondSession = sessionFactory.createSession();
> FlowFile inputFlowFile = mainSession.get();
> try (InputStream in = mainSession.read(inputFlowFile)) {
> while (stillSomethingToRead) {
> // read and process data
> inputData = in.read(...);
> transformedData = transform(inputData);
> // Create output flowfile
> FlowFile outputflowFile = mainSession.create(inputFlowfile);
> // write transformedData to outputflowFile content
> [...]
> // also put some attributes on outputflowFile
> [...]
> // Output the results without waiting
> mainSession.migrate(secondSession, outputflowFiles);
> secondSession.transfer(outputflowFiles, successRelationship);
> secondSession.commit();
> }
> }
> mainsession.commit()
> {code}
>
> It works well on Nifi Mock, however in a real Nifi environment I get a null
> pointer exception during the commit.
> {noformat}
> [id=9f6342ac-ae78-30f7-22cf-6d7517618f19] Unknown error occurred:
> java.lang.NullPointerException java.lang.NullPointerException: null
> at
> org.apache.nifi.controller.repository.StandardProcessSession.updateEventContentClaims(StandardProcessSession.java:786)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.updateProvenanceRepo(StandardProcessSession.java:600)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:353)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.commit(StandardProcessSession.java:332){noformat}
>
> {color:#00}Note: I don't do anything related to Data Provenance in my
> code.{color}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1260) Make void_t in GeneralUtils forward-compabilble with C++17
[ https://issues.apache.org/jira/browse/MINIFICPP-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz closed MINIFICPP-1260. --- > Make void_t in GeneralUtils forward-compabilble with C++17 > -- > > Key: MINIFICPP-1260 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1260 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Fix For: 0.9.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] lordgamez commented on a change in pull request #1081: MINIFICPP-1565: Minor improvements to PerformanceDataMonitor
lordgamez commented on a change in pull request #1081:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1081#discussion_r638896669
##
File path: extensions/pdh/PerformanceDataMonitor.cpp
##
@@ -264,16 +281,29 @@ void
PerformanceDataMonitor::setupMembersFromProperties(const std::shared_ptrgetProperty(OutputFormatProperty.getName(),
output_format_string)) {
-if (output_format_string == OPEN_TELEMETRY_FORMAT_STR) {
- logger_->log_trace("OutputFormat is configured to be OpenTelemetry");
+if (output_format_string == PRETTY_OPEN_TELEMETRY_FORMAT_STR ||
output_format_string == COMPACT_OPEN_TELEMETRY_FORMAT_STR) {
output_format_ = OutputFormat::OPENTELEMETRY;
-} else if (output_format_string == JSON_FORMAT_STR) {
- logger_->log_trace("OutputFormat is configured to be JSON");
+ pretty_output_ = output_format_string ==
PRETTY_OPEN_TELEMETRY_FORMAT_STR;
+ logger_->log_trace("OutputFormat is configured to be %s OpenTelemetry",
pretty_output_ ? "pretty" : "compact");
+} else if (output_format_string == PRETTY_JSON_FORMAT_STR ||
output_format_string == COMPACT_JSON_FORMAT_STR) {
output_format_ = OutputFormat::JSON;
+ pretty_output_ = output_format_string == PRETTY_JSON_FORMAT_STR;
+ logger_->log_trace("OutputFormat is configured to be %s JSON",
pretty_output_ ? "pretty" : "compact");
} else {
- logger_->log_error("Invalid OutputFormat, defaulting to JSON");
output_format_ = OutputFormat::JSON;
+ pretty_output_ = true;
+ logger_->log_error("Invalid OutputFormat, defaulting to %s JSON",
pretty_output_ ? "pretty" : "compact");
+}
Review comment:
I may extract this to a member function
##
File path: extensions/pdh/PerformanceDataMonitor.cpp
##
@@ -264,16 +281,29 @@ void
PerformanceDataMonitor::setupMembersFromProperties(const std::shared_ptrgetProperty(OutputFormatProperty.getName(),
output_format_string)) {
-if (output_format_string == OPEN_TELEMETRY_FORMAT_STR) {
- logger_->log_trace("OutputFormat is configured to be OpenTelemetry");
+if (output_format_string == PRETTY_OPEN_TELEMETRY_FORMAT_STR ||
output_format_string == COMPACT_OPEN_TELEMETRY_FORMAT_STR) {
output_format_ = OutputFormat::OPENTELEMETRY;
-} else if (output_format_string == JSON_FORMAT_STR) {
- logger_->log_trace("OutputFormat is configured to be JSON");
+ pretty_output_ = output_format_string ==
PRETTY_OPEN_TELEMETRY_FORMAT_STR;
+ logger_->log_trace("OutputFormat is configured to be %s OpenTelemetry",
pretty_output_ ? "pretty" : "compact");
+} else if (output_format_string == PRETTY_JSON_FORMAT_STR ||
output_format_string == COMPACT_JSON_FORMAT_STR) {
output_format_ = OutputFormat::JSON;
+ pretty_output_ = output_format_string == PRETTY_JSON_FORMAT_STR;
+ logger_->log_trace("OutputFormat is configured to be %s JSON",
pretty_output_ ? "pretty" : "compact");
} else {
- logger_->log_error("Invalid OutputFormat, defaulting to JSON");
output_format_ = OutputFormat::JSON;
+ pretty_output_ = true;
+ logger_->log_error("Invalid OutputFormat, defaulting to %s JSON",
pretty_output_ ? "pretty" : "compact");
+}
+ }
+
+ std::string double_precision_string;
+ if (context->getProperty(DoublePrecisionProperty.getName(),
double_precision_string)) {
Review comment:
The getProperty should do the conversion implicitly, so reading this to
`double_precision_` variable should be enough.
##
File path: extensions/pdh/PerformanceDataMonitor.h
##
@@ -75,7 +78,9 @@ class PerformanceDataMonitor : public core::Processor {
void addCustomPDHCountersFromProperty(const std::string&
custom_pdh_counters);
OutputFormat output_format_;
+ bool pretty_output_;
Review comment:
This seems to be uninitialized.
##
File path: libminifi/include/utils/MathUtils.h
##
@@ -0,0 +1,43 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+class MathUtils {
+ public:
+ static double round_to(double original, int8_t precision)
[jira] [Closed] (MINIFICPP-1142) Ninja build with libressl fails
[
https://issues.apache.org/jira/browse/MINIFICPP-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz closed MINIFICPP-1142.
---
> Ninja build with libressl fails
> ---
>
> Key: MINIFICPP-1142
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1142
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
> Environment: Ubuntu 18.04 LTS
>Reporter: Marton Szasz
>Priority: Major
> Labels: build
>
> Using the Ninja generator of CMake, then trying to build with ninja fails
> with the following error message:
> {{ninja: error: 'thirdparty/libressl-install/lib/libtls.a', needed by
> 'main/minifi', missing and no known rule to make it}}
> Steps to reproduce:
> {code:java}
> mkdir ninja; cd ninja
> cmake -G Ninja -DUSE_SHARED_LIBS= -DENABLE_MQTT=ON -DENABLE_LIBRDKAFKA=ON
> -DPORTABLE=ON -DENABLE_COAP=ON -DCMAKE_BUILD_TYPE=Debug ..
> ninja{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1121) Upgrade spdlog
[
https://issues.apache.org/jira/browse/MINIFICPP-1121?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz closed MINIFICPP-1121.
---
> Upgrade spdlog
> --
>
> Key: MINIFICPP-1121
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1121
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Affects Versions: 0.7.0
>Reporter: Marton Szasz
>Assignee: Adam Hunyadi
>Priority: Minor
> Labels: MiNiFi-CPP-Hygiene
> Fix For: 0.9.0
>
> Time Spent: 7h 10m
> Remaining Estimate: 0h
>
> Or version of spdlog is 2+ years old. The new spdlog version uses a new
> version of the cppformat (back then)/fmt (now) formatting library.
> We should consider directly depending on {{fmt}} since we already have it as
> a transitive dependency and it would be useful for e.g. formatting
> exception/error messages, etc.
>
> *Update (hunyadi):*
> Seems like we have to skip version 1.0 with upgrading. There are quite a lot
> of non-documented breaking changes, for example this commit:
>
> [https://github.com/gabime/spdlog/commit/6f4cd8d397a443f095c1dce5c025f55684c70eac#diff-9458442ae281c51018015fd2773dc688]
> breaks ::instance() on stdout/stderr sinks. Unfortunately, changes like this
> in spdlog are not documented and the codebase is kept up-to-date with commits
> sent directly to the central repository.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1122) Ensure that all flow files have a resource claim
[
https://issues.apache.org/jira/browse/MINIFICPP-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz closed MINIFICPP-1122.
---
> Ensure that all flow files have a resource claim
>
>
> Key: MINIFICPP-1122
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1122
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Marton Szasz
>Priority: Major
> Fix For: 0.9.0
>
>
> The current minifi behavior is not creating {{ResourceClaim}} for all empty
> (content) flow files. The change would change behavior of some processors, as
> {{ProcessSession::read}} only calls the passed callback when there is a
> {{ResourceClaim}} and the flow file is not empty. The behavior will be made
> configurable in MINIFICPP-1047 with the current behavior being the default
> and the default should change in 1.0 to always creating {{ResourceClaim}}.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8616) Migrate SSL socket code to use Netty
[ https://issues.apache.org/jira/browse/NIFI-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Gough updated NIFI-8616: --- Description: The PutUDP/PutTCP using SSLSocketChannel should be migrated to use Netty library code rather than maintain our custom implementation. (was: The PutUDP/PutTCP and any other processors using SSLSocketChannel should be migrated to use Netty library code rather than maintain our custom implementation.) > Migrate SSL socket code to use Netty > > > Key: NIFI-8616 > URL: https://issues.apache.org/jira/browse/NIFI-8616 > Project: Apache NiFi > Issue Type: Improvement > Components: Security >Affects Versions: 1.13.2 >Reporter: Nathan Gough >Assignee: Nathan Gough >Priority: Major > Fix For: 1.14.0 > > > The PutUDP/PutTCP using SSLSocketChannel should be migrated to use Netty > library code rather than maintain our custom implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1122) Ensure that all flow files have a resource claim
[
https://issues.apache.org/jira/browse/MINIFICPP-1122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz updated MINIFICPP-1122:
Fix Version/s: (was: 1.0.0)
0.9.0
> Ensure that all flow files have a resource claim
>
>
> Key: MINIFICPP-1122
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1122
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Marton Szasz
>Priority: Major
> Fix For: 0.9.0
>
>
> The current minifi behavior is not creating {{ResourceClaim}} for all empty
> (content) flow files. The change would change behavior of some processors, as
> {{ProcessSession::read}} only calls the passed callback when there is a
> {{ResourceClaim}} and the flow file is not empty. The behavior will be made
> configurable in MINIFICPP-1047 with the current behavior being the default
> and the default should change in 1.0 to always creating {{ResourceClaim}}.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1142) Ninja build with libressl fails
[
https://issues.apache.org/jira/browse/MINIFICPP-1142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz resolved MINIFICPP-1142.
-
Resolution: Works for Me
The problem no longer occurs on my machine. If you ran into the same issue,
please reopen.
> Ninja build with libressl fails
> ---
>
> Key: MINIFICPP-1142
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1142
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
> Environment: Ubuntu 18.04 LTS
>Reporter: Marton Szasz
>Priority: Major
> Labels: build
>
> Using the Ninja generator of CMake, then trying to build with ninja fails
> with the following error message:
> {{ninja: error: 'thirdparty/libressl-install/lib/libtls.a', needed by
> 'main/minifi', missing and no known rule to make it}}
> Steps to reproduce:
> {code:java}
> mkdir ninja; cd ninja
> cmake -G Ninja -DUSE_SHARED_LIBS= -DENABLE_MQTT=ON -DENABLE_LIBRDKAFKA=ON
> -DPORTABLE=ON -DENABLE_COAP=ON -DCMAKE_BUILD_TYPE=Debug ..
> ninja{code}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1189) Cannot compile in debug mode on Windows
[ https://issues.apache.org/jira/browse/MINIFICPP-1189?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1189: Fix Version/s: 0.9.0 > Cannot compile in debug mode on Windows > --- > > Key: MINIFICPP-1189 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1189 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > > cmake/BundledYamlCpp.cmake assumes the library name to be libyaml-cppmd.lib, > but when compiled in debug mode, it's actually libyaml-cppmdd.lib. This > breaks compilation in debug mode on Windows. > Fix: check for debug config in the file and expect libyaml-cppmdd.lib on > windows in debug mode, instead of libyaml-cppmd.lib -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1148) ClientSocket crashes on Windows after repeated failed connection attempts
[
https://issues.apache.org/jira/browse/MINIFICPP-1148?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz updated MINIFICPP-1148:
Fix Version/s: 0.9.0
> ClientSocket crashes on Windows after repeated failed connection attempts
> -
>
> Key: MINIFICPP-1148
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1148
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
> Environment: Windows
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Major
> Fix For: 0.9.0
>
> Time Spent: 1h 50m
> Remaining Estimate: 0h
>
> On Windows, when we use {{ClientSocket}} to connect to a remote node that has
> multiple addresses returned by {{getaddrinfo}} and {{connect}} repeatedly
> fails for any reason, minifi-cpp may crash due to a use-after-free bug. The
> bug can not be exploited for any other purposes than crashing minifi.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1154) Fix various Socket bugs
[ https://issues.apache.org/jira/browse/MINIFICPP-1154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1154: Fix Version/s: 0.9.0 > Fix various Socket bugs > --- > > Key: MINIFICPP-1154 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1154 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > > * buggy name resolution > * broken move constructors -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1150) Fix const-correctness of the non-virtual subset of FlowFile member functions
[ https://issues.apache.org/jira/browse/MINIFICPP-1150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1150: Fix Version/s: 0.9.0 > Fix const-correctness of the non-virtual subset of FlowFile member functions > > > Key: MINIFICPP-1150 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1150 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Fix For: 0.9.0 > > Time Spent: 20m > Remaining Estimate: 0h > > ^^ > But leave the virtuals unchanged for backwards compatibility. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1188) Fix repo tests on linux systems with tmpfs /tmp
[ https://issues.apache.org/jira/browse/MINIFICPP-1188?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1188: Fix Version/s: 0.9.0 > Fix repo tests on linux systems with tmpfs /tmp > --- > > Key: MINIFICPP-1188 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1188 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Minor > Fix For: 0.9.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Repo tests use /tmp for temporary repos. Many linux distros (e.g. Arch, > Gentoo) mount /tmp as tmpfs, which is the page cache mounted as a filesystem. > Since we use RocksDB repositories with direct IO (i.e. send IO directly to > disk, avoid page cache), the integration tests that try to open repositories > under /tmp fail on the affected systems. > Fix: Place the temporary test repositories under /var/tmp instead. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1164) Resolve libarchive compilation issues on GCC 7+
[
https://issues.apache.org/jira/browse/MINIFICPP-1164?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz updated MINIFICPP-1164:
Fix Version/s: 0.9.0
> Resolve libarchive compilation issues on GCC 7+
> ---
>
> Key: MINIFICPP-1164
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1164
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Major
> Fix For: 0.9.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> The issue:
> Due to new warnings and libarchive using the {{-Werror}} flag, it's not
> possible to compile libarchive extensions in minifi-cpp.
> Possible resolutions:
> 1. Update libarchive and pass {{ENABLE_WERROR=OFF}} to its cmake build [1]
> (preferred)
> 2. Patch our current version of libarchive to not use {{-Werror}}
> [1] [https://github.com/libarchive/libarchive/pull/1033]
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Closed] (MINIFICPP-1252) Introduce gsl-lite
[
https://issues.apache.org/jira/browse/MINIFICPP-1252?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz closed MINIFICPP-1252.
---
> Introduce gsl-lite
> --
>
> Key: MINIFICPP-1252
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1252
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Minor
> Fix For: 0.9.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Add gsl-lite to minifi c++ so that we can follow the best practices of the
> c++ core guidelines more closely.
> Highlights:
> - {{gsl::owner}}: annotate owner pointers
> - {{gsl::not_null}}: pointer wrapper class with {{!= nullptr}} invariant,
> works on smart pointers
> - {{gsl_Expects}}, {{gsl_Ensures}}: contract checking
> - {{gsl::finally}}: execute callback at the end of the scope
> - {{gsl::narrow}}: checked narrowing conversion, fails on value change
> - {{gsl::narrow_cast}}: searchable annotation for narrowing conversions
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1190) RocksDB fails to build on Clang 10
[ https://issues.apache.org/jira/browse/MINIFICPP-1190?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1190: Fix Version/s: 0.9.0 > RocksDB fails to build on Clang 10 > -- > > Key: MINIFICPP-1190 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1190 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Our self-built third-party, RocksDB, fails to build on clang. The upstream is > already patched, but our version is older. > The issue is that a const member function tries to lock a mutex that's not > marked as mutable in a rocksdb class. It should never have compiled, but it > did. Now it fails correctly. > The supplied patch marks the mutex as mutable. > Thanks to [~bakaid] for assistance on how to apply third-party patches. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1226) Improve C2 heartbeat performance
[ https://issues.apache.org/jira/browse/MINIFICPP-1226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1226: Fix Version/s: 0.9.0 > Improve C2 heartbeat performance > > > Key: MINIFICPP-1226 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1226 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 1h > Remaining Estimate: 0h > > During a profiling session I noticed that heartbeat serialization took about > 30% of the used CPU time in a simple flow, with 1 sec heartbeat interval. > This is way too long. > In C2Agent::serializeMetrics, while constructing the payload, we used > C2Payload's move facilities incorrectly resulting in a silent fallback to > copy every time. This seems to have caused the high CPU usage. After the > submitted change, heartbeat generation was no longer showing up in the my > profiler view, so it should be significantly faster. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1260) Make void_t in GeneralUtils forward-compabilble with C++17
[ https://issues.apache.org/jira/browse/MINIFICPP-1260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1260: Fix Version/s: 0.9.0 > Make void_t in GeneralUtils forward-compabilble with C++17 > -- > > Key: MINIFICPP-1260 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1260 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Fix For: 0.9.0 > > Time Spent: 0.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1301) Cannot compile with GCC 10
[
https://issues.apache.org/jira/browse/MINIFICPP-1301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz updated MINIFICPP-1301:
Fix Version/s: 0.9.0
> Cannot compile with GCC 10
> --
>
> Key: MINIFICPP-1301
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1301
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Major
> Fix For: 0.9.0
>
>
> Compilation with GCC 10 fails, most likely due to the switch from
> {{-fcommon}} to {{-fno-common}} behavior.
> see: [https://wiki.gentoo.org/wiki/Gcc_10_porting_notes/fno_common] and
> [https://gcc.gnu.org/gcc-10/porting_to.html]
> Error:
> {{[ 52%] Linking CXX executable minifi}}
>
> {{/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld:
> ../libminifi/libcore-minifi.a(SSLContextService.cpp.o): in function
> `sk_X509_num':}}
> {{/usr/include/openssl/x509.h:99: undefined reference to `OPENSSL_sk_num'}}
>
> {{/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld:
> ../libminifi/libcore-minifi.a(SSLContextService.cpp.o): in function
> `sk_X509_pop':}}
> {{/usr/include/openssl/x509.h:99: undefined reference to `OPENSSL_sk_pop'}}
>
> {{/usr/lib/gcc/x86_64-pc-linux-gnu/10.1.0/../../../../x86_64-pc-linux-gnu/bin/ld:
> ../libminifi/libcore-minifi.a(SSLContextService.cpp.o): in function
> `sk_X509_pop_free':}}
> {{/usr/include/openssl/x509.h:99: undefined reference to
> `OPENSSL_sk_pop_free'}}
> {{collect2: error: ld returned 1 exit status}}
> {{gmake[2]: *** [main/CMakeFiles/minifiexe.dir/build.make:178: main/minifi]
> Error 1}}
> {{gmake[1]: *** [CMakeFiles/Makefile2:3567:
> main/CMakeFiles/minifiexe.dir/all] Error 2}}
> {{gmake: *** [Makefile:183: all] Error 2}}
>
> Workaround: Add {{-fcommon}} to your CFLAGS and CXXFLAGS
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1351) PublishKafka notifyStop accesses the connection without synchronization
[ https://issues.apache.org/jira/browse/MINIFICPP-1351?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1351: Fix Version/s: 0.9.0 > PublishKafka notifyStop accesses the connection without synchronization > --- > > Key: MINIFICPP-1351 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1351 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 1h > Remaining Estimate: 0h > > notifyStop doesn't lock the connection_mutex_ before reseting the conn_ to > nullptr. If onTrigger is still running, this crashes MiNiFi. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1326) Improve PublishKafka logging
[ https://issues.apache.org/jira/browse/MINIFICPP-1326?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1326: Fix Version/s: 0.9.0 > Improve PublishKafka logging > > > Key: MINIFICPP-1326 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1326 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Minor > Fix For: 0.9.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > PublishKafka seems to produce a strange behavior on some deployments where > messages are delivered but the agents waits until they would time out before > continuing to send more. This issue is about adding logs to ease debugging on > the affected deployments. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1318) Migrate PersistenceTests and TailFileTests repositories to /var/tmp
[ https://issues.apache.org/jira/browse/MINIFICPP-1318?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1318: Fix Version/s: 0.9.0 > Migrate PersistenceTests and TailFileTests repositories to /var/tmp > --- > > Key: MINIFICPP-1318 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1318 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Minor > Fix For: 0.9.0 > > Time Spent: 2h 20m > Remaining Estimate: 0h > > See MINIFICPP-1188 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1361) Try improving compile times by hiding spdlog in implementation files
[ https://issues.apache.org/jira/browse/MINIFICPP-1361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz resolved MINIFICPP-1361. - Fix Version/s: 0.9.0 Resolution: Duplicate > Try improving compile times by hiding spdlog in implementation files > > > Key: MINIFICPP-1361 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1361 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.9.0 > > > spdlog and fmt are template-heavy libraries and since they are included in > almost all of our compilation units, they can have a major impact on our > compilation times. > Goal: try moving all spdlog includes to our logging-related .cpp files, > possibly by using forward declarations and extra indirections, and see if it > makes a difference. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1375) Allow users to build MSI with Universal C Runtime DLLs
[ https://issues.apache.org/jira/browse/MINIFICPP-1375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1375: Fix Version/s: 0.9.0 > Allow users to build MSI with Universal C Runtime DLLs > -- > > Key: MINIFICPP-1375 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1375 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 3h 10m > Remaining Estimate: 0h > > MiNiFi C++ fails to start on older windows versions that don't have the > Universal C Runtime embedded or the respective update installed. Microsoft > allows for the redistribution of these DLLs and they only take up about 2 MB, > so we should allow users to include them in the MSI for private/internal use. > The resulting MSI can not be redistributed under Apache-2.0, because the UCRT > license is not compatible with Apache-2.0 ([Category > X|https://www.apache.org/legal/resolved.html#category-x]). > [https://devblogs.microsoft.com/cppblog/introducing-the-universal-crt/] (see > point 6. near the bottom) > Redistributable files list: > [https://docs.microsoft.com/en-us/legal/windows-sdk/redist] > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-8630) PutEmail fails to negotiate TLS1.2 with latest JDK.
Branko Peshevski created NIFI-8630:
--
Summary: PutEmail fails to negotiate TLS1.2 with latest JDK.
Key: NIFI-8630
URL: https://issues.apache.org/jira/browse/NIFI-8630
Project: Apache NiFi
Issue Type: Bug
Reporter: Branko Peshevski
On April 20, 2021 with the latest JDK the support for TLS 1.0 and 1.1 was
removed/disabled. Using the PutEmail processor with the latest OpenJdk and TLS
set to true is failing with:
{code:java}
Failed to send email for
StandardFlowFileRecord[uuid=7e8d3482-59ac-4993-a9ea-fa9089ced554,claim=,offset=0,name=7e8d3482-59ac-4993-a9ea-fa9089ced554,size=0]:
Could not convert socket to TLS; routing to failure:
javax.mail.MessagingException: Could not convert socket to TLS;
nested exception is:
javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol
is disabled or cipher suites are inappropriate)
javax.mail.MessagingException: Could not convert socket to TLS
at com.sun.mail.smtp.SMTPTransport.startTLS(SMTPTransport.java:1907)
at
com.sun.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:666)
at javax.mail.Service.connect(Service.java:317)
at javax.mail.Service.connect(Service.java:176)
at javax.mail.Service.connect(Service.java:125)
at javax.mail.Transport.send0(Transport.java:194)
at javax.mail.Transport.send(Transport.java:124)
at org.apache.nifi.processors.standard.PutEmail.send(PutEmail.java:541)
at
org.apache.nifi.processors.standard.PutEmail.onTrigger(PutEmail.java:395)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1173)
at
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:214)
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
{code}
>From my research the javax.mail version is quite old and it is known that
>1.4.x fails to renegotiate with better tls algorithm. But I think it would be
>better to add new Property Descriptor to the PutEmail processor that will
>default the ssl protocol to TLS1.2.
* Update the javax.mail dependency.
* Add PropertyDescriptor for setting the ssl algorithm that will default to
TLS 1.2
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1370) Clarify README to not guarantee backwards compatiblity in 0.x versions
[ https://issues.apache.org/jira/browse/MINIFICPP-1370?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1370: Fix Version/s: 0.9.0 > Clarify README to not guarantee backwards compatiblity in 0.x versions > -- > > Key: MINIFICPP-1370 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1370 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 1h 10m > Remaining Estimate: 0h > > 0.x versions are development versions with no compatibility guarantees > according to semver. README incorrectly suggests backwards compatibility with > minor versions even in 0.x. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[
https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351132#comment-17351132
]
Otto Fowler commented on NIFI-8625:
---
Strangely enough, I try to reproduce in the debugger, and it works:
{noformat}
2021-05-25 11:24:14,368 INFO [Timer-Driven Process Thread-10]
o.a.n.processors.standard.LogAttribute
LogAttribute[id=a402624a-0179-1000-3c1d-e423f684707b] logging for flow file
StandardFlowFileRecord[uuid=5a7bc46c-5a63-4a32-9fac-35913fa0c087,claim=StandardContentClaim
[resourceClaim=StandardResourceClaim[id=1621954150437-1, container=default,
section=1], offset=38, length=4],offset=0,name=1.txt,size=4]
--
Standard FlowFile Attributes
Key: 'entryDate'
Value: 'Tue May 25 11:24:00 EDT 2021'
Key: 'lineageStartDate'
Value: 'Tue May 25 11:24:00 EDT 2021'
Key: 'fileSize'
Value: '4'
FlowFile Attribute Map Content
Key: 'absolute.path'
Value: '/Users/ottofowler/tmp/nifi-input/'
Key: 'file.creationTime'
Value: '2021-05-25T10:45:26-0400'
Key: 'file.group'
Value: 'staff'
Key: 'file.lastAccessTime'
Value: '2021-05-25T10:49:10-0400'
Key: 'file.lastModifiedTime'
Value: '2021-05-25T10:45:34-0400'
Key: 'file.owner'
Value: 'ottofowler'
Key: 'file.permissions'
Value: 'rw-r--r--'
Key: 'file.size'
Value: '4'
Key: 'filename'
Value: '1.txt'
Key: 'path'
Value: './'
Key: 'userCountName'
Value: 'D:\99_TestCase\4_MultiFile_Stuck\B5_ALL_CELL_.txt'
Key: 'uuid'
Value: '5a7bc46c-5a63-4a32-9fac-35913fa0c087'
--
one
{noformat}
> ExecuteScript processor always stuck after restart or multi thread
> --
>
> Key: NIFI-8625
> URL: https://issues.apache.org/jira/browse/NIFI-8625
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.13.2
>Reporter: KevinSky
>Priority: Critical
> Labels: ExecuteScript,stcuk
> Attachments: executeScript_nifi_8625.xml,
> image-2021-05-21-16-22-34-775.png
>
>
> In single thread, executeScript just stop and start, flow file always stuck
> in connections.
> In multi thread.executeScript always throw exception like is already marked
> for transfer in or is not known in this session.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1362) Windows service control handler code log level should be trace
[ https://issues.apache.org/jira/browse/MINIFICPP-1362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1362: Fix Version/s: 0.9.0 > Windows service control handler code log level should be trace > -- > > Key: MINIFICPP-1362 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1362 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Fix For: 0.9.0 > > Time Spent: 20m > Remaining Estimate: 0h > > https://docs.microsoft.com/en-us/windows/win32/services/writing-a-control-handler-function > MiNiFi is logging the control code as INFO, which is spamming the event log. > It should be TRACE. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1383) intdiv_ceil gives incorrect results on negative input
[ https://issues.apache.org/jira/browse/MINIFICPP-1383?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1383: Fix Version/s: 0.9.0 > intdiv_ceil gives incorrect results on negative input > - > > Key: MINIFICPP-1383 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1383 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Trivial > Fix For: 0.9.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1408) All thrown exceptions should be derived from std::exception
[ https://issues.apache.org/jira/browse/MINIFICPP-1408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1408: Fix Version/s: 0.9.0 > All thrown exceptions should be derived from std::exception > --- > > Key: MINIFICPP-1408 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1408 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 1h 20m > Remaining Estimate: 0h > > So that catch-all logs are able to log std::exception::what(). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1413) Improve exception logging
[ https://issues.apache.org/jira/browse/MINIFICPP-1413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1413: Fix Version/s: 0.9.0 > Improve exception logging > - > > Key: MINIFICPP-1413 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1413 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Labels: MiNiFi-CPP-Hygiene > Fix For: 0.9.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > 1. No empty std::exceptions > 2. Add typeid to catch-all logging when possible -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1398) EL date functions are not available on Windows
[
https://issues.apache.org/jira/browse/MINIFICPP-1398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz updated MINIFICPP-1398:
Fix Version/s: 0.9.0
> EL date functions are not available on Windows
> --
>
> Key: MINIFICPP-1398
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1398
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Major
> Fix For: 0.9.0
>
> Time Spent: 1.5h
> Remaining Estimate: 0h
>
> in Expression.h:
> {{#define EXPRESSION_LANGUAGE_USE_DATE}}
> {{// Disable date in EL for incompatible compilers}}
> {{#if \_\_GNUC\_\_ < 5}}
> {{#undef EXPRESSION_LANGUAGE_USE_DATE}}
> {{#endif}}
> I assume \_\_GNUC\_\_ is evaluated as 0 on Windows and that's the cause.
> update: due to problems with the date and timezone libraries on Windows, I'm
> going to add a fallback based on strftime for now.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1417) Improve FileStream error reporting
[ https://issues.apache.org/jira/browse/MINIFICPP-1417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1417: Fix Version/s: 0.9.0 > Improve FileStream error reporting > -- > > Key: MINIFICPP-1417 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1417 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Major > Fix For: 0.9.0 > > Time Spent: 40m > Remaining Estimate: 0h > > In some deployments, "Unknown exception" errors were reported, usually after > flow update. It's caused by a narrowing error from FileStream that occurs > because of incorrect error handling. The goal is to fix this bug and improve > error reporting in FileStream so that future errors don't manifest themselves > as "Unknown exception". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (MINIFICPP-1484) make linter doesn't work in a source tarball
[ https://issues.apache.org/jira/browse/MINIFICPP-1484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marton Szasz updated MINIFICPP-1484: Fix Version/s: 0.9.0 > make linter doesn't work in a source tarball > > > Key: MINIFICPP-1484 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1484 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Marton Szasz >Assignee: Marton Szasz >Priority: Blocker > Fix For: 0.9.0 > > Time Spent: 20m > Remaining Estimate: 0h > > make linter reports false positives when run from a source directory that > doesn't contain .git. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1563) Fix Wmaybe-uninitialized warnings (tested on GCC 10.3 and 11.1)
[
https://issues.apache.org/jira/browse/MINIFICPP-1563?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marton Szasz resolved MINIFICPP-1563.
-
Fix Version/s: 0.10.0
Resolution: Fixed
> Fix Wmaybe-uninitialized warnings (tested on GCC 10.3 and 11.1)
> ---
>
> Key: MINIFICPP-1563
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1563
> Project: Apache NiFi MiNiFi C++
> Issue Type: Bug
>Reporter: Marton Szasz
>Assignee: Marton Szasz
>Priority: Trivial
> Fix For: 0.10.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> {color:#00}../extensions/mqtt/../../libminifi/include/c2/PayloadSerializer.h:{color}{color:#00}
> In function ‘{color}{color:#00}static bool
> org::apache::nifi::minifi::c2::PayloadSerializer::deserializePayload(org::apache::nifi::minifi::c2::C2Payload&,
> org::apache::nifi::minifi::c2::Operation, std::string,
> org::apache::nifi::minifi::io::BaseStream*){color}{color:#00}’: {color}
> {color:#00}../extensions/mqtt/../../libminifi/include/c2/PayloadSerializer.h:233:111:{color}
> {color:#ff5454}error:
> {color}{color:#00}‘{color}{color:#00}st{color}{color:#00}’ may be
> used uninitialized in this function
> [{color}{color:#ff5454}-Werror=maybe-uninitialized{color}{color:#00}]
> {color}
> 233 \| C2Payload subPayload(operation, st == 1 ?
> state::UpdateState::NESTED :
> state::UpdateState::READ_COMPLETE{color:#ff5454}){color}{color:#00};
> {color}
> \|
> {color:#ff5454}^{color}
> {color:#00}../extensions/mqtt/../../libminifi/include/c2/PayloadSerializer.h:303:5:{color}
> {color:#ff5454}error:
> {color}{color:#00}‘{color}{color:#00}op{color}{color:#00}’ may be
> used uninitialized in this function
> [{color}{color:#ff5454}-Werror=maybe-uninitialized{color}{color:#00}]
> {color}
> 303 \| {color:#ff5454}switch{color}{color:#00} (op) { {color}
> \| {color:#ff5454}^~{color}
> {color:#00}../extensions/mqtt/../../libminifi/include/c2/PayloadSerializer.h:225:13:{color}
> {color:#54}note:
> {color}{color:#00}‘{color}{color:#00}op{color}{color:#00}’ was
> declared here {color}
> 225 \| uint8_t {color:#54}op{color}{color:#00}, st; {color}
> \| {color:#54}^~{color}
> {color:#00}../extensions/mqtt/../../libminifi/include/c2/PayloadSerializer.h:{color}{color:#00}
> In member function ‘{color}{color:#00}virtual void
> org::apache::nifi::minifi::processors::ConvertHeartBeat::onTrigger(const
> std::shared_ptr&, const
> std::shared_ptr&){color}{color:#00}’:
> {color}
> {color:#00}../extensions/mqtt/../../libminifi/include/c2/PayloadSerializer.h:303:5:{color}
> {color:#ff5454}error:
> {color}{color:#00}‘{color}{color:#00}op{color}{color:#00}’ may be
> used uninitialized in this function
> [{color}{color:#ff5454}-Werror=maybe-uninitialized{color}{color:#00}]
> {color}
> 303 \| {color:#ff5454}switch{color}{color:#00} (op) { {color}
> \| {color:#ff5454}^~{color}
> {color:#00}../extensions/mqtt/../../libminifi/include/c2/PayloadSerializer.h:260:13:{color}
> {color:#54}note:
> {color}{color:#00}‘{color}{color:#00}op{color}{color:#00}’ was
> declared here {color}
> 260 \| uint8_t {color:#54}op{color}{color:#00}, st = 0; {color}
> \| {color:#54}^~{color}
>
> and more...
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1077: MINIFICPP-1563 fix -Wmaybe-uninitialized warnings
fgerlits closed pull request #1077: URL: https://github.com/apache/nifi-minifi-cpp/pull/1077 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits closed pull request #1069: MINIFICPP-1084: Linter check should be platform independent
fgerlits closed pull request #1069: URL: https://github.com/apache/nifi-minifi-cpp/pull/1069 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1081: MINIFICPP-1565: Minor improvements to PerformanceDataMonitor
szaszm commented on a change in pull request #1081:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1081#discussion_r638895278
##
File path: libminifi/include/utils/MathUtils.h
##
@@ -0,0 +1,43 @@
+/**
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+#pragma once
+
+#include
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+class MathUtils {
+ public:
+ static double round_to(double original, int8_t precision) {
+if (precision < 0) {
+ return original;
Review comment:
Rounding to fractional decimal places like 10^-2 => 123.45 makes sense
as well. Either implement the full expected behavior or move the implementation
to somewhere local near the usage, so it's clear that this logic is related to
the performance monitor functionality.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Assigned] (NIFI-8616) Migrate SSL socket code to use Netty
[ https://issues.apache.org/jira/browse/NIFI-8616?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nathan Gough reassigned NIFI-8616: -- Assignee: Nathan Gough > Migrate SSL socket code to use Netty > > > Key: NIFI-8616 > URL: https://issues.apache.org/jira/browse/NIFI-8616 > Project: Apache NiFi > Issue Type: Improvement > Components: Security >Affects Versions: 1.13.2 >Reporter: Nathan Gough >Assignee: Nathan Gough >Priority: Major > Fix For: 1.14.0 > > > The PutUDP/PutTCP and any other processors using SSLSocketChannel should be > migrated to use Netty library code rather than maintain our custom > implementation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] szaszm commented on a change in pull request #1081: MINIFICPP-1565: Minor improvements to PerformanceDataMonitor
szaszm commented on a change in pull request #1081:
URL: https://github.com/apache/nifi-minifi-cpp/pull/1081#discussion_r638890480
##
File path: extensions/pdh/PerformanceDataMonitor.cpp
##
@@ -45,9 +45,15 @@ core::Property PerformanceDataMonitor::CustomPDHCounters(
core::Property PerformanceDataMonitor::OutputFormatProperty(
core::PropertyBuilder::createProperty("Output Format")->
withDescription("Format of the created flowfiles")->
-withAllowableValue(JSON_FORMAT_STR)->
-withAllowableValue(OPEN_TELEMETRY_FORMAT_STR)->
-withDefaultValue(JSON_FORMAT_STR)->build());
+withAllowableValue(PRETTY_JSON_FORMAT_STR)->
+withAllowableValue(COMPACT_JSON_FORMAT_STR)->
+withAllowableValue(PRETTY_OPEN_TELEMETRY_FORMAT_STR)->
+withAllowableValue(COMPACT_OPEN_TELEMETRY_FORMAT_STR)->
+withDefaultValue(PRETTY_JSON_FORMAT_STR)->build());
+
+core::Property PerformanceDataMonitor::DoublePrecisionProperty(
+ core::PropertyBuilder::createProperty("Double Precision")->
+ withDescription("Rounds the double values to this precision (blank for
maximum precision)")->build());
Review comment:
What does precision mean in this context? Some power of 10 decimal
place? Or power of 2? Or number of significant decimal digits? Or ...
This could use some more precise description.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Commented] (NIFI-8462) Refactor PutSyslog and ListenSyslog using Netty
[
https://issues.apache.org/jira/browse/NIFI-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351114#comment-17351114
]
ASF subversion and git services commented on NIFI-8462:
---
Commit a3365c883359d8a06c0ef4e9b370aa83e7d6c67b in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a3365c8 ]
NIFI-8462 Refactored PutSyslog and ListenSyslog using Netty
- Added nifi-event-transport module encapsulating Netty classes
- Refactored unit tests for PutSyslog and ListenSyslog
- Removed integration tests for PutSyslog and ListenSyslog
NIFI-8462 Added context.yield() in PutSyslog when no FlowFiles and addressed
other issues
NIFI-8462 Removed unused import of ExpressionLanguageScope
Signed-off-by: Nathan Gough
This closes #5044.
> Refactor PutSyslog and ListenSyslog using Netty
> ---
>
> Key: NIFI-8462
> URL: https://issues.apache.org/jira/browse/NIFI-8462
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.13.2
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Labels: Netty, SSLEngine, security
> Fix For: 1.14.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Multiple extension and framework components leverage custom socket handling
> classes for sending and receiving messages. These custom classes include
> {{SSLSocketChannel}}, which handles handshaking and interaction with
> {{SSLEngine}}. Custom {{SSLEngine}} handling is prone to errors, and the
> current implementation has issues with TLS 1.3. Rather than continuing to
> maintain custom {{SSLEngine}} handling, moving to a solution based on Netty
> provides a cleaner approach.
> The purpose of this issue is to refactor {{PutSyslog}} and {{ListenSyslog}}
> using Netty to provide a way forward for other impacted components. The
> changes should maintain compatibility with existing TCP and UDP protocol
> configurations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8462) Refactor PutSyslog and ListenSyslog using Netty
[
https://issues.apache.org/jira/browse/NIFI-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351115#comment-17351115
]
ASF subversion and git services commented on NIFI-8462:
---
Commit a3365c883359d8a06c0ef4e9b370aa83e7d6c67b in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a3365c8 ]
NIFI-8462 Refactored PutSyslog and ListenSyslog using Netty
- Added nifi-event-transport module encapsulating Netty classes
- Refactored unit tests for PutSyslog and ListenSyslog
- Removed integration tests for PutSyslog and ListenSyslog
NIFI-8462 Added context.yield() in PutSyslog when no FlowFiles and addressed
other issues
NIFI-8462 Removed unused import of ExpressionLanguageScope
Signed-off-by: Nathan Gough
This closes #5044.
> Refactor PutSyslog and ListenSyslog using Netty
> ---
>
> Key: NIFI-8462
> URL: https://issues.apache.org/jira/browse/NIFI-8462
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.13.2
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Labels: Netty, SSLEngine, security
> Fix For: 1.14.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Multiple extension and framework components leverage custom socket handling
> classes for sending and receiving messages. These custom classes include
> {{SSLSocketChannel}}, which handles handshaking and interaction with
> {{SSLEngine}}. Custom {{SSLEngine}} handling is prone to errors, and the
> current implementation has issues with TLS 1.3. Rather than continuing to
> maintain custom {{SSLEngine}} handling, moving to a solution based on Netty
> provides a cleaner approach.
> The purpose of this issue is to refactor {{PutSyslog}} and {{ListenSyslog}}
> using Netty to provide a way forward for other impacted components. The
> changes should maintain compatibility with existing TCP and UDP protocol
> configurations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-8462) Refactor PutSyslog and ListenSyslog using Netty
[
https://issues.apache.org/jira/browse/NIFI-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Nathan Gough updated NIFI-8462:
---
Fix Version/s: 1.14.0
Resolution: Resolved
Status: Resolved (was: Patch Available)
> Refactor PutSyslog and ListenSyslog using Netty
> ---
>
> Key: NIFI-8462
> URL: https://issues.apache.org/jira/browse/NIFI-8462
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.13.2
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Labels: Netty, SSLEngine, security
> Fix For: 1.14.0
>
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Multiple extension and framework components leverage custom socket handling
> classes for sending and receiving messages. These custom classes include
> {{SSLSocketChannel}}, which handles handshaking and interaction with
> {{SSLEngine}}. Custom {{SSLEngine}} handling is prone to errors, and the
> current implementation has issues with TLS 1.3. Rather than continuing to
> maintain custom {{SSLEngine}} handling, moving to a solution based on Netty
> provides a cleaner approach.
> The purpose of this issue is to refactor {{PutSyslog}} and {{ListenSyslog}}
> using Netty to provide a way forward for other impacted components. The
> changes should maintain compatibility with existing TCP and UDP protocol
> configurations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-8462) Refactor PutSyslog and ListenSyslog using Netty
[
https://issues.apache.org/jira/browse/NIFI-8462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351113#comment-17351113
]
ASF subversion and git services commented on NIFI-8462:
---
Commit a3365c883359d8a06c0ef4e9b370aa83e7d6c67b in nifi's branch
refs/heads/main from David Handermann
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=a3365c8 ]
NIFI-8462 Refactored PutSyslog and ListenSyslog using Netty
- Added nifi-event-transport module encapsulating Netty classes
- Refactored unit tests for PutSyslog and ListenSyslog
- Removed integration tests for PutSyslog and ListenSyslog
NIFI-8462 Added context.yield() in PutSyslog when no FlowFiles and addressed
other issues
NIFI-8462 Removed unused import of ExpressionLanguageScope
Signed-off-by: Nathan Gough
This closes #5044.
> Refactor PutSyslog and ListenSyslog using Netty
> ---
>
> Key: NIFI-8462
> URL: https://issues.apache.org/jira/browse/NIFI-8462
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
>Affects Versions: 1.13.2
>Reporter: David Handermann
>Assignee: David Handermann
>Priority: Major
> Labels: Netty, SSLEngine, security
> Time Spent: 3.5h
> Remaining Estimate: 0h
>
> Multiple extension and framework components leverage custom socket handling
> classes for sending and receiving messages. These custom classes include
> {{SSLSocketChannel}}, which handles handshaking and interaction with
> {{SSLEngine}}. Custom {{SSLEngine}} handling is prone to errors, and the
> current implementation has issues with TLS 1.3. Rather than continuing to
> maintain custom {{SSLEngine}} handling, moving to a solution based on Netty
> provides a cleaner approach.
> The purpose of this issue is to refactor {{PutSyslog}} and {{ListenSyslog}}
> using Netty to provide a way forward for other impacted components. The
> changes should maintain compatibility with existing TCP and UDP protocol
> configurations.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] thenatog closed pull request #5044: NIFI-8462 Refactored PutSyslog and ListenSyslog using Netty
thenatog closed pull request #5044: URL: https://github.com/apache/nifi/pull/5044 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-8625) ExecuteScript processor always stuck after restart or multi thread
[
https://issues.apache.org/jira/browse/NIFI-8625?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17351112#comment-17351112
]
Otto Fowler commented on NIFI-8625:
---
Thanks!
I just ran the template against main:
* change input directory
* remove filter
* input has 4 text files
* change log message ( success ) to logattributes
I am seeing exceptions in the nifi-app.log that are rolling back the session.
Can you see if you see those as well?
{noformat}
2021-05-25 10:53:01,402 ERROR [Timer-Driven Process Thread-9]
o.a.nifi.processors.script.ExecuteScript
ExecuteScript[id=bb911eff-0bbc-346e-6a34-3875ae197ff0]
ExecuteScript[id=bb911eff-0bbc-346e-6a34-3875ae197ff0] failed to process due to
org.apache.nifi.processor.exception.ProcessException:
javax.script.ScriptException: java.lang.NullPointerException:
java.lang.NullPointerException in
[GitHub] [nifi] thenatog commented on pull request #5044: NIFI-8462 Refactored PutSyslog and ListenSyslog using Netty
thenatog commented on pull request #5044: URL: https://github.com/apache/nifi/pull/5044#issuecomment-847937054 +1 will merge. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1454) Reduce duplication of CMake parameters in docker arguments
[ https://issues.apache.org/jira/browse/MINIFICPP-1454?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gabor Gyimesi resolved MINIFICPP-1454. -- Resolution: Fixed > Reduce duplication of CMake parameters in docker arguments > -- > > Key: MINIFICPP-1454 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1454 > Project: Apache NiFi MiNiFi C++ > Issue Type: Improvement >Reporter: Gabor Gyimesi >Assignee: Gabor Gyimesi >Priority: Minor > Labels: MiNiFi-CPP-Hygiene > Time Spent: 1h 10m > Remaining Estimate: 0h > > MINIFICPP-1452 introduced the ability for Docker builds to consume the > configured CMake options. This way CMake options set regarding extension > availability are passed to Docker builds when the docker target is built. > Unfortunately these options need to be defined and passed through > DockerConfig.cmake, DockerBuild.sh and finally the Dockerfile, which causes > duplication of these arguments. We need to find a way to reduced this > duplication. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1360) Make scheduling strategy configurable in integration tests
[
https://issues.apache.org/jira/browse/MINIFICPP-1360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gabor Gyimesi resolved MINIFICPP-1360.
--
Assignee: Gabor Gyimesi
Resolution: Fixed
With the new design all processors can get a
schedule=\{'scheduling strategy': 'EVENT_DRIVEN'} parameter to change the
default scheduling strategy if needed.
> Make scheduling strategy configurable in integration tests
> --
>
> Key: MINIFICPP-1360
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1360
> Project: Apache NiFi MiNiFi C++
> Issue Type: Improvement
>Reporter: Gabor Gyimesi
>Assignee: Gabor Gyimesi
>Priority: Minor
> Time Spent: 20m
> Remaining Estimate: 0h
>
> In docker/test/integration/minifi/__init__.py the scheduling strategy is hard
> coded to be TIMER_DRIVEN for all processors. This should be configurable and
> updated for all processors used by integration tests.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
