[jira] [Created] (NIFI-12351) Enhance Minifi Java agent
Robert Liszli created NIFI-12351: Summary: Enhance Minifi Java agent Key: NIFI-12351 URL: https://issues.apache.org/jira/browse/NIFI-12351 Project: Apache NiFi Issue Type: Improvement Reporter: Robert Liszli Fix For: 2.0.0 *Includes* * Fix start/stop bat script * Fix install-service.bat, needs to update Apache Commons Daemon version * check/fix delete-service.bat, status-minifi.bat, install-service.bat * Check and eliminate strange log files in log directory -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-11142) Security fix for SnakeYAML
[ https://issues.apache.org/jira/browse/NIFI-11142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Liszli resolved NIFI-11142. -- Resolution: Not A Problem The questionable Constructor is not used in the codebase. > Security fix for SnakeYAML > -- > > Key: NIFI-11142 > URL: https://issues.apache.org/jira/browse/NIFI-11142 > Project: Apache NiFi > Issue Type: Improvement > Components: MiNiFi >Reporter: Robert Liszli >Assignee: Robert Liszli >Priority: Minor > > *Fix for:* > SnakeYaml's Constructor() class does not restrict types which can be > instantiated during deserialization. Deserializing yaml content provided by > an attacker can lead to remote code execution. We recommend using SnakeYaml's > SafeConsturctor when parsing untrusted content to restrict deserialization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-11142) Security fix for SnakeYAML
[ https://issues.apache.org/jira/browse/NIFI-11142?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17684698#comment-17684698 ] Robert Liszli commented on NIFI-11142: -- Thanks for the info. Checked the code and you're right. I close this ticket. > Security fix for SnakeYAML > -- > > Key: NIFI-11142 > URL: https://issues.apache.org/jira/browse/NIFI-11142 > Project: Apache NiFi > Issue Type: Improvement > Components: MiNiFi >Reporter: Robert Liszli >Assignee: Robert Liszli >Priority: Minor > Fix For: 1.20.0 > > > *Fix for:* > SnakeYaml's Constructor() class does not restrict types which can be > instantiated during deserialization. Deserializing yaml content provided by > an attacker can lead to remote code execution. We recommend using SnakeYaml's > SafeConsturctor when parsing untrusted content to restrict deserialization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-11142) Security fix for SnakeYAML
[ https://issues.apache.org/jira/browse/NIFI-11142?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Liszli reassigned NIFI-11142: Assignee: Robert Liszli > Security fix for SnakeYAML > -- > > Key: NIFI-11142 > URL: https://issues.apache.org/jira/browse/NIFI-11142 > Project: Apache NiFi > Issue Type: Improvement > Components: MiNiFi >Reporter: Robert Liszli >Assignee: Robert Liszli >Priority: Minor > Fix For: 1.20.0 > > > *Fix for:* > SnakeYaml's Constructor() class does not restrict types which can be > instantiated during deserialization. Deserializing yaml content provided by > an attacker can lead to remote code execution. We recommend using SnakeYaml's > SafeConsturctor when parsing untrusted content to restrict deserialization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-11142) Security fix for SnakeYAML
Robert Liszli created NIFI-11142: Summary: Security fix for SnakeYAML Key: NIFI-11142 URL: https://issues.apache.org/jira/browse/NIFI-11142 Project: Apache NiFi Issue Type: Improvement Components: MiNiFi Reporter: Robert Liszli Fix For: 1.20.0 *Fix for:* SnakeYaml's Constructor() class does not restrict types which can be instantiated during deserialization. Deserializing yaml content provided by an attacker can lead to remote code execution. We recommend using SnakeYaml's SafeConsturctor when parsing untrusted content to restrict deserialization. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Resolved] (NIFI-10556) Create processor to support DeltaLake tables
[ https://issues.apache.org/jira/browse/NIFI-10556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Liszli resolved NIFI-10556. -- Resolution: Won't Do > Create processor to support DeltaLake tables > > > Key: NIFI-10556 > URL: https://issues.apache.org/jira/browse/NIFI-10556 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Robert Liszli >Assignee: Robert Liszli >Priority: Major > Attachments: processor_usages.png > > Time Spent: 4h 10m > Remaining Estimate: 0h > > *Plan for the new processor* > The new processor will use the Delta Standalone library to generate the delta > table for the parquet data files. This processor also capable to process > other processors output file and upload it to the data store. > *Processors input:* > * The path of the parquet files(a single directory). Located at local > filesystem or in cloud storage(S3, GCP or Azure). > * Structure of the parquet file in json format. > * If we want the processor to process other processors output file, then the > attribute names of the output files path and filename should be set > * Partition columns, separated by comma > *Processors parameter:* > * Dropdown selector for storage type selection. > * Credentials for the selected storage type. > *On Trigger:* > * If we want the processor to process other processors output file, first it > copies the new file to the desired data directory. > * The processor will compare the files in the data directory to the files > already added to the delta table. If new data file exist, it will add it to > the delta table. > * If there is no delta table exists, the processor will create one and the > delta table will be generated. > *Output of the processor:* > * Up to date Delta table in the chosen storage system. > > Delta Standalone: [https://github.com/delta-io/connectors#delta-standalone] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10556) Create processor to support DeltaLake tables
[ https://issues.apache.org/jira/browse/NIFI-10556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Liszli updated NIFI-10556: - Description: *Plan for the new processor* The new processor will use the Delta Standalone library to generate the delta table for the parquet data files. This processor also capable to process other processors output file and upload it to the data store. *Processors input:* * The path of the parquet files(a single directory). Located at local filesystem or in cloud storage(S3, GCP or Azure). * Structure of the parquet file in json format. * If we want the processor to process other processors output file, then the attribute names of the output files path and filename should be set * Partition columns, separated by comma *Processors parameter:* * Dropdown selector for storage type selection. * Credentials for the selected storage type. *On Trigger:* * If we want the processor to process other processors output file, first it copies the new file to the desired data directory. * The processor will compare the files in the data directory to the files already added to the delta table. If new data file exist, it will add it to the delta table. * If there is no delta table exists, the processor will create one and the delta table will be generated. *Output of the processor:* * Up to date Delta table in the chosen storage system. Delta Standalone: [https://github.com/delta-io/connectors#delta-standalone] was: *Plan for the new processor* The new processor will use the Delta Standalone library to generate delta table for a set of parquet data files located locally or in cloud storage. *Processors input:* * The path of the parquet files(a single directory). Located at local filesystem or in cloud storage(S3, GCP or Azure). * Structure of the parquet file in json format. *Processors parameter:* * Dropdown selector for storage type selection. * Credentials for the selected storage type. *On Trigger:* * The processor will compare the files in the data directory to the files already added to the delta table. If new data file exist, it will add it to the delta table. * If there is no delta table exists, the processor will create one and the delta table will be generated. *Output of the processor:* * Up to date Delta table in the chosen storage system. Delta Standalone: [https://github.com/delta-io/connectors#delta-standalone] > Create processor to support DeltaLake tables > > > Key: NIFI-10556 > URL: https://issues.apache.org/jira/browse/NIFI-10556 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Robert Liszli >Assignee: Robert Liszli >Priority: Major > Attachments: processor_usages.png > > Time Spent: 1h > Remaining Estimate: 0h > > *Plan for the new processor* > The new processor will use the Delta Standalone library to generate the delta > table for the parquet data files. This processor also capable to process > other processors output file and upload it to the data store. > *Processors input:* > * The path of the parquet files(a single directory). Located at local > filesystem or in cloud storage(S3, GCP or Azure). > * Structure of the parquet file in json format. > * If we want the processor to process other processors output file, then the > attribute names of the output files path and filename should be set > * Partition columns, separated by comma > *Processors parameter:* > * Dropdown selector for storage type selection. > * Credentials for the selected storage type. > *On Trigger:* > * If we want the processor to process other processors output file, first it > copies the new file to the desired data directory. > * The processor will compare the files in the data directory to the files > already added to the delta table. If new data file exist, it will add it to > the delta table. > * If there is no delta table exists, the processor will create one and the > delta table will be generated. > *Output of the processor:* > * Up to date Delta table in the chosen storage system. > > Delta Standalone: [https://github.com/delta-io/connectors#delta-standalone] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (NIFI-10556) Create processor to support DeltaLake tables
[ https://issues.apache.org/jira/browse/NIFI-10556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Liszli updated NIFI-10556: - Attachment: processor_usages.png > Create processor to support DeltaLake tables > > > Key: NIFI-10556 > URL: https://issues.apache.org/jira/browse/NIFI-10556 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Robert Liszli >Assignee: Robert Liszli >Priority: Major > Attachments: processor_usages.png > > Time Spent: 1h > Remaining Estimate: 0h > > *Plan for the new processor* > The new processor will use the Delta Standalone library to generate the delta > table for the parquet data files. This processor also capable to process > other processors output file and upload it to the data store. > *Processors input:* > * The path of the parquet files(a single directory). Located at local > filesystem or in cloud storage(S3, GCP or Azure). > * Structure of the parquet file in json format. > * If we want the processor to process other processors output file, then the > attribute names of the output files path and filename should be set > * Partition columns, separated by comma > *Processors parameter:* > * Dropdown selector for storage type selection. > * Credentials for the selected storage type. > *On Trigger:* > * If we want the processor to process other processors output file, first it > copies the new file to the desired data directory. > * The processor will compare the files in the data directory to the files > already added to the delta table. If new data file exist, it will add it to > the delta table. > * If there is no delta table exists, the processor will create one and the > delta table will be generated. > *Output of the processor:* > * Up to date Delta table in the chosen storage system. > > Delta Standalone: [https://github.com/delta-io/connectors#delta-standalone] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (NIFI-10556) Create processor to support DeltaLake tables
Robert Liszli created NIFI-10556: Summary: Create processor to support DeltaLake tables Key: NIFI-10556 URL: https://issues.apache.org/jira/browse/NIFI-10556 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Robert Liszli Assignee: Robert Liszli *Plan for the new processor* The new processor will use the Delta Standalone library to generate delta table for a set of parquet data files located locally or in cloud storage. *Processors input:* * The path of the parquet files(a single directory). Located at local filesystem or in cloud storage(S3, GCP or Azure). * Structure of the parquet file in json format. *Processors parameter:* * Dropdown selector for storage type selection. * Credentials for the selected storage type. *On Trigger:* * The processor will compare the files in the data directory to the files already added to the delta table. If new data file exist, it will add it to the delta table. * If there is no delta table exists, the processor will create one and the delta table will be generated. *Output of the processor:* * Up to date Delta table in the chosen storage system. Delta Standalone: [https://github.com/delta-io/connectors#delta-standalone] -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (NIFI-10312) Broken flowid handling in minifi-c2 service
[ https://issues.apache.org/jira/browse/NIFI-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17577509#comment-17577509 ] Robert Liszli commented on NIFI-10312: -- PR: https://github.com/apache/nifi/pull/6281 > Broken flowid handling in minifi-c2 service > --- > > Key: NIFI-10312 > URL: https://issues.apache.org/jira/browse/NIFI-10312 > Project: Apache NiFi > Issue Type: Bug > Components: C2, MiNiFi >Reporter: Csaba Bejan >Assignee: Robert Liszli >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > When MiNiFi is communication with minifi-c2 service via the C2 protocol the > flowId parsing seems to be broken so the publishing mechanism is not working. > Pattern needs to be updated to match the expected url structure or flow id > needs to be sent in a more robust way as current approach is error prone. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (NIFI-10312) Broken flowid handling in minifi-c2 service
[ https://issues.apache.org/jira/browse/NIFI-10312?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Robert Liszli reassigned NIFI-10312: Assignee: Robert Liszli > Broken flowid handling in minifi-c2 service > --- > > Key: NIFI-10312 > URL: https://issues.apache.org/jira/browse/NIFI-10312 > Project: Apache NiFi > Issue Type: Bug > Components: C2, MiNiFi >Reporter: Csaba Bejan >Assignee: Robert Liszli >Priority: Major > > When MiNiFi is communication with minifi-c2 service via the C2 protocol the > flowId parsing seems to be broken so the publishing mechanism is not working. > Pattern needs to be updated to match the expected url structure or flow id > needs to be sent in a more robust way as current approach is error prone. -- This message was sent by Atlassian Jira (v8.20.10#820010)
