GitHub user zoomingrocket edited a comment on the discussion: 2.18 roadmap and goals
Some more suggestions based on our current use 1. Standardize SFTP support in FTP Delete to support Public Key Authentication. 2. Move file shape for SFTP / FTP 3. Insert/Update Transform - Support Bulk Pattern by offloading the insert vs update to the database instead of external lookup row by row, slowing down the entire transform. Current workaround is to spool a local lookup cache, but that becomes a burden when we deal with 1B+ rows 4. Produce base metrics for each pipeline run via hoprun as an output to keep track of total rows processed. GitHub link: https://github.com/apache/hop/discussions/6445#discussioncomment-15610652 ---- This is an automatically sent email for [email protected]. To unsubscribe, please send an email to: [email protected]
