absolutely! Once we have these plugins grouped in modules, it will be a lot easier to clean them up and reduce code duplication.
On Tue, Mar 18, 2025 at 4:49 PM Hans Van Akelyen <hans.van.akel...@gmail.com> wrote: > Hi All, > > I have been pondering a bit and we have quite some modules that that > actually need each other to work properly. > We might have been a bit overzealous when splitting everything in separate > modules in the project. > I would like to propose combining following actions/transforms and love to > hear your thoughts. > > Move Actions/Transforms related to a specific database type to the > plugins/database. > eg. > hop-transform-pgbulkloader -> hop-databases-postgresql > > hop-action-mysqlbulkfile -> hop-databases-mysql > hop-action-mysqlbulkload -> hop-databases-mysql > hop-transform-mysqlbulkloader -> hop-databases-mysql > ... > > Combine File related transforms into a single module: > > hop-transform-fileexists -> hop-transform-files > hop-transform-filelocked -> hop-transform-files > hop-transform-filemetadata -> hop-transform-files > hop-transform-filesfromresult -> hop-transform-files > hop-transform-filestoresult -> hop-transform-files > hop-transform-getfilenames -> hop-transform-files > hop-transform-getfilesrowcount -> hop-transform-files > hop-transform-getsubfolders -> hop-transform-files > hop-transform-processfiles -> hop-transform-files > > Same for actions > > We also have some in/outputs of the same type split into separate modules > and might also make more sense to combine these: > eg. > hop-transform-cubeinput -> hop-transform-cube > hop-transform-cubeoutput -> hop-transform-cube > > I would like to make more logical groups and shared code in the > transforms/actions/databases. > This will make our build and deployment more simple and faster as we are > losing a lot of time on all these very small modules. > It also has the benefit of users being able to remove "groups" of > functionality. > > Let me know what you think, if we have consensus I'll draw up a full list > which can be used for further discussion. > > Hans >
