Hi Gerard! Why not use bcp in/out command directly with Airflow? We currently wrap bcp in/out command into an Hook/Operator for working with SQL Server. Obiously you need to install the tools in airflow workers.
Cheers, Philip El 31-05-2019, a la(s) 09:57, Gerard Toonstra <[email protected]> escribió: > Hi all, > > We're running airflow and write data to sqlserver. Python and sqlserver > isn't a marriage made in heaven and we've literally spent hours fixing > driver issues, installations and working around things that should be easy, > time that's better spent writing app code. > > I looked at Sqoop, but that runs on Hadoop only. I didn't find anything > that was opensource / standalone to use as a "generic data pump", so > started a new project for that (let me know if something does exist). The > overall idea is that an airflow operator generates a command file for this > process and then spawns a java process to run the pipeline. > > It's available here: > > https://github.com/gtoonstra/genevere > > If you have similar issues with your environment, maybe this helps you out. > And if you want to contribute, just reach out! > > Rgds, > > Gerard
