Magnus Hagander wrote:
Magnus, any chance of getting a look at what you currently
have to help guide me in the appropriate direction?
Umm. That would kind of assume it's written down in a comprehensible
way. Which it isn't, of course :-)
The general ideas so far have been, off the top of my head:
* Pluginnable set of "readers" and "writers". Originally I'd see
postgresql, odbc, xml and possibly csv. Pg driver would be optimised to
use COPY when available.
* Pluginnable set of "transforms" that would operate on the rows. By
default things like copy and concatenate and maybe regexp. Future
enhancement would be a python extension, as Dave mentioned. (Or really,
anything else)
* I was envisioning a split of say "package", "job", "step" (terms of
course subject to discussion). package basically a set of job, job a set
of steps. Things like connections would be defined at the "job" level,
along wiht parmaeters for transaction control etc. (So you can use it to
transfer 10 different tables within a single transaction, something I
need all the time).
* I'd like to see the job format stored as XML with a well defined
schema, so different appliations can generate it - both manually
(GUI-wise from pgadmin and phppgadmin etc) and automatically.
* The "engine" should be available both as a commandline tool (which
must not require X libraries etc, because it should be deployable
"everywhere") and as acommand inside pgadmin (like MS DTS)
Um. I think that's about it. I had some sketches of classes and
interfaces around (not complete, but an idea), but I can't find them :(
This sounds like an awful lot of work.
A somewhat reduced version (IIRC Dave and me discussed something like
the following briefly) a more raw import (maybe into temp tables) could
be a big step, giving the admin the chance to create views on that
tables that do the extractions he likes. PostgreSQL already has all
functions you'd like, no need to reimplement them.
Regards,
Andreas
---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?
http://www.postgresql.org/docs/faq
