Re: [pgadmin-hackers] Import from CSV - Questions
> > Umm. That would kind of assume it's written down in a > comprehensible > > way. Which it isn't, of course :-) > > > > The general ideas so far have been, off the top of my head: > > > > * Pluginnable set of "readers" and "writers". Originally I'd see > > postgresql, odbc, xml and possibly csv. Pg driver would be > optimised > > to use COPY when available. > > > > * Pluginnable set of "transforms" that would operate on the > rows. By > > default things like copy and concatenate and maybe regexp. Future > > enhancement would be a python extension, as Dave mentioned. (Or > > really, anything else) > > > > * I was envisioning a split of say "package", "job", "step" > (terms of > > course subject to discussion). package basically a set of > job, job a > > set of steps. Things like connections would be defined at the "job" > > level, along wiht parmaeters for transaction control etc. > (So you can > > use it to transfer 10 different tables within a single transaction, > > something I need all the time). > > > > * I'd like to see the job format stored as XML with a well defined > > schema, so different appliations can generate it - both manually > > (GUI-wise from pgadmin and phppgadmin etc) and automatically. > > > > * The "engine" should be available both as a commandline > tool (which > > must not require X libraries etc, because it should be deployable > > "everywhere") and as acommand inside pgadmin (like MS DTS) > > > > > > Um. I think that's about it. I had some sketches of classes and > > interfaces around (not complete, but an idea), but I can't > find them > > :( > > > > > This sounds like an awful lot of work. > A somewhat reduced version (IIRC Dave and me discussed > something like the following briefly) a more raw import > (maybe into temp tables) could be a big step, giving the > admin the chance to create views on that tables that do the > extractions he likes. PostgreSQL already has all functions > you'd like, no need to reimplement them. it is. That's why I wanted to do that first, but I wanted ot have some sort of generic framework ready first so it could be expanded on later. I'm definitly not saying it should have all that from the beginning :-) //Magnus ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
Re: [pgadmin-hackers] Import from CSV - Questions
Magnus Hagander wrote: Magnus, any chance of getting a look at what you currently have to help guide me in the appropriate direction? Umm. That would kind of assume it's written down in a comprehensible way. Which it isn't, of course :-) The general ideas so far have been, off the top of my head: * Pluginnable set of "readers" and "writers". Originally I'd see postgresql, odbc, xml and possibly csv. Pg driver would be optimised to use COPY when available. * Pluginnable set of "transforms" that would operate on the rows. By default things like copy and concatenate and maybe regexp. Future enhancement would be a python extension, as Dave mentioned. (Or really, anything else) * I was envisioning a split of say "package", "job", "step" (terms of course subject to discussion). package basically a set of job, job a set of steps. Things like connections would be defined at the "job" level, along wiht parmaeters for transaction control etc. (So you can use it to transfer 10 different tables within a single transaction, something I need all the time). * I'd like to see the job format stored as XML with a well defined schema, so different appliations can generate it - both manually (GUI-wise from pgadmin and phppgadmin etc) and automatically. * The "engine" should be available both as a commandline tool (which must not require X libraries etc, because it should be deployable "everywhere") and as acommand inside pgadmin (like MS DTS) Um. I think that's about it. I had some sketches of classes and interfaces around (not complete, but an idea), but I can't find them :( This sounds like an awful lot of work. A somewhat reduced version (IIRC Dave and me discussed something like the following briefly) a more raw import (maybe into temp tables) could be a big step, giving the admin the chance to create views on that tables that do the extractions he likes. PostgreSQL already has all functions you'd like, no need to reimplement them. Regards, Andreas ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [pgadmin-hackers] Import from CSV - Questions
> > Magnus, any chance of getting a look at what you currently > have to help guide me in the appropriate direction? Umm. That would kind of assume it's written down in a comprehensible way. Which it isn't, of course :-) The general ideas so far have been, off the top of my head: * Pluginnable set of "readers" and "writers". Originally I'd see postgresql, odbc, xml and possibly csv. Pg driver would be optimised to use COPY when available. * Pluginnable set of "transforms" that would operate on the rows. By default things like copy and concatenate and maybe regexp. Future enhancement would be a python extension, as Dave mentioned. (Or really, anything else) * I was envisioning a split of say "package", "job", "step" (terms of course subject to discussion). package basically a set of job, job a set of steps. Things like connections would be defined at the "job" level, along wiht parmaeters for transaction control etc. (So you can use it to transfer 10 different tables within a single transaction, something I need all the time). * I'd like to see the job format stored as XML with a well defined schema, so different appliations can generate it - both manually (GUI-wise from pgadmin and phppgadmin etc) and automatically. * The "engine" should be available both as a commandline tool (which must not require X libraries etc, because it should be deployable "everywhere") and as acommand inside pgadmin (like MS DTS) Um. I think that's about it. I had some sketches of classes and interfaces around (not complete, but an idea), but I can't find them :( //Magnus ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [pgadmin-hackers] Import from CSV - Questions
Magnus, any chance of getting a look at what you currently have to help guide me in the appropriate direction?On 3/10/06, Magnus Hagander < [EMAIL PROTECTED] > wrote: > To be honest I didn't realise it was there. The tool I have> in mind for more general data movement is a simplified> version of something like Microsoft's DTS, essentially with> input, transform and output modules. The input modules might > get data from ODBC or a CSV file for example, the transform> module would allow either straight mapping of source to> target columns, and (probably) python based transformation> for more complex stuff, and the output module would insert > the data into PostgreSQL, a CSV file or and ODBC data source etc.>> That is clearly a complex project though... An alternative> might be a simple wizard, activated by a 'load data' option > on the table context menu>> Magnus was also talkng about XML export (and import?) so> perhaps he has some ideas.Yeah, I've got some very sketchy design and code started aroundsomewhere. Nothing ready to be looked at yet though, much less something usable.(And yes, the design was modularised to plugin thigs like ODBC andstuff, just in this particular case I needed XML specificaly)BTW - one important aspect is that the actual transfer part should run as a separate program and not just inside pgadmin. pgadmin would do theGUI to set it up only.//Magnus
Re: [pgadmin-hackers] SVN Commit by dpage: r5054 - trunk/pgadmin3/pkg/win32
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 > [Don't hijack threads] eek. sorry. it was 1 am. i was asleep. :-S > Err, dunno - no Mac to hand atm. It *shouldn't* have changed - can you > see anything obvious in the source? i'll take a look ... as well as browse wx list. i'll let you know ... in a new thread! - -- /"\ \ / ASCII Ribbon Campaign X against HTML email, vCards / \ & micro$oft attachments [GPG] OpenMacNews at gmail dot com fingerprint: 50C9 1C46 2F8F DE42 2EDB D460 95F7 DDBD 3671 08C6 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (Darwin) iEYEAREDAAYFAkQW/MYACgkQlffdvTZxCMbN/ACeJxdBTaKe7gPLWsHVyBSuSDAz LMYAn0IbMfG/VNN+pGZMG1qtRo638xqJ =j+qr -END PGP SIGNATURE- ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [pgadmin-hackers] focus problem reported at debian.org
> -Original Message- > From: Raphaël Enrici [mailto:[EMAIL PROTECTED] > Sent: 14 March 2006 11:00 > To: Dave Page > Cc: Andreas Pflug; PgAdmin Hackers > Subject: Re: [pgadmin-hackers] focus problem reported at debian.org > > Problem is reproduceable under debian with a fresh pgAdmin > III 1.4.2 build. :-( > Could you point me to a usefull wx-examples test case that uses same > piece of code as pgAdmin III so that I try it? I'd start with $WXSRC/contrib/samples/stc/ Regards, Dave ---(end of broadcast)--- TIP 3: Have you checked our extensive FAQ? http://www.postgresql.org/docs/faq
Re: [pgadmin-hackers] SVN Commit by dpage: r5054 - trunk/pgadmin3/pkg/win32
> -Original Message- > From: [EMAIL PROTECTED] > [mailto:[EMAIL PROTECTED] On Behalf Of OpenMacNews > Sent: 14 March 2006 08:43 > Cc: pgadmin-hackers@postgresql.org > Subject: Re: [pgadmin-hackers] SVN Commit by dpage: r5054 - > trunk/pgadmin3/pkg/win32 > > -BEGIN PGP SIGNED MESSAGE- > Hash: RIPEMD160 > > hi all, > > just built up r5053 on OSX ... wha' happen'd to the "Quit ..." menu > item? greyed out and non-functional ... though closing the > main window > now quits the app. [Don't hijack threads] Err, dunno - no Mac to hand atm. It *shouldn't* have changed - can you see anything obvious in the source? /D ---(end of broadcast)--- TIP 5: don't forget to increase your free space map settings
Re: [pgadmin-hackers] SVN Commit by dpage: r5054 - trunk/pgadmin3/pkg/win32
-BEGIN PGP SIGNED MESSAGE- Hash: RIPEMD160 hi all, just built up r5053 on OSX ... wha' happen'd to the "Quit ..." menu item? greyed out and non-functional ... though closing the main window now quits the app. richard - -- /"\ \ / ASCII Ribbon Campaign X against HTML email, vCards / \ & micro$oft attachments [GPG] OpenMacNews at gmail dot com fingerprint: 50C9 1C46 2F8F DE42 2EDB D460 95F7 DDBD 3671 08C6 -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.2.2 (Darwin) iEYEAREDAAYFAkQWggwACgkQlffdvTZxCMbj0gCgoPUL0an6NXADHfKw7nAevu1A o18AoKwyT0+Kvf2J77fkddo4AlpICYq8 =+E/b -END PGP SIGNATURE- ---(end of broadcast)--- TIP 2: Don't 'kill -9' the postmaster
[pgadmin-hackers] SVN Commit by dpage: r5054 - trunk/pgadmin3/pkg/win32
Author: dpage Date: 2006-03-14 08:39:44 + (Tue, 14 Mar 2006) New Revision: 5054 Revision summary: http://svn.pgadmin.org/cgi-bin/viewcvs.cgi/?rev=5054&view=rev Log: Fix typo per Glen Edmonds Modified: trunk/pgadmin3/pkg/win32/upgrade.bat ---(end of broadcast)--- TIP 1: if posting/reading through Usenet, please send an appropriate subscribe-nomail command to [EMAIL PROTECTED] so that your message can get through to the mailing list cleanly
[pgadmin-hackers] SVN Commit by dpage: r5053 - branches/REL-1_4_0_PATCHES/pgadmin3/pkg/win32
Author: dpage Date: 2006-03-14 08:39:03 + (Tue, 14 Mar 2006) New Revision: 5053 Revision summary: http://svn.pgadmin.org/cgi-bin/viewcvs.cgi/?rev=5053&view=rev Log: Fix typo per Glen Edmonds Modified: branches/REL-1_4_0_PATCHES/pgadmin3/pkg/win32/upgrade.bat ---(end of broadcast)--- TIP 4: Have you searched our list archives? http://archives.postgresql.org
