Thanks Mark - I shall investigate (although some FME Object tickerypokery
would probably be the best way to go anyway).
I'm just trying to ensure that I can provide users with an unbreakable
interface; whereas if no errors were visibly thrown, they might assume that
data had uploaded properly. Is it not a bit odd that whereas the workbench
would throw errors, the command line does not? (Presumably in the workbench,
checking whether files/tables exist is done when you first load them in, not
when the fmw is run).
Anyway, I've written a bit of code to read in the log file, and it seems to
work fairly efficiently, so that will probably do the job.
Kevin - thanks also, but I've been there & done that already. I refer you to
this line in my original message:
"Oh, and there's a very good reason why I'm using FME for doing this... "
... which is that the data needs reprojecting, and our PostGIS isn't built
with PROJ.4 (and it doesn't handle the transformation properly anyway).
Anyway, thanks again for all your help,
Roland.
On 17/01/07, mark2atsafe <[EMAIL PROTECTED]> wrote:
Hi Roland,
You're looking for something to happen when the CSV file you're trying
to load doesn't exist? Took me a while but I got there in the end!
I think you're correct that the log file is the way to go.
However, I think the best way to implement this is a post-workspace
TCL script. The reason I suggest this is that there are a number of
global variables made available to this script, including
FME_FeaturesWritten(<featureType>) - so you could do a simple test like...
if {$FME_FeaturesWritten(test_data1) == "0"}
...followed by whatever you want to do in that situation.
In FME2007 you can enter the script into the TCL editor window under
Navigator Window > Workspace Settings > Advanced. In FME2006GB or
earlier you'd have to create an external script and enter a keyword
pointing to that script into the workspace header.
You can find out more info on these TCL scripts on fmepedia.com at...
http://tinyurl.com/3yyhsu
Hope this helps,
Mark
Mark Ireland, Senior Product Specialist
Safe Software Inc. Surrey, BC, CANADA
[EMAIL PROTECTED] <support%40safe.com> http://www.safe.com
Solutions for Spatial Data Translation, Distribution and Access
--- In [email protected] <fme%40yahoogroups.com>, "Roland Martin" <
[EMAIL PROTECTED]> wrote:
>
> Hi.
>
> I'm trying to load a CSV into PostGIS using the command line prompt
(so it
> can be automated). I was hoping it might provide me with something
useful
> when the conversion fails...
>
> Here's a trimmed-down example of the log, for when I invent an input
file
> 'test_data1.csv' (the file doesn't exist).
>
> 2007-01-17 15:57:04| 1.7| 0.0|INFORM|CSV Reader: Opening CSV files in
> directory `D:\test_data1.csv\'.
> 2007-01-17 15:57:04| 1.7| 0.0|INFORM|Using Dynamic Reader
> $Revision: 17.1$ ( $Date: 2006/01/06 17:33:15 $ ) with module CSV to
> read data from dataset
> `D:\test_data1.csv'
> 2007-01-17 15:57:04| 1.7| 0.0|INFORM|Using Dynamic Writer
> $Revision: 17.0$ ( $Date: 2005/10/28 21:07:49 $ ) with module POSTGIS
> to write dataset
> `spatial'
> 2007-01-17 15:57:04| 1.7| 0.0|INFORM|Module `POSTGIS' API version
matches
> current core version (3.1 20031010).
> 2007-01-17 15:57:04| 1.7| 0.0|ERROR |CSV Reader: Missing or
invalid input
> dataset 'D:\test_data1.csv\test_data.CSV'.
> 2007-01-17 15:57:04| 1.7| 0.0|ERROR |CSV Reader: Missing or
invalid input
> dataset 'D:\test_data1.csv\test_data.CSV'.
> 2007-01-17 15:57:04| 1.7| 0.0|INFORM|Emptying factory pipeline
> 2007-01-17 15:57:04| 1.7| 0.0|STATS |Unexpected Input
> Remover(TestFactory): Tested 0 input features -- 0 features passed, 0
> features failed.
> 2007-01-17 15:57:04| 1.7| 0.0|STATS |Source -> Generic(TeeFactory):
> Cloned 0 input feature(s) into 0 output feature(s).
> 2007-01-17 15:57:04| 1.7| 0.0|STATS
|2DPOINTREPLACER(TestFactory): Tested
> 0 input features -- 0 features passed, 0 features failed.
> 2007-01-17 15:57:04| 1.7| 0.0|STATS |2DPOINTREPLACER_POINT ->
> public.banana Correlator(TeeFactory): Cloned 0 input feature(s) into 0
> output feature(s).
> 2007-01-17 15:57:04| 1.7| 0.0|STATS |Final Output Feature Type
> Router(TestFactory): Tested 0 input features -- 0 features passed, 0
> features failed.
> 2007-01-17 15:57:04| 1.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 15:57:04| 1.7| 0.0|STATS |
Features
> Read Summary
> 2007-01-17 15:57:04| 1.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 15:57:04| 1.7|
>
0.0|STATS|==============================================================================
> 2007-01-17 15:57:04| 1.7| 0.0|STATS |Total Features
> Read 0
> 2007-01-17 15:57:04| 1.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 15:57:04| 1.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 15:57:04| 1.7| 0.0|STATS |
Features
> Written Summary
> 2007-01-17 15:57:04| 1.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 15:57:04| 1.7|
>
0.0|STATS|==============================================================================
> 2007-01-17 15:57:04| 1.7| 0.0|STATS |Total Features
> Written 0
> 2007-01-17 15:57:04| 1.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 15:57:04| 1.7| 0.0|INFORM|CSV Reader: Done reading CSV
files
> in directory `D:\test_data1.csv\'.
> 2007-01-17 15:57:04| 1.7| 0.0|INFORM|Translation was SUCCESSFUL (0
> feature(s)/0 coordinate(s) output)
> 2007-01-17 15:57:04| 1.7| 0.0|INFORM|FME Session Duration: 2.0
seconds.
>
> As you can see, it's claiming the translation was successful; and
prints the
> same message to the command window. If I tell it to load to an invalid
> PostGIS table (i.e. one that exists, but has the wrong schema), it's
even
> more bizarre:
>
> 2007-01-17 16:09:45| 2.7| 0.1|INFORM|Module `POSTGIS' API version
matches
> current core version (3.1 20031010).
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|Opening POSTGIS writer for
dataset
> 'spatial'...
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|PostgreSQL 8.1.1 on
i686-pc-mingw32,
> compiled by GCC gcc.exe (GCC) 3.4.2 (mingw-special)
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|1.0 USE_GEOS=1 USE_PROJ=1
> USE_STATS=1
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|PostGIS client encoding:
'LATIN1'.
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|Testing for the existence of
table '
> public.meh'...
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|Appending to existing table '
> public.meh'...
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|CSV Reader: Closing dataset
> 'D:\test_data.csv'.
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|Emptying factory pipeline
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |Unexpected Input
> Remover(TestFactory): Tested 4 input features -- 4 features passed, 0
> features failed.
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |Source -> Generic(TeeFactory):
> Cloned 4 input feature(s) into 4 output feature(s).
> 2007-01-17 16:09:45| 2.7| 0.0|STATS
|2DPOINTREPLACER(TestFactory): Tested
> 4 input features -- 0 features passed, 4 features failed.
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |2DPOINTREPLACER_POINT ->
> public.mehCorrelator(TeeFactory): Cloned 4 input feature(s) into 4
> output feature(s).
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |Final Output Feature Type
> Router(TestFactory): Tested 4 input features -- 4 features passed, 0
> features failed.
> 2007-01-17 16:09:45| 2.7| 0.0|ERROR |Bulk copy failed on table '
> public.meh' using delimiter ':'. Error was 'ERROR: null value in column
> "session" violates not-null constraint
> CONTEXT: COPY meh, line 1: "0000-01-01:NULL:NULL:NULL"
> '.
> 2007-01-17 16:09:45| 2.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |
Features
> Read Summary
> 2007-01-17 16:09:45| 2.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 16:09:45| 2.7| 0.0|STATS|test_data
> 4
> 2007-01-17 16:09:45| 2.7|
>
0.0|STATS|==============================================================================
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |Total Features
> Read 4
> 2007-01-17 16:09:45| 2.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 16:09:45| 2.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |
Features
> Written Summary
> 2007-01-17 16:09:45| 2.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |public.meh (public.meh)
> 4
> 2007-01-17 16:09:45| 2.7|
>
0.0|STATS|==============================================================================
> 2007-01-17 16:09:45| 2.7| 0.0|STATS |Total Features
> Written 4
> 2007-01-17 16:09:45| 2.7|
>
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|CSV Reader: Done reading CSV
files
> in directory `D:\'.
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|Translation was SUCCESSFUL (4
> feature(s)/4 coordinate(s) output)
> 2007-01-17 16:09:45| 2.7| 0.0|INFORM|FME Session Duration: 2.5
seconds.
>
> And despite claiming to have written four features, it actually does
nothing
> (and still returns 'successful').
>
> FME2006GB, by the way. Oh, and there's a very good reason why I'm
using FME
> for doing this...
>
> So, I guess there are 2 things going on here:
> 1. It's not spotting me passing it an invalid input
> 2. It's not dealing properly with invalid outputs
>
> My question, then... any bright ideas how I can spot if translation has
> failed? At the moment I'm leaning towards loading up the log and
searching
> for the word "error"...
>
> Thanks,
> Roland.
>
