Hi.
I'm trying to load a CSV into PostGIS using the command line prompt (so
it can be automated). I was hoping it might provide me with something
useful when the conversion fails...
Here's a trimmed-down example of the log, for when I invent an input
file 'test_data1.csv' (the file doesn't exist).
2007-01-17
15:57:04| 1.7| 0.0|INFORM|CSV Reader: Opening CSV files in directory
`D:\test_data1.csv\'.
2007-01-17
15:57:04| 1.7| 0.0|INFORM|Using Dynamic Reader $Revision: 17.1 $ (
$Date: 2006/01/06 17:33:15 $ ) with module CSV to read data from
dataset `D:\test_data1.csv'
2007-01-17
15:57:04| 1.7| 0.0|INFORM|Using Dynamic Writer $Revision: 17.0 $ (
$Date: 2005/10/28 21:07:49 $ ) with module POSTGIS to write dataset
`spatial'
2007-01-17
15:57:04| 1.7| 0.0|INFORM|Module `POSTGIS' API version matches
current core version (3.1 20031010).
2007-01-17
15:57:04| 1.7| 0.0|ERROR |CSV Reader: Missing or invalid input
dataset 'D:\test_data1.csv\test_data.CSV'.
2007-01-17
15:57:04| 1.7| 0.0|ERROR |CSV Reader: Missing or invalid input
dataset 'D:\test_data1.csv\test_data.CSV'.
2007-01-17
15:57:04| 1.7| 0.0|INFORM|Emptying factory pipeline
2007-01-17
15:57:04| 1.7| 0.0|STATS |Unexpected Input Remover(TestFactory):
Tested 0 input features -- 0 features passed, 0 features failed.
2007-01-17
15:57:04| 1.7| 0.0|STATS |Source -> Generic(TeeFactory):
Cloned 0 input feature(s) into 0 output feature(s).
2007-01-17
15:57:04| 1.7| 0.0|STATS |2DPOINTREPLACER(TestFactory):
Tested 0 input features -- 0 features passed, 0 features failed.
2007-01-17
15:57:04| 1.7| 0.0|STATS |2DPOINTREPLACER_POINT ->
public.banana Correlator(TeeFactory): Cloned 0 input feature(s)
into 0 output feature(s).
2007-01-17
15:57:04| 1.7| 0.0|STATS |Final Output Feature Type
Router(TestFactory): Tested 0 input features -- 0 features passed,
0 features failed.
2007-01-17
15:57:04| 1.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
15:57:04| 1.7| 0.0|STATS | Features Read
Summary
2007-01-17
15:57:04| 1.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
15:57:04| 1.7| 0.0|STATS |==============================================================================
2007-01-17
15:57:04| 1.7| 0.0|STATS |Total Features
Read 0
2007-01-17
15:57:04| 1.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
15:57:04| 1.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
15:57:04| 1.7| 0.0|STATS | Features
Written Summary
2007-01-17
15:57:04| 1.7|
0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
15:57:04| 1.7|
0.0|STATS |==============================================================================
2007-01-17
15:57:04| 1.7|
0.0|STATS |Total Features
Written 0
2007-01-17
15:57:04| 1.7|
0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
15:57:04| 1.7|
0.0|INFORM|CSV Reader: Done reading CSV files in directory
`D:\test_data1.csv\'.
2007-01-17
15:57:04| 1.7| 0.0|INFORM|Translation was SUCCESSFUL (0
feature(s)/0 coordinate(s) output)
2007-01-17
15:57:04| 1.7| 0.0|INFORM|FME Session Duration: 2.0 seconds.
As you can see, it's claiming the translation was successful; and
prints the same message to the command window. If I tell it to load to
an invalid PostGIS table (i.e. one that exists, but has the wrong
schema), it's even more bizarre:
2007-01-17
16:09:45| 2.7| 0.1|INFORM|Module `POSTGIS' API version matches
current core version (3.1 20031010).
2007-01-17
16:09:45| 2.7| 0.0|INFORM|Opening POSTGIS writer for dataset
'spatial'...
2007-01-17
16:09:45| 2.7| 0.0|INFORM|PostgreSQL 8.1.1 on i686-pc-mingw32,
compiled by GCC gcc.exe (GCC) 3.4.2 (mingw-special)
2007-01-17
16:09:45| 2.7| 0.0|INFORM|1.0 USE_GEOS=1 USE_PROJ=1 USE_STATS=1
2007-01-17
16:09:45| 2.7| 0.0|INFORM|PostGIS client encoding: 'LATIN1'.
2007-01-17
16:09:45| 2.7| 0.0|INFORM|Testing for the existence of table
'public.meh
'...
2007-01-17
16:09:45| 2.7| 0.0|INFORM|Appending to existing table
'public.meh'...
2007-01-17
16:09:45| 2.7| 0.0|INFORM|CSV Reader: Closing dataset 'D:\test_data.csv'.
2007-01-17
16:09:45| 2.7| 0.0|INFORM|Emptying factory pipeline
2007-01-17
16:09:45| 2.7| 0.0|STATS |Unexpected Input Remover(TestFactory):
Tested 4 input features -- 4 features passed, 0 features failed.
2007-01-17
16:09:45| 2.7| 0.0|STATS |Source -> Generic(TeeFactory):
Cloned 4 input feature(s) into 4 output feature(s).
2007-01-17
16:09:45| 2.7| 0.0|STATS |2DPOINTREPLACER(TestFactory):
Tested 4 input features -- 0 features passed, 4 features failed.
2007-01-17
16:09:45| 2.7| 0.0|STATS |2DPOINTREPLACER_POINT ->
public.meh Correlator(TeeFactory): Cloned 4 input feature(s) into
4 output feature(s).
2007-01-17
16:09:45| 2.7| 0.0|STATS |Final Output Feature Type
Router(TestFactory): Tested 4 input features -- 4 features passed,
0 features failed.
2007-01-17
16:09:45| 2.7| 0.0|ERROR |Bulk copy failed on table 'public.meh'
using delimiter ':'. Error was 'ERROR: null value in column "session"
violates not-null constraint
CONTEXT: COPY meh,
line 1: "0000-01-01:NULL:NULL:NULL"
'.
2007-01-17
16:09:45| 2.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
16:09:45| 2.7| 0.0|STATS | Features Read
Summary
2007-01-17
16:09:45| 2.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
16:09:45| 2.7| 0.0|STATS
|test_data
4
2007-01-17
16:09:45| 2.7| 0.0|STATS |==============================================================================
2007-01-17
16:09:45| 2.7| 0.0|STATS |Total Features
Read 4
2007-01-17
16:09:45| 2.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
16:09:45| 2.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
16:09:45| 2.7| 0.0|STATS | Features
Written Summary
2007-01-17
16:09:45| 2.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
16:09:45| 2.7| 0.0|STATS |public.meh (public.meh)
4
2007-01-17
16:09:45| 2.7| 0.0|STATS |==============================================================================
2007-01-17
16:09:45| 2.7| 0.0|STATS |Total Features
Written 4
2007-01-17
16:09:45| 2.7| 0.0|STATS |=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17
16:09:45| 2.7| 0.0|INFORM|CSV Reader: Done reading CSV files in
directory `D:\'.
2007-01-17
16:09:45| 2.7| 0.0|INFORM|Translation was SUCCESSFUL (4
feature(s)/4 coordinate(s) output)
2007-01-17
16:09:45| 2.7| 0.0|INFORM|FME Session Duration: 2.5 seconds.
And despite claiming to have
written four features, it actually does nothing (and still returns
'successful').
FME2006GB, by the way. Oh, and there's a very good reason why I'm using
FME for doing this...
So, I guess there are 2 things going on here:
1. It's not spotting me passing it an invalid input
2. It's not dealing properly with invalid outputs
My question, then... any bright ideas how I can spot if translation has
failed? At the moment I'm leaning towards loading up the log and
searching for the word "error"...
Thanks,
Roland.
