Hi.
I'm trying to load a CSV into PostGIS using the command line prompt (so it
can be automated). I was hoping it might provide me with something useful
when the conversion fails...
Here's a trimmed-down example of the log, for when I invent an input file
'test_data1.csv' (the file doesn't exist).
2007-01-17 15:57:04| 1.7| 0.0|INFORM|CSV Reader: Opening CSV files in
directory `D:\test_data1.csv\'.
2007-01-17 15:57:04| 1.7| 0.0|INFORM|Using Dynamic Reader
$Revision: 17.1$ ( $Date: 2006/01/06 17:33:15 $ ) with module CSV to
read data from dataset
`D:\test_data1.csv'
2007-01-17 15:57:04| 1.7| 0.0|INFORM|Using Dynamic Writer
$Revision: 17.0$ ( $Date: 2005/10/28 21:07:49 $ ) with module POSTGIS
to write dataset
`spatial'
2007-01-17 15:57:04| 1.7| 0.0|INFORM|Module `POSTGIS' API version matches
current core version (3.1 20031010).
2007-01-17 15:57:04| 1.7| 0.0|ERROR |CSV Reader: Missing or invalid input
dataset 'D:\test_data1.csv\test_data.CSV'.
2007-01-17 15:57:04| 1.7| 0.0|ERROR |CSV Reader: Missing or invalid input
dataset 'D:\test_data1.csv\test_data.CSV'.
2007-01-17 15:57:04| 1.7| 0.0|INFORM|Emptying factory pipeline
2007-01-17 15:57:04| 1.7| 0.0|STATS |Unexpected Input
Remover(TestFactory): Tested 0 input features -- 0 features passed, 0
features failed.
2007-01-17 15:57:04| 1.7| 0.0|STATS |Source -> Generic(TeeFactory):
Cloned 0 input feature(s) into 0 output feature(s).
2007-01-17 15:57:04| 1.7| 0.0|STATS |2DPOINTREPLACER(TestFactory): Tested
0 input features -- 0 features passed, 0 features failed.
2007-01-17 15:57:04| 1.7| 0.0|STATS |2DPOINTREPLACER_POINT ->
public.banana Correlator(TeeFactory): Cloned 0 input feature(s) into 0
output feature(s).
2007-01-17 15:57:04| 1.7| 0.0|STATS |Final Output Feature Type
Router(TestFactory): Tested 0 input features -- 0 features passed, 0
features failed.
2007-01-17 15:57:04| 1.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 15:57:04| 1.7| 0.0|STATS | Features
Read Summary
2007-01-17 15:57:04| 1.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 15:57:04| 1.7|
0.0|STATS|==============================================================================
2007-01-17 15:57:04| 1.7| 0.0|STATS |Total Features
Read 0
2007-01-17 15:57:04| 1.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 15:57:04| 1.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 15:57:04| 1.7| 0.0|STATS | Features
Written Summary
2007-01-17 15:57:04| 1.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 15:57:04| 1.7|
0.0|STATS|==============================================================================
2007-01-17 15:57:04| 1.7| 0.0|STATS |Total Features
Written 0
2007-01-17 15:57:04| 1.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 15:57:04| 1.7| 0.0|INFORM|CSV Reader: Done reading CSV files
in directory `D:\test_data1.csv\'.
2007-01-17 15:57:04| 1.7| 0.0|INFORM|Translation was SUCCESSFUL (0
feature(s)/0 coordinate(s) output)
2007-01-17 15:57:04| 1.7| 0.0|INFORM|FME Session Duration: 2.0 seconds.
As you can see, it's claiming the translation was successful; and prints the
same message to the command window. If I tell it to load to an invalid
PostGIS table (i.e. one that exists, but has the wrong schema), it's even
more bizarre:
2007-01-17 16:09:45| 2.7| 0.1|INFORM|Module `POSTGIS' API version matches
current core version (3.1 20031010).
2007-01-17 16:09:45| 2.7| 0.0|INFORM|Opening POSTGIS writer for dataset
'spatial'...
2007-01-17 16:09:45| 2.7| 0.0|INFORM|PostgreSQL 8.1.1 on i686-pc-mingw32,
compiled by GCC gcc.exe (GCC) 3.4.2 (mingw-special)
2007-01-17 16:09:45| 2.7| 0.0|INFORM|1.0 USE_GEOS=1 USE_PROJ=1
USE_STATS=1
2007-01-17 16:09:45| 2.7| 0.0|INFORM|PostGIS client encoding: 'LATIN1'.
2007-01-17 16:09:45| 2.7| 0.0|INFORM|Testing for the existence of table '
public.meh'...
2007-01-17 16:09:45| 2.7| 0.0|INFORM|Appending to existing table '
public.meh'...
2007-01-17 16:09:45| 2.7| 0.0|INFORM|CSV Reader: Closing dataset
'D:\test_data.csv'.
2007-01-17 16:09:45| 2.7| 0.0|INFORM|Emptying factory pipeline
2007-01-17 16:09:45| 2.7| 0.0|STATS |Unexpected Input
Remover(TestFactory): Tested 4 input features -- 4 features passed, 0
features failed.
2007-01-17 16:09:45| 2.7| 0.0|STATS |Source -> Generic(TeeFactory):
Cloned 4 input feature(s) into 4 output feature(s).
2007-01-17 16:09:45| 2.7| 0.0|STATS |2DPOINTREPLACER(TestFactory): Tested
4 input features -- 0 features passed, 4 features failed.
2007-01-17 16:09:45| 2.7| 0.0|STATS |2DPOINTREPLACER_POINT ->
public.mehCorrelator(TeeFactory): Cloned 4 input feature(s) into 4
output feature(s).
2007-01-17 16:09:45| 2.7| 0.0|STATS |Final Output Feature Type
Router(TestFactory): Tested 4 input features -- 4 features passed, 0
features failed.
2007-01-17 16:09:45| 2.7| 0.0|ERROR |Bulk copy failed on table '
public.meh' using delimiter ':'. Error was 'ERROR: null value in column
"session" violates not-null constraint
CONTEXT: COPY meh, line 1: "0000-01-01:NULL:NULL:NULL"
'.
2007-01-17 16:09:45| 2.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 16:09:45| 2.7| 0.0|STATS | Features
Read Summary
2007-01-17 16:09:45| 2.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 16:09:45| 2.7| 0.0|STATS|test_data
4
2007-01-17 16:09:45| 2.7|
0.0|STATS|==============================================================================
2007-01-17 16:09:45| 2.7| 0.0|STATS |Total Features
Read 4
2007-01-17 16:09:45| 2.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 16:09:45| 2.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 16:09:45| 2.7| 0.0|STATS | Features
Written Summary
2007-01-17 16:09:45| 2.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 16:09:45| 2.7| 0.0|STATS |public.meh (public.meh)
4
2007-01-17 16:09:45| 2.7|
0.0|STATS|==============================================================================
2007-01-17 16:09:45| 2.7| 0.0|STATS |Total Features
Written 4
2007-01-17 16:09:45| 2.7|
0.0|STATS|=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-=-
2007-01-17 16:09:45| 2.7| 0.0|INFORM|CSV Reader: Done reading CSV files
in directory `D:\'.
2007-01-17 16:09:45| 2.7| 0.0|INFORM|Translation was SUCCESSFUL (4
feature(s)/4 coordinate(s) output)
2007-01-17 16:09:45| 2.7| 0.0|INFORM|FME Session Duration: 2.5 seconds.
And despite claiming to have written four features, it actually does nothing
(and still returns 'successful').
FME2006GB, by the way. Oh, and there's a very good reason why I'm using FME
for doing this...
So, I guess there are 2 things going on here:
1. It's not spotting me passing it an invalid input
2. It's not dealing properly with invalid outputs
My question, then... any bright ideas how I can spot if translation has
failed? At the moment I'm leaning towards loading up the log and searching
for the word "error"...
Thanks,
Roland.
