We have an issue with producing all files from running a series of
fme workbenches at the command line. Apologies if this is a lengthy
explanation - if you have experience of anything similar we'd very
much appreciate hearing!
By way of context, Transport for London have a c45 Gbyte database of
Ordnance Survey Master Map data, loaded using Dotted Eyes SuperpOSe
We have produced a workbench which extracts data directly from this
database to produce Microstation DGN format
The output is tiled using clipper (to clip data in the continuous
database against Ordnance Survey map tiles) and fanout via map tile
name using a MapInfo file from Ordnance Survey with map tiles and
extents provided
This is running on Windows Server 2003 (32 bit) under VMWare ESX
Given the volumes of data we have had to split the extract into 65
10x10km squares so that the extract does not run out of memory
To "automate" this, we produced a simple batch file which looks like
fme [workbench_name] [extract co-ordinates1]
fme [workbench_name] [extract co-ordinates2]
...
and so on.
When we run this however we do not get all the output files expected -
even though when you run any one of these commands it does work
[NB: we have tried using the COMMAND_FILE option but this runs out of
memory for some reason]
It appears that there is some latency in the physical disk writing -
so the next command is processed before all files are physically
written to disk or similar...
We have used a tool called Process Monitor to see what is happening
The most obvious output is that there is a FASTIO_WRITE thread which
gets a message FAST IO DISALLOWED - which is unusual
Apart from this we are not clear as to what is happening - any advice
or even awareness as to whether others have encountered similar
issues would be appreciated
For insights into what's up at Safe Software and what's on the development
horizon, visit Safe's blog at spatial-etl.blogspot.com.
Safe Software has also made slides available that outline enhancements planned
for FME 2007. The slides are from the "Road Ahead" presentation given on Day 2
of the FME Worldwide Users Conference. To view these slides, visit
www.safe.com/2006uc.
Yahoo! Groups Links
<*> To visit your group on the web, go to:
http://groups.yahoo.com/group/fme/
<*> Your email settings:
Individual Email | Traditional
<*> To change settings online go to:
http://groups.yahoo.com/group/fme/join
(Yahoo! ID required)
<*> To change settings via email:
mailto:[EMAIL PROTECTED]
mailto:[EMAIL PROTECTED]
<*> To unsubscribe from this group, send an email to:
[EMAIL PROTECTED]
<*> Your use of Yahoo! Groups is subject to:
http://docs.yahoo.com/info/terms/
