Simon,
After taking a look at your zos_*.nc file, I believe the problem is that the
bounds variables (specifically lon_vertices and lat_vertices) are incorrect and
causing malformed cells. The proper format for these variables according to
the CF convention is documented here –
http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/ch07.html#cell-boundaries
– under "Bounds for 2-D coordinate variables with 4-sided cells". These
variables are referred to as lonbnd and latbnd in the CF documentation, but
otherwise the name is inconsequential.
These bounds variables are supposed to contain a 4-tuple for each cell in the
grid. The 4 values in each tuple specify the longitude or latitude coordinates
in counterclockwise order starting with the "lower left" value. Adjacent cells
should have duplicate longitude and latitude coordinates where appropriate.
So, assuming that the bounds of the first cell (which I am extrapolating from
other coordinate variables in the zos file) is –280 to –279 in the longitude
direction and –82 to –81 in the latitude direction, then the first 4-tuple in
lon_vertices should be [–280, –279, –279, –280] and the first tuple in
lat_vertices should be [–82, –82, –81, –81]. The next entries in the
lon_vertices and lat_vertices arrays are for the adjacent cell in the
horizontal direction and should be [–279, –278, –278, –279] and [–82, –82, –81,
–81], respectively.
In this regard, the values stored in zon*.nc's lon_vertices and lat_vertices
are nonsensical. ncdump shows the first two entries of each as:
lon_vertices =
-280, -279, -278, -277,
-276, -275, -274, -273,
lat_vertices =
-82, -82, -82, -82,
-82, -82, -82, -82,
As you can see, these bounds do not form proper quadrilaterals. They are all
degenerate polygons on the –82 latitude coordinate. My guess is that whatever
wrote out this file mistook the format for boundary variables for coordinate
variables.
-Ken
**** Kenneth Moreland
*** Sandia National Laboratories
***********
*** *** *** email: [email protected]
** *** ** phone: (505) 844-8919
*** web: http://www.cs.unm.edu/~kmorel
From: Simon Su <[email protected]<mailto:[email protected]>>
Date: Tue, 22 Mar 2011 11:40:46 -0400
To: Kenneth Moreland <[email protected]<mailto:[email protected]>>
Cc: Andy Bauer <[email protected]<mailto:[email protected]>>,
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: [Paraview] loading nc formatted data
Hi Ken,
Thank you again for your help.
Is there another wiki page that also describe all the xml used syntax in the
http://www.vtk.org/Wiki/ParaView/Plugin_HowTo ? It may help for me to have an
overall picture of the xml syntax used? thanks. And is there a complete example
out there that show the whole process on how one can create a custom data
loader plugin into ParaView? Not sure if I am saying this correctly but
something like, say I have this data file in my own format that has grid
information and a scalar variable. How do I write a data loader plugin so that
I can use ParaView to visualize the data. Then if I have to guess, first you
write this xml file to describe the data loader plugin to ParaView and then use
use sort of utility to generate the plugin standard code, and then implement
the function in the code to load and create your vtk data structure (grid data
type and scalar variable data type) and then pass it to ParaView. something
like that? thanks...
yes. it make sense that it is a part of VTK. ParaView is to expose VTK
functionality after all.
good thing is, I think we settle on a single large nc file that has all the
information needed to plot the visualization, including tripolar grid
information. Hopefully that will simplify things for me.
I am sorry that the ftp site is not behaving. I have placed the files at
http://www.cs.uh.edu/~ssu/4ParaView together with the nc file I used to
generate the image 4.png (what looks like a double grid to me) and 5.png
(coloring by the variable zos which I don't thin it should look like that )
Hopefully vtkNetCDFCFReader will do the job for me.
thanks
-simon
On Mon, Mar 21, 2011 at 7:12 PM, Moreland, Kenneth
<[email protected]<mailto:[email protected]>> wrote:
It looks like the documentation for a reader plugin is slightly messed up.
I'll leave it to Andy (or anyone who is not me) to fix. The example in the
ParaView source does not list a file in SERVER_MANAGER_SOURCES because it uses
a source (vtkPNGReader.cxx) that is already compiled as part of VTK. The Wiki
is trying to capture the fact that usually you are building a plugin with your
own reader, and you will have to list your source code for the reader there.
vtkMyPNGReader.cxx is just a stand in for some reader that you wrote. It
probably does not really exist. I noticed, however, that the Wiki
documentation is also wrong in that the XML file is pointing to the
vtkPNGReader class when the CMake configuration is implying that it should be
pointing to the (imaginary) vtkMyPNGReader class.
The vtkNetCDFCFReader.cxx file is located in
ParaView/VTK/IO/vtkNetCDFCFReader.cxx. Don't get too wrapped up around the
complexity of the NetCDF reader definition in readers.xml. It's complicated
because it supports defining a time series as a collection of files
(http://www.vtk.org/Wiki/Animating_legacy_VTK_file_series#Making_custom_readers_work_with_file_series).
If you collect all times in a single netCDF file (or don't have time steps),
you can skip all that and just define a new object much like that of
netCDFReaderCore in reader.xml (in the sources ProxyGroup of course).
That said, I think this should all be unnecessary since the CF convention (and
ParaView reader) already contains a means of defining cells that bridge across
these seems. I suspect it has nothing to do with the CF convention version.
It is probably that your data files with openings at the seems are not defining
a bounds attribute, which is optional in CF. If you could post some example
data (or even the listing of the file header from ncdump) I might be able to
verify that.
I'm not sure what is going on with the new data set (4.png). I can't comment
on it much because your ftp server seems to be broken for me and I don't
remember the image too well. At any rate, I would probably say something like
I couldn't tell you what was going on without seeing the nc file.
-Ken
**** Kenneth Moreland
*** Sandia National Laboratories
***********
*** *** *** email: [email protected]<http://[email protected]>
** *** ** phone: (505) 844-8919<tel:%28505%29%20844-8919>
*** web:
http://www.cs.unm.edu/~kmorel<http://www.cs.unm.edu/%7Ekmorel>
From: Simon Su <[email protected]<mailto:[email protected]>>
Date: Mon, 21 Mar 2011 15:19:24 -0400
To: Kenneth Moreland <[email protected]<mailto:[email protected]>>
Cc: Andy Bauer <[email protected]<mailto:[email protected]>>,
"[email protected]<mailto:[email protected]>"
<[email protected]<mailto:[email protected]>>
Subject: Re: [Paraview] loading nc formatted data
Hi Ken,
Thank you for explaining.
The reader that is being used is vtkNetCDFCFReader. It is defined in
ParaView/Servers/ServerManager/Resources/readers.xml, although it is not
obvious.
Can you (or Andy) point me to documentation more documentation beside
http://www.vtk.org/Wiki/ParaView/Plugin_HowTo ? Andy also metioned readers.xml
file you mentioned. But after going through ParaView/Servers/
ServerManager/Resources/readers.xml, I am still fuzzy about where the file
vtkNetCDFCFReader.cxx ( if any - I would like to know how it is implemented
and maybe start by modifying it in my learning process). The plugin wiki page
on "Adding a Reader" section mentioned CMakeList.txt file
FIND_PACKAGE(ParaView REQUIRED)
INCLUDE(${PARAVIEW_USE_FILE})
ADD_PARAVIEW_PLUGIN(MyReader "1.0"
SERVER_MANAGER_XML MyPNGReader.xml
SERVER_MANAGER_SOURCES vtkMyPNGReader.cxx
GUI_RESOURCE_FILES MyReaderGUI.xml)
but the one on my source tree
ParaView-3.10.0/Examples/Plugins/Reader/CMakeList.txt looks like
ADD_PARAVIEW_PLUGIN(MyPNGReader "1.0"
SERVER_MANAGER_XML readers.xml
GUI_RESOURCE_FILES pqReader.xml
)
note no vtkMyPNGReader.cxx file mention and the file is also not in the
directory. I also did a search on source tree for vtkNetCDFCFReader.cxx file
but can't find it. I think I am missing a big piece of something since I am
not getting the point that I should be getting after looking at readers.xml. In
fact, I can't find any of the netcdf file readers (other than CF reader). It is
also not listed under my Plugin Manager GUI on the ParaView that I compiled but
it is loading the CF convention nc file. Are they (the *.cxx files) generated
on the fly during compile time?
Another thing, besides http://www.paraview.org/Wiki/VisIt_Database_Bridge , is
there a place that I can find how to use that VisIt bridge in ParaView. Maybe
an example of how to load silo file? I actually have a plugin of my own in
VisIt that can load the climate modeling data that I have. I would like to see
if I can use my VisIt plugin in ParaView and not write another plugin for
ParaView? :)
As the name implies, this reader reads netCDF files using the CF convention.
(As the default netCDF reader, it also gracefully handles files that do not
follow this convention.) With this assumption, I will try to explain what it
does. It reads arrays as regular 1, 2, or 3D arrays, possibly with time. The
CF convention also provides a means to assign coordinates to each grid point
and to identify the coordinates as longitude or latitude.
The published nc files we have do adhere to CF convention we also use CMOR in
one of the pipeline. I noticed Data_tos_O1_2001-2002.nc is at CF-1.0 and since
I am new at CF Convention, I can only speculate that what I see at
ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/4.png with what looks like double
grid is due to difference between CF-1.0 and CF-1.4? Will vtkNetCDFCFReader
handle CF-1.4 data?
Even though coordinates are defined as longitude and latitude, the topology of
the grid itself is still a grid. Thus, the grid gets wrapped around, but still
has these seems that you see because topologically the one end of the regular
grid is not attached to the other. I can't think of any filter that will
identify and close these seems. In fact, it's not straightforward to do at
all. If you look at your topology, it is not lain out on a simple spherical
grid.
The "right" way to solve your problem, which may or may not be in your control,
is to create netCDF files that specify cell boundaries for a closed topology.
Your netCDF file must be following at least some parts of the CF convention;
your data would not show up as a sphere if it were not. The CF convention
provides a way of defining cells that are not constrained by a regular grid
topology. It is done through a "bounds" attribute on the dimension descriptor
variables. You need either 1D or 2D bounds. They are described in this
section of the CF convention documentation:
http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#cell-boundaries.
Agreed. The hack we have in VisIt loader (replace the last grid value with the
1st grid value to create a "closed topology" ) is not "generalizable" and will
blow up in my face if I hand it to the user.
thanks
-simon
The VTK test data has examples of both 1D bounds (tos_O1_2001-2002.nc) and 2D
bounds (sampleCurveGrid4.nc). You can get the VTK test data from git
(http://vtk.org/VTKData.git), or download them directly from the gitourious
server:
http://vtk.org/gitweb?p=VTKData.git;a=blob;f=Data/tos_O1_2001-2002.nc;h=30aa4a9b5e08b9bdf64540f2b144d83b279cca6c;hb=HEAD
http://vtk.org/gitweb?p=VTKData.git;a=blob;f=Data/sampleCurveGrid4.nc;h=0ab89c27a25f92c047b58dca8b3057ca8d4df017;hb=HEAD
-Ken
**** Kenneth Moreland
*** Sandia National Laboratories
***********
*** *** *** email: [email protected]<http://[email protected]>
** *** ** phone: (505) 844-8919<tel:%28505%29%20844-8919>
*** web:
http://www.cs.unm.edu/~kmorel<http://www.cs.unm.edu/%7Ekmorel>
From: Simon Su <[email protected]<mailto:[email protected]>>
Date: Fri, 18 Mar 2011 16:43:42 -0400
To: Andy Bauer <[email protected]<mailto:[email protected]>>
Cc: <[email protected]<mailto:[email protected]>>
Subject: Re: [Paraview] loading nc formatted data
Hi Andy,
python trace gave me
zos_Omon_GFDLESM2M_historical_r1i1p1_186101188012_nc = NetCDFReader(
FileName=['/work/sms/data/cmor-20110128/mon/ocean/zos/r1i1p1/zos_Omon_GFDL-ESM2M_historical_r1i1p1_186101-188012.nc']
)
and there are tons of netcdf reader in ParaView and it is not in the plugin
directory
sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> pwd
/local/home/build/paraview/ParaView-3.10.0/Plugins
sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> ll
total 84
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 AdiosReader/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 AnalyzeNIfTIReaderWriter/
-rw-r--r-- 1 sms t 3261 Mar 9 13:31 CMakeLists.txt
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 CoProcessingScriptGenerator/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 CosmoFilters/
drwxr-xr-x 4 sms t 4096 Mar 10 12:37 EyeDomeLighting/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 ForceTime/
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 H5PartReader/
drwxr-xr-x 4 sms t 4096 Mar 10 12:37 Manta/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 Moments/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 NetDMFReader/
drwxr-xr-x 6 sms t 4096 Mar 10 12:37 PointSprite/
drwxr-xr-x 4 sms t 4096 Mar 10 12:37 PrismPlugins/
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 pvblot/
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 SierraPlotTools/
drwxr-xr-x 3 sms t 4096 Mar 10 12:37 SLACTools/
drwxr-xr-x 4 sms t 4096 Mar 10 12:37 StreamingView/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 SurfaceLIC/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 Vapor/
drwxr-xr-x 2 sms t 4096 Mar 10 12:37 VRPN/
drwxr-xr-x 2 sms t 4096 Mar 9 13:31 VRUI/
sms:/local/home/build/paraview/ParaView-3.10.0/Plugins>
Can you help describe how Netcdf files plugins are done in ParaView. Do they
have a super class of Netcdf that they all derive from to write the different
flavors of netcdf readers? If so, where can the code be found?
thanks
-simon
On Fri, Mar 18, 2011 at 10:30 AM, Andy Bauer
<[email protected]<mailto:[email protected]>> wrote:
Hi Simon,
Replies below...
On Thu, Mar 17, 2011 at 12:41 PM, Simon Su
<[email protected]<mailto:[email protected]>> wrote:
Hi All,
ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/1.png
ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/2.png
ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/3.png
the above are screen shot from older version of *.nc files that I have that I
loaded up with Paraview 3.10.0 64-bit which I compiled myself. As you can see,
the grid is correctly loaded. But in the data, there is a crack. Is there a
filter that can fix this in ParaView? :)
I'm not aware of any filter that will fix this automatically. Can you
describe the grid a bit more? It kind of looks like a multiblock of structured
grids.
ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/4.png is the latest nc file that I
have of similar simulation preprocessed output. When I loaded it up, it is
clearly doing making assumption on the grid that is not correct and hence, the
double looking grid.
The questions now are:
1. ParaView has lots of *.nc file loader. How do I know which loader is
ParaView using to load the data? If I pick a type for the Files of type option
in the open File window, will that gurantees that ParaView will be using that
particular file loader?
If there is an ambiguity for which file loader to use (i.e. multiple readers
assume the same extension), then the GUI should pop up a dialog for you to
specify which one to use. You can use the python trace to figure out exactly
what reader is being used to load the file.
2. If I know which loader is used, I would like to see if I can modify the
existing loader to create a new type of *.nc file loader to fix the grid of my
data. Where is the code in the source tree of ParaView is the loader plugin
placed?
Based on the name of the name of the reader from the python script, you can
look up the actual class name in the
ParaView/Servers/ServerManager/Resources/readers.xml file.
or if there is an easier solution to fix my grid problem that doesn't involve
developing a new data loader plugin that would be better.... :) please let me
know...
thanks
-simon
_______________________________________________
Powered by www.kitware.com<http://www.kitware.com>
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
_______________________________________________ Powered by
www.kitware.com<http://www.kitware.com> Visit other Kitware open-source
projects at http://www.kitware.com/opensource/opensource.html Please keep
messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
_______________________________________________
Powered by www.kitware.com
Visit other Kitware open-source projects at
http://www.kitware.com/opensource/opensource.html
Please keep messages on-topic and check the ParaView Wiki at:
http://paraview.org/Wiki/ParaView
Follow this link to subscribe/unsubscribe:
http://www.paraview.org/mailman/listinfo/paraview
