Hi Andy, thanks.... :) Ken, the newer version produced by the scientific programming team has some improvement. But the crack along the north pole (tripolar poles between Asia and America continent) is still there. I saved a screenshot at http://www.cs.uh.edu/~ssu/4ParaView/20110324/1.png
the data file is a little more than 800MB if you can get to ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/20110324/thetao_Omon_GFDL-ESM2M_historical_r1i1p1_194601-195012.nc I don't think it is right but I have don't have a point of reference. If you have the time, can you point me to the data file which is proper or a screen shot of a properly rendered vis of the CF convention file? thank you very much -simon On Wed, Mar 23, 2011 at 10:47 PM, Andy Bauer <[email protected]> wrote: > FYI: I will take care of the wrong documentation. Not today and probably > not tomorrow but soon :) > > Andy > > > On Tue, Mar 22, 2011 at 3:07 PM, Simon Su <[email protected]> wrote: > >> Hi Ken, >> >> This is great. I will work with the scientific programming team that came >> up with the file to see if we can go anywhere with it. Thank you so much >> for taking the time to look into the file. >> >> best >> -simon >> >> >> On Tue, Mar 22, 2011 at 2:22 PM, Moreland, Kenneth <[email protected]>wrote: >> >>> Simon, >>> >>> After taking a look at your zos_*.nc file, I believe the problem is >>> that the bounds variables (specifically lon_vertices and lat_vertices) are >>> incorrect and causing malformed cells. The proper format for these >>> variables according to the CF convention is documented here – >>> http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/ch07.html#cell-boundaries >>> – >>> under "Bounds for 2-D coordinate variables with 4-sided cells". These >>> variables are referred to as lonbnd and latbnd in the CF documentation, but >>> otherwise the name is inconsequential. >>> >>> These bounds variables are supposed to contain a 4-tuple for each cell >>> in the grid. The 4 values in each tuple specify the longitude or latitude >>> coordinates in counterclockwise order starting with the "lower left" value. >>> Adjacent cells should have duplicate longitude and latitude coordinates >>> where appropriate. >>> >>> So, assuming that the bounds of the first cell (which I am >>> extrapolating from other coordinate variables in the zos file) is –280 to >>> –279 in the longitude direction and –82 to –81 in the latitude direction, >>> then the first 4-tuple in lon_vertices should be [–280, –279, –279, –280] >>> and the first tuple in lat_vertices should be [–82, –82, –81, –81]. The >>> next entries in the lon_vertices and lat_vertices arrays are for the >>> adjacent cell in the horizontal direction and should be >>> [–279, –278, –278, –279] and [–82, –82, –81, –81], respectively. >>> >>> In this regard, the values stored in zon*.nc's lon_vertices and >>> lat_vertices are nonsensical. ncdump shows the first two entries of each >>> as: >>> >>> lon_vertices = >>> -280, -279, -278, -277, >>> -276, -275, -274, -273, >>> >>> lat_vertices = >>> -82, -82, -82, -82, >>> -82, -82, -82, -82, >>> >>> >>> As you can see, these bounds do not form proper quadrilaterals. They >>> are all degenerate polygons on the –82 latitude coordinate. My guess is >>> that whatever wrote out this file mistook the format for boundary variables >>> for coordinate variables. >>> >>> -Ken >>> >>> **** Kenneth Moreland >>> *** Sandia National Laboratories >>> *********** >>> *** *** *** email: [email protected] >>> ** *** ** phone: (505) 844-8919 >>> *** web: http://www.cs.unm.edu/~kmorel >>> >>> From: Simon Su <[email protected]> >>> Date: Tue, 22 Mar 2011 11:40:46 -0400 >>> >>> To: Kenneth Moreland <[email protected]> >>> Cc: Andy Bauer <[email protected]>, "[email protected]" < >>> [email protected]> >>> Subject: Re: [Paraview] loading nc formatted data >>> >>> Hi Ken, >>> >>> Thank you again for your help. >>> >>> Is there another wiki page that also describe all the xml used syntax in >>> the http://www.vtk.org/Wiki/ParaView/Plugin_HowTo ? It may help for me >>> to have an overall picture of the xml syntax used? thanks. And is there a >>> complete example out there that show the whole process on how one can create >>> a custom data loader plugin into ParaView? Not sure if I am saying this >>> correctly but something like, say I have this data file in my own format >>> that has grid information and a scalar variable. How do I write a data >>> loader plugin so that I can use ParaView to visualize the data. Then if I >>> have to guess, first you write this xml file to describe the data loader >>> plugin to ParaView and then use use sort of utility to generate the plugin >>> standard code, and then implement the function in the code to load and >>> create your vtk data structure (grid data type and scalar variable data >>> type) and then pass it to ParaView. something like that? thanks... >>> >>> yes. it make sense that it is a part of VTK. ParaView is to expose VTK >>> functionality after all. >>> >>> good thing is, I think we settle on a single large nc file that has all >>> the information needed to plot the visualization, including tripolar grid >>> information. Hopefully that will simplify things for me. >>> >>> I am sorry that the ftp site is not behaving. I have placed the files at >>> http://www.cs.uh.edu/~ssu/4ParaView together with the nc file I used to >>> generate the image 4.png (what looks like a double grid to me) and 5.png >>> (coloring by the variable zos which I don't thin it should look like that >>> ) Hopefully vtkNetCDFCFReader will do the job for me. >>> >>> thanks >>> -simon >>> >>> >>> >>> On Mon, Mar 21, 2011 at 7:12 PM, Moreland, Kenneth <[email protected]>wrote: >>> >>>> It looks like the documentation for a reader plugin is slightly >>>> messed up. I'll leave it to Andy (or anyone who is not me) to fix. The >>>> example in the ParaView source does not list a file in >>>> SERVER_MANAGER_SOURCES because it uses a source (vtkPNGReader.cxx) that is >>>> already compiled as part of VTK. The Wiki is trying to capture the fact >>>> that usually you are building a plugin with your own reader, and you will >>>> have to list your source code for the reader there. vtkMyPNGReader.cxx is >>>> just a stand in for some reader that you wrote. It probably does not >>>> really >>>> exist. I noticed, however, that the Wiki documentation is also wrong in >>>> that the XML file is pointing to the vtkPNGReader class when the CMake >>>> configuration is implying that it should be pointing to the (imaginary) >>>> vtkMyPNGReader class. >>>> >>>> The vtkNetCDFCFReader.cxx file is located >>>> in ParaView/VTK/IO/vtkNetCDFCFReader.cxx. Don't get too wrapped up around >>>> the complexity of the NetCDF reader definition in readers.xml. It's >>>> complicated because it supports defining a time series as a collection of >>>> files ( >>>> http://www.vtk.org/Wiki/Animating_legacy_VTK_file_series#Making_custom_readers_work_with_file_series). >>>> If you collect all times in a single netCDF file (or don't have time >>>> steps), you can skip all that and just define a new object much like that >>>> of >>>> netCDFReaderCore in reader.xml (in the sources ProxyGroup of course). >>>> >>>> That said, I think this should all be unnecessary since the CF >>>> convention (and ParaView reader) already contains a means of defining cells >>>> that bridge across these seems. I suspect it has nothing to do with the CF >>>> convention version. It is probably that your data files with openings at >>>> the seems are not defining a bounds attribute, which is optional in CF. If >>>> you could post some example data (or even the listing of the file header >>>> from ncdump) I might be able to verify that. >>>> >>>> I'm not sure what is going on with the new data set (4.png). I can't >>>> comment on it much because your ftp server seems to be broken for me and I >>>> don't remember the image too well. At any rate, I would probably say >>>> something like I couldn't tell you what was going on without seeing the nc >>>> file. >>>> >>>> -Ken >>>> >>>> **** Kenneth Moreland >>>> *** Sandia National Laboratories >>>> *********** >>>> *** *** *** email: [email protected] >>>> ** *** ** phone: (505) 844-8919 >>>> *** web: http://www.cs.unm.edu/~kmorel >>>> >>>> From: Simon Su <[email protected]> >>>> Date: Mon, 21 Mar 2011 15:19:24 -0400 >>>> To: Kenneth Moreland <[email protected]> >>>> Cc: Andy Bauer <[email protected]>, "[email protected]" < >>>> [email protected]> >>>> >>>> Subject: Re: [Paraview] loading nc formatted data >>>> >>>> Hi Ken, >>>> >>>> Thank you for explaining. >>>> >>>> The reader that is being used is vtkNetCDFCFReader. It is defined >>>>> in ParaView/Servers/ServerManager/Resources/readers.xml, although it is >>>>> not >>>>> obvious. >>>>> >>>>> >>>> Can you (or Andy) point me to documentation more documentation beside >>>> http://www.vtk.org/Wiki/ParaView/Plugin_HowTo ? Andy also metioned >>>> readers.xml file you mentioned. But after going through ParaView/Servers/ >>>> ServerManager/Resources/readers.xml, I am still fuzzy about where the >>>> file vtkNetCDFCFReader.cxx ( if any - I would like to know how it is >>>> implemented and maybe start by modifying it in my learning process). The >>>> plugin wiki page on "Adding a Reader" section mentioned CMakeList.txt file >>>> >>>> FIND_PACKAGE(ParaView REQUIRED) >>>> INCLUDE(${PARAVIEW_USE_FILE}) >>>> ADD_PARAVIEW_PLUGIN(MyReader "1.0" >>>> >>>> >>>> >>>> >>>> SERVER_MANAGER_XML MyPNGReader.xml >>>> SERVER_MANAGER_SOURCES vtkMyPNGReader.cxx >>>> GUI_RESOURCE_FILES MyReaderGUI.xml) >>>> >>>> >>>> but the one on my source tree >>>> ParaView-3.10.0/Examples/Plugins/Reader/CMakeList.txt looks like >>>> >>>> ADD_PARAVIEW_PLUGIN(MyPNGReader "1.0" >>>> SERVER_MANAGER_XML readers.xml >>>> GUI_RESOURCE_FILES pqReader.xml >>>> ) >>>> >>>> note no vtkMyPNGReader.cxx file mention and the file is also not in the >>>> directory. I also did a search on source tree for vtkNetCDFCFReader.cxx >>>> file but can't find it. I think I am missing a big piece of something >>>> since >>>> I am not getting the point that I should be getting after looking at >>>> readers.xml. In fact, I can't find any of the netcdf file readers (other >>>> than CF reader). It is also not listed under my Plugin Manager GUI on the >>>> ParaView that I compiled but it is loading the CF convention nc file. Are >>>> they (the *.cxx files) generated on the fly during compile time? >>>> >>>> Another thing, besides >>>> http://www.paraview.org/Wiki/VisIt_Database_Bridge , is there a place >>>> that I can find how to use that VisIt bridge in ParaView. Maybe an example >>>> of how to load silo file? I actually have a plugin of my own in VisIt that >>>> can load the climate modeling data that I have. I would like to see if I >>>> can >>>> use my VisIt plugin in ParaView and not write another plugin for ParaView? >>>> :) >>>> >>>> >>>> >>>>> As the name implies, this reader reads netCDF files using the CF >>>>> convention. (As the default netCDF reader, it also gracefully handles >>>>> files >>>>> that do not follow this convention.) With this assumption, I will try to >>>>> explain what it does. It reads arrays as regular 1, 2, or 3D arrays, >>>>> possibly with time. The CF convention also provides a means to assign >>>>> coordinates to each grid point and to identify the coordinates as >>>>> longitude >>>>> or latitude. >>>>> >>>>> >>>> The published nc files we have do adhere to CF convention we also use >>>> CMOR in one of the pipeline. I noticed Data_tos_O1_2001-2002.nc is at >>>> CF-1.0 and since I am new at CF Convention, I can only speculate that what >>>> I >>>> see at ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/4.png with what looks >>>> like double grid is due to difference between CF-1.0 and CF-1.4? Will >>>> vtkNetCDFCFReader handle CF-1.4 data? >>>> >>>> >>>> >>>> >>>>> Even though coordinates are defined as longitude and latitude, the >>>>> topology of the grid itself is still a grid. Thus, the grid gets wrapped >>>>> around, but still has these seems that you see because topologically the >>>>> one >>>>> end of the regular grid is not attached to the other. I can't think of >>>>> any >>>>> filter that will identify and close these seems. In fact, it's not >>>>> straightforward to do at all. If you look at your topology, it is not >>>>> lain >>>>> out on a simple spherical grid. >>>>> >>>>> The "right" way to solve your problem, which may or may not be in >>>>> your control, is to create netCDF files that specify cell boundaries for a >>>>> closed topology. Your netCDF file must be following at least some parts >>>>> of >>>>> the CF convention; your data would not show up as a sphere if it were not. >>>>> The CF convention provides a way of defining cells that are not >>>>> constrained >>>>> by a regular grid topology. It is done through a "bounds" attribute on >>>>> the >>>>> dimension descriptor variables. You need either 1D or 2D bounds. They >>>>> are >>>>> described in this section of the CF convention documentation: >>>>> http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#cell-boundaries >>>>> . >>>>> >>>>> >>>> Agreed. The hack we have in VisIt loader (replace the last grid value >>>> with the 1st grid value to create a "closed topology" ) is not >>>> "generalizable" and will blow up in my face if I hand it to the user. >>>> >>>> >>>> thanks >>>> -simon >>>> >>>> >>>> >>>> >>>> >>>>> The VTK test data has examples of both 1D bounds >>>>> (tos_O1_2001-2002.nc) and 2D bounds (sampleCurveGrid4.nc). You can get >>>>> the >>>>> VTK test data from git (http://vtk.org/VTKData.git), or download them >>>>> directly from the gitourious server: >>>>> >>>>> >>>>> http://vtk.org/gitweb?p=VTKData.git;a=blob;f=Data/tos_O1_2001-2002.nc;h=30aa4a9b5e08b9bdf64540f2b144d83b279cca6c;hb=HEAD >>>>> >>>>> >>>>> http://vtk.org/gitweb?p=VTKData.git;a=blob;f=Data/sampleCurveGrid4.nc;h=0ab89c27a25f92c047b58dca8b3057ca8d4df017;hb=HEAD >>>>> >>>>> >>>>> -Ken >>>>> >>>>> **** Kenneth Moreland >>>>> *** Sandia National Laboratories >>>>> *********** >>>>> *** *** *** email: *[email protected] >>>>> *** *** ** phone: (505) 844-8919 >>>>> *** web: *http://www.cs.unm.edu/~kmorel* >>>>> >>>>> From: Simon Su <[email protected]> >>>>> Date: Fri, 18 Mar 2011 16:43:42 -0400 >>>>> To: Andy Bauer <[email protected]> >>>>> Cc: <[email protected]> >>>>> Subject: Re: [Paraview] loading nc formatted data >>>>> >>>>> Hi Andy, >>>>> >>>>> python trace gave me >>>>> >>>>> zos_Omon_GFDLESM2M_historical_r1i1p1_186101188012_nc = >>>>> NetCDFReader( >>>>> FileName=['/work/sms/data/cmor-20110128/mon/ocean/zos/r1i1p1/zos_Omon_GFDL-ESM2M_historical_r1i1p1_186101-188012.nc'] >>>>> ) >>>>> >>>>> and there are tons of netcdf reader in ParaView and it is not in the >>>>> plugin directory >>>>> >>>>> sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> pwd >>>>> /local/home/build/paraview/ParaView-3.10.0/Plugins >>>>> sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> ll >>>>> total 84 >>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 AdiosReader/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 AnalyzeNIfTIReaderWriter/ >>>>> -rw-r--r-- 1 sms t 3261 Mar 9 13:31 CMakeLists.txt >>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 CoProcessingScriptGenerator/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 CosmoFilters/ >>>>> drwxr-xr-x 4 sms t 4096 Mar 10 12:37 EyeDomeLighting/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 ForceTime/ >>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 H5PartReader/ >>>>> drwxr-xr-x 4 sms t 4096 Mar 10 12:37 Manta/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 Moments/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 NetDMFReader/ >>>>> drwxr-xr-x 6 sms t 4096 Mar 10 12:37 PointSprite/ >>>>> drwxr-xr-x 4 sms t 4096 Mar 10 12:37 PrismPlugins/ >>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 pvblot/ >>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 SierraPlotTools/ >>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 SLACTools/ >>>>> drwxr-xr-x 4 sms t 4096 Mar 10 12:37 StreamingView/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 SurfaceLIC/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 Vapor/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 VRPN/ >>>>> drwxr-xr-x 2 sms t 4096 Mar 9 13:31 VRUI/ >>>>> sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> >>>>> >>>>> >>>>> Can you help describe how Netcdf files plugins are done in ParaView. Do >>>>> they have a super class of Netcdf that they all derive from to write the >>>>> different flavors of netcdf readers? If so, where can the code be found? >>>>> >>>>> thanks >>>>> -simon >>>>> >>>>> >>>>> >>>>> On Fri, Mar 18, 2011 at 10:30 AM, Andy Bauer >>>>> <[email protected]>wrote: >>>>> >>>>>> Hi Simon, >>>>>> >>>>>> Replies below... >>>>>> >>>>>> On Thu, Mar 17, 2011 at 12:41 PM, Simon Su >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> Hi All, >>>>>>> >>>>>>> ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/1.png >>>>>>> ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/2.png >>>>>>> ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/3.png >>>>>>> >>>>>>> the above are screen shot from older version of *.nc files that I >>>>>>> have that I loaded up with Paraview 3.10.0 64-bit which I compiled >>>>>>> myself. >>>>>>> As you can see, the grid is correctly loaded. But in the data, there is >>>>>>> a >>>>>>> crack. Is there a filter that can fix this in ParaView? :) >>>>>>> >>>>>> >>>>>> I'm not aware of any filter that will fix this automatically. Can >>>>>> you describe the grid a bit more? It kind of looks like a multiblock of >>>>>> structured grids. >>>>>> >>>>>>> >>>>>>> ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/4.png is the latest nc >>>>>>> file that I have of similar simulation preprocessed output. When I >>>>>>> loaded it >>>>>>> up, it is clearly doing making assumption on the grid that is not >>>>>>> correct >>>>>>> and hence, the double looking grid. >>>>>>> >>>>>>> The questions now are: >>>>>>> >>>>>>> 1. ParaView has lots of *.nc file loader. How do I know which loader >>>>>>> is ParaView using to load the data? If I pick a type for the Files of >>>>>>> type >>>>>>> option in the open File window, will that gurantees that ParaView will >>>>>>> be >>>>>>> using that particular file loader? >>>>>>> >>>>>> >>>>>> If there is an ambiguity for which file loader to use (i.e. multiple >>>>>> readers assume the same extension), then the GUI should pop up a dialog >>>>>> for >>>>>> you to specify which one to use. You can use the python trace to figure >>>>>> out >>>>>> exactly what reader is being used to load the file. >>>>>> >>>>>> >>>>>>> >>>>>>> 2. If I know which loader is used, I would like to see if I can >>>>>>> modify the existing loader to create a new type of *.nc file loader to >>>>>>> fix >>>>>>> the grid of my data. Where is the code in the source tree of ParaView >>>>>>> is the >>>>>>> loader plugin placed? >>>>>>> >>>>>> Based on the name of the name of the reader from the python script, >>>>>> you can look up the actual class name in the >>>>>> ParaView/Servers/ServerManager/Resources/readers.xml file. >>>>>> >>>>>> >>>>>>> >>>>>>> or if there is an easier solution to fix my grid problem that doesn't >>>>>>> involve developing a new data loader plugin that would be better.... :) >>>>>>> please let me know... >>>>>>> >>>>>>> thanks >>>>>>> -simon >>>>>>> >>>>>>> >>>>>>> _______________________________________________ >>>>>>> Powered by www.kitware.com >>>>>>> >>>>>>> Visit other Kitware open-source projects at >>>>>>> http://www.kitware.com/opensource/opensource.html >>>>>>> >>>>>>> Please keep messages on-topic and check the ParaView Wiki at: >>>>>>> http://paraview.org/Wiki/ParaView >>>>>>> >>>>>>> Follow this link to subscribe/unsubscribe: >>>>>>> http://www.paraview.org/mailman/listinfo/paraview >>>>>>> >>>>>>> >>>>>> >>>>> _______________________________________________ Powered by >>>>> www.kitware.com Visit other Kitware open-source projects at >>>>> http://www.kitware.com/opensource/opensource.html Please keep messages >>>>> on-topic and check the ParaView Wiki at: >>>>> http://paraview.org/Wiki/ParaView Follow this link to >>>>> subscribe/unsubscribe: >>>>> http://www.paraview.org/mailman/listinfo/paraview >>>>> >>>> >>>> >>> >> >
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
