The wiki has now been updated and is working on my machine for the reader plugin example. I just converted the existing vtkPNGReader to vtkMyPNGReader to test. Thanks for letting us know it was incorrect.
Andy On Thu, Mar 24, 2011 at 1:24 PM, Simon Su <[email protected]> wrote: > Hi Andy, thanks.... :) > > Ken, the newer version produced by the scientific programming team has some > improvement. But the crack along the north pole (tripolar poles between Asia > and America continent) is still there. I saved a screenshot at > http://www.cs.uh.edu/~ssu/4ParaView/20110324/1.png > > the data file is a little more than 800MB if you can get to > > > ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/20110324/thetao_Omon_GFDL-ESM2M_historical_r1i1p1_194601-195012.nc > > I don't think it is right but I have don't have a point of reference. If > you have the time, can you point me to the data file which is proper or a > screen shot of a properly rendered vis of the CF convention file? > > thank you very much > -simon > > > > > > On Wed, Mar 23, 2011 at 10:47 PM, Andy Bauer <[email protected]>wrote: > >> FYI: I will take care of the wrong documentation. Not today and probably >> not tomorrow but soon :) >> >> Andy >> >> >> On Tue, Mar 22, 2011 at 3:07 PM, Simon Su <[email protected]>wrote: >> >>> Hi Ken, >>> >>> This is great. I will work with the scientific programming team that came >>> up with the file to see if we can go anywhere with it. Thank you so much >>> for taking the time to look into the file. >>> >>> best >>> -simon >>> >>> >>> On Tue, Mar 22, 2011 at 2:22 PM, Moreland, Kenneth <[email protected]>wrote: >>> >>>> Simon, >>>> >>>> After taking a look at your zos_*.nc file, I believe the problem is >>>> that the bounds variables (specifically lon_vertices and lat_vertices) are >>>> incorrect and causing malformed cells. The proper format for these >>>> variables according to the CF convention is documented here – >>>> http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/ch07.html#cell-boundaries >>>> – >>>> under "Bounds for 2-D coordinate variables with 4-sided cells". These >>>> variables are referred to as lonbnd and latbnd in the CF documentation, but >>>> otherwise the name is inconsequential. >>>> >>>> These bounds variables are supposed to contain a 4-tuple for each cell >>>> in the grid. The 4 values in each tuple specify the longitude or latitude >>>> coordinates in counterclockwise order starting with the "lower left" value. >>>> Adjacent cells should have duplicate longitude and latitude coordinates >>>> where appropriate. >>>> >>>> So, assuming that the bounds of the first cell (which I am >>>> extrapolating from other coordinate variables in the zos file) is –280 to >>>> –279 in the longitude direction and –82 to –81 in the latitude direction, >>>> then the first 4-tuple in lon_vertices should be [–280, –279, –279, –280] >>>> and the first tuple in lat_vertices should be [–82, –82, –81, –81]. The >>>> next entries in the lon_vertices and lat_vertices arrays are for the >>>> adjacent cell in the horizontal direction and should be >>>> [–279, –278, –278, –279] and [–82, –82, –81, –81], respectively. >>>> >>>> In this regard, the values stored in zon*.nc's lon_vertices and >>>> lat_vertices are nonsensical. ncdump shows the first two entries of each >>>> as: >>>> >>>> lon_vertices = >>>> -280, -279, -278, -277, >>>> -276, -275, -274, -273, >>>> >>>> lat_vertices = >>>> -82, -82, -82, -82, >>>> -82, -82, -82, -82, >>>> >>>> >>>> As you can see, these bounds do not form proper quadrilaterals. They >>>> are all degenerate polygons on the –82 latitude coordinate. My guess is >>>> that whatever wrote out this file mistook the format for boundary variables >>>> for coordinate variables. >>>> >>>> -Ken >>>> >>>> **** Kenneth Moreland >>>> *** Sandia National Laboratories >>>> *********** >>>> *** *** *** email: [email protected] >>>> ** *** ** phone: (505) 844-8919 >>>> *** web: http://www.cs.unm.edu/~kmorel >>>> >>>> From: Simon Su <[email protected]> >>>> Date: Tue, 22 Mar 2011 11:40:46 -0400 >>>> >>>> To: Kenneth Moreland <[email protected]> >>>> Cc: Andy Bauer <[email protected]>, "[email protected]" < >>>> [email protected]> >>>> Subject: Re: [Paraview] loading nc formatted data >>>> >>>> Hi Ken, >>>> >>>> Thank you again for your help. >>>> >>>> Is there another wiki page that also describe all the xml used syntax in >>>> the http://www.vtk.org/Wiki/ParaView/Plugin_HowTo ? It may help for me >>>> to have an overall picture of the xml syntax used? thanks. And is there a >>>> complete example out there that show the whole process on how one can >>>> create >>>> a custom data loader plugin into ParaView? Not sure if I am saying this >>>> correctly but something like, say I have this data file in my own format >>>> that has grid information and a scalar variable. How do I write a data >>>> loader plugin so that I can use ParaView to visualize the data. Then if I >>>> have to guess, first you write this xml file to describe the data loader >>>> plugin to ParaView and then use use sort of utility to generate the plugin >>>> standard code, and then implement the function in the code to load and >>>> create your vtk data structure (grid data type and scalar variable data >>>> type) and then pass it to ParaView. something like that? thanks... >>>> >>>> yes. it make sense that it is a part of VTK. ParaView is to expose VTK >>>> functionality after all. >>>> >>>> good thing is, I think we settle on a single large nc file that has all >>>> the information needed to plot the visualization, including tripolar grid >>>> information. Hopefully that will simplify things for me. >>>> >>>> I am sorry that the ftp site is not behaving. I have placed the files at >>>> http://www.cs.uh.edu/~ssu/4ParaView together with the nc file I used to >>>> generate the image 4.png (what looks like a double grid to me) and 5.png >>>> (coloring by the variable zos which I don't thin it should look like that >>>> ) Hopefully vtkNetCDFCFReader will do the job for me. >>>> >>>> thanks >>>> -simon >>>> >>>> >>>> >>>> On Mon, Mar 21, 2011 at 7:12 PM, Moreland, Kenneth >>>> <[email protected]>wrote: >>>> >>>>> It looks like the documentation for a reader plugin is slightly >>>>> messed up. I'll leave it to Andy (or anyone who is not me) to fix. The >>>>> example in the ParaView source does not list a file in >>>>> SERVER_MANAGER_SOURCES because it uses a source (vtkPNGReader.cxx) that is >>>>> already compiled as part of VTK. The Wiki is trying to capture the fact >>>>> that usually you are building a plugin with your own reader, and you will >>>>> have to list your source code for the reader there. vtkMyPNGReader.cxx is >>>>> just a stand in for some reader that you wrote. It probably does not >>>>> really >>>>> exist. I noticed, however, that the Wiki documentation is also wrong in >>>>> that the XML file is pointing to the vtkPNGReader class when the CMake >>>>> configuration is implying that it should be pointing to the (imaginary) >>>>> vtkMyPNGReader class. >>>>> >>>>> The vtkNetCDFCFReader.cxx file is located >>>>> in ParaView/VTK/IO/vtkNetCDFCFReader.cxx. Don't get too wrapped up around >>>>> the complexity of the NetCDF reader definition in readers.xml. It's >>>>> complicated because it supports defining a time series as a collection of >>>>> files ( >>>>> http://www.vtk.org/Wiki/Animating_legacy_VTK_file_series#Making_custom_readers_work_with_file_series). >>>>> If you collect all times in a single netCDF file (or don't have time >>>>> steps), you can skip all that and just define a new object much like that >>>>> of >>>>> netCDFReaderCore in reader.xml (in the sources ProxyGroup of course). >>>>> >>>>> That said, I think this should all be unnecessary since the CF >>>>> convention (and ParaView reader) already contains a means of defining >>>>> cells >>>>> that bridge across these seems. I suspect it has nothing to do with the >>>>> CF >>>>> convention version. It is probably that your data files with openings at >>>>> the seems are not defining a bounds attribute, which is optional in CF. >>>>> If >>>>> you could post some example data (or even the listing of the file header >>>>> from ncdump) I might be able to verify that. >>>>> >>>>> I'm not sure what is going on with the new data set (4.png). I can't >>>>> comment on it much because your ftp server seems to be broken for me and I >>>>> don't remember the image too well. At any rate, I would probably say >>>>> something like I couldn't tell you what was going on without seeing the nc >>>>> file. >>>>> >>>>> -Ken >>>>> >>>>> **** Kenneth Moreland >>>>> *** Sandia National Laboratories >>>>> *********** >>>>> *** *** *** email: [email protected] >>>>> ** *** ** phone: (505) 844-8919 >>>>> *** web: http://www.cs.unm.edu/~kmorel >>>>> >>>>> From: Simon Su <[email protected]> >>>>> Date: Mon, 21 Mar 2011 15:19:24 -0400 >>>>> To: Kenneth Moreland <[email protected]> >>>>> Cc: Andy Bauer <[email protected]>, "[email protected]" < >>>>> [email protected]> >>>>> >>>>> Subject: Re: [Paraview] loading nc formatted data >>>>> >>>>> Hi Ken, >>>>> >>>>> Thank you for explaining. >>>>> >>>>> The reader that is being used is vtkNetCDFCFReader. It is defined >>>>>> in ParaView/Servers/ServerManager/Resources/readers.xml, although it is >>>>>> not >>>>>> obvious. >>>>>> >>>>>> >>>>> Can you (or Andy) point me to documentation more documentation beside >>>>> http://www.vtk.org/Wiki/ParaView/Plugin_HowTo ? Andy also metioned >>>>> readers.xml file you mentioned. But after going through ParaView/Servers/ >>>>> ServerManager/Resources/readers.xml, I am still fuzzy about where the >>>>> file vtkNetCDFCFReader.cxx ( if any - I would like to know how it is >>>>> implemented and maybe start by modifying it in my learning process). The >>>>> plugin wiki page on "Adding a Reader" section mentioned CMakeList.txt file >>>>> >>>>> FIND_PACKAGE(ParaView REQUIRED) >>>>> INCLUDE(${PARAVIEW_USE_FILE}) >>>>> ADD_PARAVIEW_PLUGIN(MyReader "1.0" >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> SERVER_MANAGER_XML MyPNGReader.xml >>>>> SERVER_MANAGER_SOURCES vtkMyPNGReader.cxx >>>>> GUI_RESOURCE_FILES MyReaderGUI.xml) >>>>> >>>>> >>>>> but the one on my source tree >>>>> ParaView-3.10.0/Examples/Plugins/Reader/CMakeList.txt looks like >>>>> >>>>> ADD_PARAVIEW_PLUGIN(MyPNGReader "1.0" >>>>> SERVER_MANAGER_XML readers.xml >>>>> GUI_RESOURCE_FILES pqReader.xml >>>>> ) >>>>> >>>>> note no vtkMyPNGReader.cxx file mention and the file is also not in the >>>>> directory. I also did a search on source tree for vtkNetCDFCFReader.cxx >>>>> file but can't find it. I think I am missing a big piece of something >>>>> since >>>>> I am not getting the point that I should be getting after looking at >>>>> readers.xml. In fact, I can't find any of the netcdf file readers (other >>>>> than CF reader). It is also not listed under my Plugin Manager GUI on the >>>>> ParaView that I compiled but it is loading the CF convention nc file. Are >>>>> they (the *.cxx files) generated on the fly during compile time? >>>>> >>>>> Another thing, besides >>>>> http://www.paraview.org/Wiki/VisIt_Database_Bridge , is there a place >>>>> that I can find how to use that VisIt bridge in ParaView. Maybe an example >>>>> of how to load silo file? I actually have a plugin of my own in VisIt that >>>>> can load the climate modeling data that I have. I would like to see if I >>>>> can >>>>> use my VisIt plugin in ParaView and not write another plugin for ParaView? >>>>> :) >>>>> >>>>> >>>>> >>>>>> As the name implies, this reader reads netCDF files using the CF >>>>>> convention. (As the default netCDF reader, it also gracefully handles >>>>>> files >>>>>> that do not follow this convention.) With this assumption, I will try to >>>>>> explain what it does. It reads arrays as regular 1, 2, or 3D arrays, >>>>>> possibly with time. The CF convention also provides a means to assign >>>>>> coordinates to each grid point and to identify the coordinates as >>>>>> longitude >>>>>> or latitude. >>>>>> >>>>>> >>>>> The published nc files we have do adhere to CF convention we also use >>>>> CMOR in one of the pipeline. I noticed Data_tos_O1_2001-2002.nc is at >>>>> CF-1.0 and since I am new at CF Convention, I can only speculate that >>>>> what I >>>>> see at ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/4.png with what looks >>>>> like double grid is due to difference between CF-1.0 and CF-1.4? Will >>>>> vtkNetCDFCFReader handle CF-1.4 data? >>>>> >>>>> >>>>> >>>>> >>>>>> Even though coordinates are defined as longitude and latitude, the >>>>>> topology of the grid itself is still a grid. Thus, the grid gets wrapped >>>>>> around, but still has these seems that you see because topologically the >>>>>> one >>>>>> end of the regular grid is not attached to the other. I can't think of >>>>>> any >>>>>> filter that will identify and close these seems. In fact, it's not >>>>>> straightforward to do at all. If you look at your topology, it is not >>>>>> lain >>>>>> out on a simple spherical grid. >>>>>> >>>>>> The "right" way to solve your problem, which may or may not be in >>>>>> your control, is to create netCDF files that specify cell boundaries for >>>>>> a >>>>>> closed topology. Your netCDF file must be following at least some parts >>>>>> of >>>>>> the CF convention; your data would not show up as a sphere if it were >>>>>> not. >>>>>> The CF convention provides a way of defining cells that are not >>>>>> constrained >>>>>> by a regular grid topology. It is done through a "bounds" attribute on >>>>>> the >>>>>> dimension descriptor variables. You need either 1D or 2D bounds. They >>>>>> are >>>>>> described in this section of the CF convention documentation: >>>>>> http://cf-pcmdi.llnl.gov/documents/cf-conventions/1.5/cf-conventions.html#cell-boundaries >>>>>> . >>>>>> >>>>>> >>>>> Agreed. The hack we have in VisIt loader (replace the last grid value >>>>> with the 1st grid value to create a "closed topology" ) is not >>>>> "generalizable" and will blow up in my face if I hand it to the user. >>>>> >>>>> >>>>> thanks >>>>> -simon >>>>> >>>>> >>>>> >>>>> >>>>> >>>>>> The VTK test data has examples of both 1D bounds >>>>>> (tos_O1_2001-2002.nc) and 2D bounds (sampleCurveGrid4.nc). You can get >>>>>> the >>>>>> VTK test data from git (http://vtk.org/VTKData.git), or download them >>>>>> directly from the gitourious server: >>>>>> >>>>>> >>>>>> http://vtk.org/gitweb?p=VTKData.git;a=blob;f=Data/tos_O1_2001-2002.nc;h=30aa4a9b5e08b9bdf64540f2b144d83b279cca6c;hb=HEAD >>>>>> >>>>>> >>>>>> http://vtk.org/gitweb?p=VTKData.git;a=blob;f=Data/sampleCurveGrid4.nc;h=0ab89c27a25f92c047b58dca8b3057ca8d4df017;hb=HEAD >>>>>> >>>>>> >>>>>> -Ken >>>>>> >>>>>> **** Kenneth Moreland >>>>>> *** Sandia National Laboratories >>>>>> *********** >>>>>> *** *** *** email: *[email protected] >>>>>> *** *** ** phone: (505) 844-8919 >>>>>> *** web: *http://www.cs.unm.edu/~kmorel* >>>>>> >>>>>> From: Simon Su <[email protected]> >>>>>> Date: Fri, 18 Mar 2011 16:43:42 -0400 >>>>>> To: Andy Bauer <[email protected]> >>>>>> Cc: <[email protected]> >>>>>> Subject: Re: [Paraview] loading nc formatted data >>>>>> >>>>>> Hi Andy, >>>>>> >>>>>> python trace gave me >>>>>> >>>>>> zos_Omon_GFDLESM2M_historical_r1i1p1_186101188012_nc = >>>>>> NetCDFReader( >>>>>> FileName=['/work/sms/data/cmor-20110128/mon/ocean/zos/r1i1p1/zos_Omon_GFDL-ESM2M_historical_r1i1p1_186101-188012.nc'] >>>>>> ) >>>>>> >>>>>> and there are tons of netcdf reader in ParaView and it is not in the >>>>>> plugin directory >>>>>> >>>>>> sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> pwd >>>>>> /local/home/build/paraview/ParaView-3.10.0/Plugins >>>>>> sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> ll >>>>>> total 84 >>>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 AdiosReader/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 AnalyzeNIfTIReaderWriter/ >>>>>> -rw-r--r-- 1 sms t 3261 Mar 9 13:31 CMakeLists.txt >>>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 CoProcessingScriptGenerator/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 CosmoFilters/ >>>>>> drwxr-xr-x 4 sms t 4096 Mar 10 12:37 EyeDomeLighting/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 ForceTime/ >>>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 H5PartReader/ >>>>>> drwxr-xr-x 4 sms t 4096 Mar 10 12:37 Manta/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 Moments/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 NetDMFReader/ >>>>>> drwxr-xr-x 6 sms t 4096 Mar 10 12:37 PointSprite/ >>>>>> drwxr-xr-x 4 sms t 4096 Mar 10 12:37 PrismPlugins/ >>>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 pvblot/ >>>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 SierraPlotTools/ >>>>>> drwxr-xr-x 3 sms t 4096 Mar 10 12:37 SLACTools/ >>>>>> drwxr-xr-x 4 sms t 4096 Mar 10 12:37 StreamingView/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 SurfaceLIC/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 Vapor/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 10 12:37 VRPN/ >>>>>> drwxr-xr-x 2 sms t 4096 Mar 9 13:31 VRUI/ >>>>>> sms:/local/home/build/paraview/ParaView-3.10.0/Plugins> >>>>>> >>>>>> >>>>>> Can you help describe how Netcdf files plugins are done in ParaView. >>>>>> Do they have a super class of Netcdf that they all derive from to write >>>>>> the >>>>>> different flavors of netcdf readers? If so, where can the code be found? >>>>>> >>>>>> thanks >>>>>> -simon >>>>>> >>>>>> >>>>>> >>>>>> On Fri, Mar 18, 2011 at 10:30 AM, Andy Bauer >>>>>> <[email protected]>wrote: >>>>>> >>>>>>> Hi Simon, >>>>>>> >>>>>>> Replies below... >>>>>>> >>>>>>> On Thu, Mar 17, 2011 at 12:41 PM, Simon Su <[email protected] >>>>>>> > wrote: >>>>>>> >>>>>>>> Hi All, >>>>>>>> >>>>>>>> ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/1.png >>>>>>>> ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/2.png >>>>>>>> ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/3.png >>>>>>>> >>>>>>>> the above are screen shot from older version of *.nc files that I >>>>>>>> have that I loaded up with Paraview 3.10.0 64-bit which I compiled >>>>>>>> myself. >>>>>>>> As you can see, the grid is correctly loaded. But in the data, there >>>>>>>> is a >>>>>>>> crack. Is there a filter that can fix this in ParaView? :) >>>>>>>> >>>>>>> >>>>>>> I'm not aware of any filter that will fix this automatically. Can >>>>>>> you describe the grid a bit more? It kind of looks like a multiblock of >>>>>>> structured grids. >>>>>>> >>>>>>>> >>>>>>>> ftp://ftp.gfdl.noaa.gov/pub/sms/4ParaView/4.png is the latest nc >>>>>>>> file that I have of similar simulation preprocessed output. When I >>>>>>>> loaded it >>>>>>>> up, it is clearly doing making assumption on the grid that is not >>>>>>>> correct >>>>>>>> and hence, the double looking grid. >>>>>>>> >>>>>>>> The questions now are: >>>>>>>> >>>>>>>> 1. ParaView has lots of *.nc file loader. How do I know which loader >>>>>>>> is ParaView using to load the data? If I pick a type for the Files of >>>>>>>> type >>>>>>>> option in the open File window, will that gurantees that ParaView will >>>>>>>> be >>>>>>>> using that particular file loader? >>>>>>>> >>>>>>> >>>>>>> If there is an ambiguity for which file loader to use (i.e. multiple >>>>>>> readers assume the same extension), then the GUI should pop up a dialog >>>>>>> for >>>>>>> you to specify which one to use. You can use the python trace to >>>>>>> figure out >>>>>>> exactly what reader is being used to load the file. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> 2. If I know which loader is used, I would like to see if I can >>>>>>>> modify the existing loader to create a new type of *.nc file loader to >>>>>>>> fix >>>>>>>> the grid of my data. Where is the code in the source tree of ParaView >>>>>>>> is the >>>>>>>> loader plugin placed? >>>>>>>> >>>>>>> Based on the name of the name of the reader from the python script, >>>>>>> you can look up the actual class name in the >>>>>>> ParaView/Servers/ServerManager/Resources/readers.xml file. >>>>>>> >>>>>>> >>>>>>>> >>>>>>>> or if there is an easier solution to fix my grid problem that >>>>>>>> doesn't involve developing a new data loader plugin that would be >>>>>>>> better.... >>>>>>>> :) please let me know... >>>>>>>> >>>>>>>> thanks >>>>>>>> -simon >>>>>>>> >>>>>>>> >>>>>>>> _______________________________________________ >>>>>>>> Powered by www.kitware.com >>>>>>>> >>>>>>>> Visit other Kitware open-source projects at >>>>>>>> http://www.kitware.com/opensource/opensource.html >>>>>>>> >>>>>>>> Please keep messages on-topic and check the ParaView Wiki at: >>>>>>>> http://paraview.org/Wiki/ParaView >>>>>>>> >>>>>>>> Follow this link to subscribe/unsubscribe: >>>>>>>> http://www.paraview.org/mailman/listinfo/paraview >>>>>>>> >>>>>>>> >>>>>>> >>>>>> _______________________________________________ Powered by >>>>>> www.kitware.com Visit other Kitware open-source projects at >>>>>> http://www.kitware.com/opensource/opensource.html Please keep >>>>>> messages on-topic and check the ParaView Wiki at: >>>>>> http://paraview.org/Wiki/ParaView Follow this link to >>>>>> subscribe/unsubscribe: >>>>>> http://www.paraview.org/mailman/listinfo/paraview >>>>>> >>>>> >>>>> >>>> >>> >> >
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
