[Paraview] netcdf reader
Hi, I was having trouble reading data from a netcdf file to paraview. Is there some syntax that lets paraview know that it is reading in data in spherical coordinates? I was going through the 'help' section in paraview and found that 'if spherical coordinates is turned on' the netcdf reader will read in data in spherical coordinates. Is it something I have to implement as a part of the syntax in the input cdf file? Leo Paul. ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
Re: [Paraview] netCDF reader
>> I'm not sure vtkImageData (and everything that uses it) really supports >> negative spacing. See, for example, this email thread: >> http://www.vtk.org/pipermail/vtkusers/2009-May/100989.html. >> > I agree. And that's why I have added the function fabs to make positive the > tolerance variable. > > From ParaView3/VTK/IO/vtkNetCDFCOARDSReader.cxx (line 105...) > this->Spacing > = (this->Coordinates->GetValue(dimLen-1) - this->Origin)/(dimLen-1); > this->HasRegularSpacing = true; // Then check to see if it is false. > double tolerance = 0.01*this->Spacing; > for (size_t i = 1; i < dimLen; i++) > { > double expectedValue = this->Origin + i*this->Spacing; > double actualValue = this->Coordinates->GetValue(i); > if ( (actualValue < expectedValue-tolerance) > || (actualValue > expectedValue+tolerance) ) > { > this->HasRegularSpacing = false; > fprintf(stderr,"HasRegularSpacing = false"); > break; > } > } > > As I have written, if a coordinate variable goes from 90 to -90 > (decreasing) with dimLen=7 > (-90, -60, -30, 0, 30, 60, 90) > then Spacing variable is (-90 -90)/6 = -30 so negative, as tolerance > value equals to -0.3 > expectedValue for i equals 90 +30 = 120 instead of 60 ! > Consequence you get a "HasRegularSpacing = false" whereas the grid is > regular. > Put a simple fabs solves the problem > double tolerance = fabs(0.01*this->Spacing); > Is it clearer ? No. I already understand that if the tolerance gets a negative value, the HasRegularSpacing check will fail even if the spacing really is regular. And I understand how using abs will fix that. My point is that if the spacing is negative, then you cannot use a vtkImageData because vtkImageData does not support negative spacing. And the only reason for the HasRegularSpacing check is there in the first place is to decide whether to use a vtkImageData. So, the code would be clearer if there was a statement like if(this->Spacing<=0.0) { skip HasRegularSpacing check and use vtkRectilinearGrid instead of vtkImagedata }. But regardless, the code ends up doing the right thing even if technically HasRegularSpacing is set to false instead of true. >> >>> - Test on units should be case independant >>> line 133, units.find(" since ") >>> I have netCDF files where "since" is typed with uppercase (generated by >>> ferret software from the NOAA/PMEL). >>> >> >> That is easy enough to change. >> > Do you mean that the user have to change it by himself using ncrename > (netcdf operators). > I hope not and I consider that it should be treated within the reader. A > user does not care > about a typology of an attribut. > Having said that, I don't know how to lowercase a vtkStdString. I mean it is an easy fix in the code. I checked in a fix this morning. -Ken Kenneth Moreland *** Sandia National Laboratories *** *** *** *** email: kmo...@sandia.gov ** *** ** phone: (505) 844-8919 *** web: http://www.cs.unm.edu/~kmorel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
Re: [Paraview] netCDF reader
I have identified those points which causes problems with the actual /ParaView3/VTK/IO/vtkNetCDFCOARDSReader.cxx - if a regular spacing coordinate variable decreases (90 to -90 for example), you always get a vtkRectilinear grid. It should reach, wth the computing choice made, to the creation of vtkImageData. This is the case if you put the axis in an increasing order. You can correct this by placing a fabs set on tolerance variable 108c108 < double tolerance = fabs(0.01*this->Spacing); --- double tolerance = 0.01*this->Spacing; I'm not sure vtkImageData (and everything that uses it) really supports negative spacing. See, for example, this email thread: http://www.vtk.org/pipermail/vtkusers/2009-May/100989.html. I agree. And that's why I have added the function fabs to make positive the tolerance variable. From ParaView3/VTK/IO/vtkNetCDFCOARDSReader.cxx (line 105...) this->Spacing = (this->Coordinates->GetValue(dimLen-1) - this->Origin)/(dimLen-1); this->HasRegularSpacing = true; // Then check to see if it is false. double tolerance = 0.01*this->Spacing; for (size_t i = 1; i < dimLen; i++) { double expectedValue = this->Origin + i*this->Spacing; double actualValue = this->Coordinates->GetValue(i); if ( (actualValue < expectedValue-tolerance) || (actualValue > expectedValue+tolerance) ) { this->HasRegularSpacing = false; fprintf(stderr,"HasRegularSpacing = false"); break; } } As I have written, if a coordinate variable goes from 90 to -90 (decreasing) with dimLen=7 (-90, -60, -30, 0, 30, 60, 90) then Spacing variable is (-90 -90)/6 = -30 so negative, as tolerance value equals to -0.3 expectedValue for i equals 90 +30 = 120 instead of 60 ! Consequence you get a "HasRegularSpacing = false" whereas the grid is regular. Put a simple fabs solves the problem double tolerance = fabs(0.01*this->Spacing); Is it clearer ? - Test on units should be case independant line 133, units.find(" since ") I have netCDF files where "since" is typed with uppercase (generated by ferret software from the NOAA/PMEL). That is easy enough to change. Do you mean that the user have to change it by himself using ncrename (netcdf operators). I hope not and I consider that it should be treated within the reader. A user does not care about a typology of an attribut. Having said that, I don't know how to lowercase a vtkStdString. User choices for zorigin and zscale should be taken in consideration . Both set to 1 by default. In addition, a user choice zpositive to tell the direction in which the coordinate values are increasing should also be proposed. With allowed values : up or down (up by default). Perhaps also a zorder choice to treat files where zdimension has to be reordered to get k=1 representing surface. I am confused here. Where are zorigin and zscale defined? I do not see any mention of them in the CF or COARDS specifications. Indeed CF or COARDS conventions are not concerned by the projection made from a lon,lat grid. But define a height variable as follows height = zorigin + fabs(zpositive) * zscale * heightCoords->GetValue(k) (with zorigin, zorigin, zscale = 1 by default) would be better than simply define it as height = heightCoords->GetValue(k) because heightCoords can be negative, have to be scaled, can be equal to 0 (surface). There is also the problem of connectivity and blank cells (and Z clipping range - in spherical coordinates you don't want to see the globe when missing cells exist). Of course I have noticed this. I have left it this way because it is not clear what the right solution is. There are two issues to closing the grid. The first is that the CF/COARDS convention does not really specify whether the grid is actually supposed to be closed. It is perfectly valid to have an open sphere (in fact common in the latitudinal dimension). Of course, a simple check of the distance between the two should clear that up. Or perhaps looking at the bounds of the data. The modulo aspect concerns longitude, not latitude. In the COARDS convention, one can consider that the bounds of each cell is defined at the middle of 2 center cells. From a coordinate variable as follows lon(imt)= {30, 90, 150, 210, 270, 330} we can calculate its boundaries bounds_lon(imt,2) = {0,60, 60,120, 120,180, 180,240, 240,300, 300,360} There is a modulo longitude to set if first lower cell equals to last upper cell (use cosinus). Here cos(0)=cos(360) so the grid is modulo and the vtkStructuredGrid has to be topologically closed Plus, the fact that CF conventions covers also curvilinear grids where longitudes and latitudes are described with a 2D arrays. Uh, isn't this the only way to define curvilinear grids in CF? It is already supported by the netCDF reader. Not sure of that. Try from the example file: http://dods.ipsl.jussieu.fr/prism/gridsCF/sample
Re: [Paraview] netCDF reader
> I have identified those points which causes problems with the actual > /ParaView3/VTK/IO/vtkNetCDFCOARDSReader.cxx > - if a regular spacing coordinate variable decreases (90 to -90 for > example), you always get a vtkRectilinear grid. > It should reach, wth the computing choice made, to the creation of > vtkImageData. This is the case if you put > the axis in an increasing order. > You can correct this by placing a fabs set on tolerance variable > 108c108 > < double tolerance = fabs(0.01*this->Spacing); > --- > > double tolerance = 0.01*this->Spacing; I'm not sure vtkImageData (and everything that uses it) really supports negative spacing. See, for example, this email thread: http://www.vtk.org/pipermail/vtkusers/2009-May/100989.html. > - Test on units should be case independant > line 133, units.find(" since ") > I have netCDF files where "since" is typed with uppercase (generated by > ferret software from the NOAA/PMEL). That is easy enough to change. > Spherical coordinates should be possible in any case (regular spacing or > not), even with a 2D variable where > height is 0 because field is at surface. Yup. I already made those changes. > User choices for zorigin and zscale should be taken in consideration . > Both set to 1 by default. > In addition, a user choice zpositive to tell the direction in which the > coordinate values are increasing > should also be proposed. With allowed values : up or down (up by default). > Perhaps also a zorder choice to treat files where zdimension has to be > reordered to get k=1 representing > surface. I am confused here. Where are zorigin and zscale defined? I do not see any mention of them in the CF or COARDS specifications. > > There is also the problem of connectivity and blank cells (and Z > clipping range - in spherical coordinates you > don't want to see the globe when missing cells exist). Of course I have noticed this. I have left it this way because it is not clear what the right solution is. There are two issues to closing the grid. The first is that the CF/COARDS convention does not really specify whether the grid is actually supposed to be closed. It is perfectly valid to have an open sphere (in fact common in the latitudinal dimension). Of course, a simple check of the distance between the two should clear that up. Or perhaps looking at the bounds of the data. The second issue is one of connectivity. More on that below. > Plus, the fact that CF conventions covers also curvilinear grids where > longitudes and latitudes > are described with a 2D arrays. Uh, isn't this the only way to define curvilinear grids in CF? It is already supported by the netCDF reader. > This is why I haved used a vtkPolydata > structure (with limitation > that the application was not really to visualize 3D field, only 2D > fields in a 3D mapping when > projection uses spherical coordinates. Oh. I see why you proposed using poly data now. I still don't think this is feasible because of the 3D grid issue. However, we could change the netCDF reader to generate a structured grid that would be spatially closed but not topologically closed. If you needed the data to be topologically closed, you could always run the clean to grid filter, which would convert it to an unstructured grid that would support such topology and remove the duplicated points. -Ken ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
Re: [Paraview] netCDF reader
Moreland, Kenneth a écrit : The netCDF reader in ParaView implements the COARDS convention, which is a subset of the CF convention. To the best of my knowledge, both conventions define the spherical coordinates in the same way. So if you have a netCDF file containing the CF convention for latitude and longitude, then the ParaView netCDF reader should understand that. That said, I see that the netCDF reader will only actually create spherical coordinates for 3D grids. I am not sure I remember the logic behind that, but I think it was that in general it is usually better to visualize a map surface as a 2D projection to prevent occlusion and that it is ambiguous what the height of the surface should be. That said, it is easy to implement spherical coordinates to 2D grids. I just checked in a change that creates a sphere at radius 1. The idea of the netCDF reader outputting a vtkPolyData is unworkable. Many netCDF files contain 3D grids, and vtkPolyData is incapable of representing those. It could always output a vtkUnstructuredGrid, but this is a highly inefficient format for storing the structured arrays stored in netCDF COARDS files. It is conceivable that the netCDF reader could always output a vtkStructuredGrid. But that means that non-COARDS files are forced to also output vtkStructuredGrid and they loose the features for using vtkImageData. I know there are issues with the VTK/ParaView pipeline when you change the output of the reader when you change the dimensions. But there are also similar problems when you change the extent and dimensionality anyway, so I don’t know how much help such a change would make. Kenneth, Thank you for your answer. Indeed my main reproach for that filter is that I get different structures (a vtkImage or a vtkRectilinear Grid or a vtkStructuredGrid) following the netCDF grid read. COARDS convention deals with regular grids in a sens that the cross product of the coordinates variables (1D) define the grid. Define different structures because the grid has regular spacing or not is not what I would have computed. I am doing some tests and I will propose something using only vtkStructuredGrid. I have identified those points which causes problems with the actual /ParaView3/VTK/IO/vtkNetCDFCOARDSReader.cxx - if a regular spacing coordinate variable decreases (90 to -90 for example), you always get a vtkRectilinear grid. It should reach, wth the computing choice made, to the creation of vtkImageData. This is the case if you put the axis in an increasing order. You can correct this by placing a fabs set on tolerance variable 108c108 < double tolerance = fabs(0.01*this->Spacing); --- > double tolerance = 0.01*this->Spacing; - Test on units should be case independant line 133, units.find(" since ") I have netCDF files where "since" is typed with uppercase (generated by ferret software from the NOAA/PMEL). Spherical coordinates should be possible in any case (regular spacing or not), even with a 2D variable where height is 0 because field is at surface. User choices for zorigin and zscale should be taken in consideration . Both set to 1 by default. In addition, a user choice zpositive to tell the direction in which the coordinate values are increasing should also be proposed. With allowed values : up or down (up by default). Perhaps also a zorder choice to treat files where zdimension has to be reordered to get k=1 representing surface. There is also the problem of connectivity and blank cells (and Z clipping range - in spherical coordinates you don't want to see the globe when missing cells exist). Plus, the fact that CF conventions covers also curvilinear grids where longitudes and latitudes are described with a 2D arrays. This is why I haved used a vtkPolydata structure (with limitation that the application was not really to visualize 3D field, only 2D fields in a 3D mapping when projection uses spherical coordinates. Thanks for reading. Patrick -- LSCE/IPSL, Laboratoire CEA-CNRS-UVSQ Data Analysis and Visualization Engineer IPSL Global Climate Modelling Group mailto:Patrick.Brockmann @ cea.fr -- ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
Re: [Paraview] netCDF reader
The netCDF reader in ParaView implements the COARDS convention, which is a subset of the CF convention. To the best of my knowledge, both conventions define the spherical coordinates in the same way. So if you have a netCDF file containing the CF convention for latitude and longitude, then the ParaView netCDF reader should understand that. That said, I see that the netCDF reader will only actually create spherical coordinates for 3D grids. I am not sure I remember the logic behind that, but I think it was that in general it is usually better to visualize a map surface as a 2D projection to prevent occlusion and that it is ambiguous what the height of the surface should be. That said, it is easy to implement spherical coordinates to 2D grids. I just checked in a change that creates a sphere at radius 1. The idea of the netCDF reader outputting a vtkPolyData is unworkable. Many netCDF files contain 3D grids, and vtkPolyData is incapable of representing those. It could always output a vtkUnstructuredGrid, but this is a highly inefficient format for storing the structured arrays stored in netCDF COARDS files. It is conceivable that the netCDF reader could always output a vtkStructuredGrid. But that means that non-COARDS files are forced to also output vtkStructuredGrid and they loose the features for using vtkImageData. I know there are issues with the VTK/ParaView pipeline when you change the output of the reader when you change the dimensions. But there are also similar problems when you change the extent and dimensionality anyway, so I don't know how much help such a change would make. -Ken On 1/12/10 5:52 PM, "Brockmann Patrick" wrote: Hi all, Testing the netCDF reader with "spherical coordinates" set, I get a vtkStructured Grid incomplete. There is no circular topology. I was expecting a complete sphere where points from first column would be connected to points from last column in the vtkStructuredGrid created. Is there a way to do this from an existing filter or from the python programmable filter ? Or better could the netCDF reader be corrected (VTK/IO/vtkNetCDFCOARDSReader.cxx) ? By the past, I have installed and used the external plugin CSCSnetCDF reader from John Biddiscombe of the Swiss National Supercomputing Centre. I was quite happy with it. Today, I discover that paraview binairies are distributed with the netCDFCOARDS reader with some features I don't like. For example, that fact that the vtk structure created are different following the netCDF grid topology (regular spacing or not, spherical coordinates or not). I think it would be attractive for all the General Circulation Model (GCM)/ /community to provide a netCDF reader, closer to the CF (Climate and Forecast netCDF convention) widely adopted. This reader would create for example a same vtkPolydata structure whatever the netCDF grid read as in the vtk application described from http://www.prism.enes.org/Publications/Reports/Report19.pdf Is there any interest to develop this netCDFCF reader among offcial dev team ? Thanks Patrick ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview Kenneth Moreland *** Sandia National Laboratories *** *** *** *** email: kmo...@sandia.gov ** *** ** phone: (505) 844-8919 *** web: http://www.cs.unm.edu/~kmorel ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview
[Paraview] netCDF reader
Hi all, Testing the netCDF reader with "spherical coordinates" set, I get a vtkStructured Grid incomplete. There is no circular topology. I was expecting a complete sphere where points from first column would be connected to points from last column in the vtkStructuredGrid created. Is there a way to do this from an existing filter or from the python programmable filter ? Or better could the netCDF reader be corrected (VTK/IO/vtkNetCDFCOARDSReader.cxx) ? By the past, I have installed and used the external plugin CSCSnetCDF reader from John Biddiscombe of the Swiss National Supercomputing Centre. I was quite happy with it. Today, I discover that paraview binairies are distributed with the netCDFCOARDS reader with some features I don't like. For example, that fact that the vtk structure created are different following the netCDF grid topology (regular spacing or not, spherical coordinates or not). I think it would be attractive for all the General Circulation Model (GCM)/ /community to provide a netCDF reader, closer to the CF (Climate and Forecast netCDF convention) widely adopted. This reader would create for example a same vtkPolydata structure whatever the netCDF grid read as in the vtk application described from http://www.prism.enes.org/Publications/Reports/Report19.pdf Is there any interest to develop this netCDFCF reader among offcial dev team ? Thanks Patrick ___ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Follow this link to subscribe/unsubscribe: http://www.paraview.org/mailman/listinfo/paraview