Hi David, Here's the output from the code changes (parsed output of one scalar field), focusing on XdmfValuesBinary.cxx.
XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 231 (Accessing Binary CDATA) Seek as string is 3221225664 XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 267 (Data Size : 134217728) XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 268 (Size[Byte]: 1073741824) XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 269 ( Byte 8) XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 287 (Opening Binary Data for Reading : F:/rstrt//r SEEK as size_t is 2147483647 XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 322 (Seek: 2147483647) XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 331 (Hyperslab data) XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 161 (Reduce Rank: 3 to 1) XDMF Debug : ..\..\..\..\..\..\VTK\ThirdParty\xdmf2\vtkxdmf2\libsrc\XdmfValuesBinary.cxx line 178 (Contiguous byte: 1073741824) As a string, it looks like the seek location is being read right. Somewhere when it's converted to size_t, it defaults to 2147483647. Jesus ________________________________ From: David E DeMarle [[email protected]] Sent: Friday, August 14, 2015 6:33 AM To: Pulido, Jesus Cc: [email protected] Subject: Re: [Paraview] Memory seeking problems in using the xdmf2 reader Mind trying this? diff --git a/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx b/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx index 04ea0a7..d4cb2ed 100644 --- a/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx +++ b/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx @@ -187,7 +187,7 @@ public: size_t XdmfValuesBinary::getSeek(){ if(this->Seek==NULL)return 0; - return static_cast<size_t>(atoi(this->Seek)); + return static_cast<size_t>(atol(this->Seek)); } @@ -244,6 +244,7 @@ XdmfValuesBinary::Read(XdmfArray *anArray){ }else{ this->Seek = NULL; } + cerr << "Seek as string is " << Value << endl; } { XdmfConstString Value = this->Get("Compression"); @@ -291,6 +292,7 @@ XdmfValuesBinary::Read(XdmfArray *anArray){ //strcpy(path+strlen(this->DOM->GetWorkingDirectory()), DataSetName); try{ size_t seek = this->getSeek(); + cerr << "SEEK as size_t is " << seek << endl; switch(getCompressionType()){ case Zlib: XdmfDebug("Compression: Zlib"); David E DeMarle Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4909 On Thu, Jul 30, 2015 at 5:55 PM, Pulido, Jesus <[email protected]<mailto:[email protected]>> wrote: Hi David, The patch did not work, the file is still not read correctly after the second scalar component. After applying your patch, I did a bit of debugging myself. I enabled XDMF Debug in the code and found that the seek values are incorrect for scalar fields 3, 4, and 6. They seem to be stuck at the value 2147483647<tel:2147483647>. On the plus side, it looks like the array size (array size in bytes) is being computed correctly. Please take a look at the parsed debug log file that I've attached. Maybe the seek value is being malformed somewhere before it reaches XdmfValuesBinary.cxx? Thanks, Jesus ________________________________ From: David E DeMarle [[email protected]<mailto:[email protected]>] Sent: Wednesday, July 29, 2015 10:36 PM To: Pulido, Jesus Cc: [email protected]<mailto:[email protected]> Subject: Re: [Paraview] Memory seeking problems in using the xdmf2 reader Would you mind trying this Jesus? diff --git a/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx b/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx index 04ea0a7..6fe37d0 100644 --- a/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx +++ b/ThirdParty/xdmf2/vtkxdmf2/libsrc/XdmfValuesBinary.cxx @@ -187,7 +187,7 @@ public: size_t XdmfValuesBinary::getSeek(){ if(this->Seek==NULL)return 0; - return static_cast<size_t>(atoi(this->Seek)); + return static_cast<size_t>(atol(this->Seek)); } Once you confirm it fixes it for you too I'll check it in. Thanks, David E DeMarle Kitware, Inc. R&D Engineer 21 Corporate Drive Clifton Park, NY 12065-8662 Phone: 518-881-4909<tel:518-881-4909> On Fri, Jul 24, 2015 at 12:43 PM, Pulido, Jesus <[email protected]<mailto:[email protected]>> wrote: Hello, I have written a custom xmf file used by the xdmf2 reader to properly read in (raw) binary data into Paraview. This normally works but only for small datasets. The file requires the input of a memory "seek" location to start reading the different scalar components of the data. A problem arises when there is a 512^3 dataset, with 5 scalar components that are required to be read in. The first two scalar components are read in correctly, then the next two components receive the error: ERROR: In /Users/kitware/Dashboards/MyTests/NightlyMaster/ParaViewSuperbuild-Release-Python27/paraview/src/paraview/VTK/IO/Xdmf2/vtkXdmfHeavyData.cxx, line 1128 vtkXdmfReader (0x7f8614672af0): Failed to read attribute data The 5th component then becomes a duplicate of the 1st component. What I believe that's happening is that the "seek" value is only being represented as a signed integer, this would explain the memory location rolling from MAX_INT to -MAX_INT, and failing to read in scalar components 3 and 4. Their seek locations fall within this case. By the time the 5th component is read, the seek value may have rolled over back to positive values (seek > 0) and then duplicates the same data as the 1st component. Is this assumption correct and intended? Is there fix for this or an alternative way around reading large, raw datasets using this format? I'm attaching a sample xmf file Thank you, Jesus _______________________________________________ Powered by www.kitware.com<http://www.kitware.com> Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Search the list archives at: http://markmail.org/search/?q=ParaView Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/paraview
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Search the list archives at: http://markmail.org/search/?q=ParaView Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/paraview
