I believe Visual studio 2010 does not properly handle file seeks/etc beyond 32 bit even on 64 bit builds. If you are using VS 2010 or earlier that is possibly the issue. I believe it is fixed in VS 2013 and later.
On Tue, Nov 22, 2016 at 5:11 PM, Moreland, Kenneth <[email protected]> wrote: > Although we have certainly run into issues with communicating large > messages, I’m not sure this is the issue with MPI-I/O. File sizes and > linear offsets use a type MPI_Offset, which according to the specification > should be large enough to hold the size of any file supported by the file > system. Although it is true that creating the 3D subarray uses 32-bit > indices, those are for each of the separate dimensions, and 2000 falls well > below that limit. > > > > Besides, it looks like that error happens when checking an ifstream > object, so it looks like the reader is bypassing the MPI-I/O path anyway. > > > > I would follow Berk’s suggesting of looking into the filesystem before > diving into the guts of MPI-I/O. > > > > -Ken > > > > *From:* ParaView [mailto:[email protected]] *On Behalf Of *Burlen > Loring > *Sent:* Tuesday, November 22, 2016 12:55 PM > *To:* Geveci, Berk (External Contact) <[email protected]>; Keyes > S.D. <[email protected]> > *Cc:* [email protected] > *Subject:* [EXTERNAL] Re: [Paraview] RAW (binary) file import issues > > > > I've hit that before. the RAW reader uses MPI-I/O, and MPI uses signed int > everywhere in its API, so there are limits that you end up hitting. For > instance to partition the data among processors one must give MPI-I/O the > starting offset of the data in the file as a signed int, and that is > limited to 2^31. > > Burlen > > On 11/22/2016 11:20 AM, Berk Geveci wrote: > > Hmmm that happens to be around 4 GB which makes me wonder if there is a > limit somewhere in that reader... Any chance you can try this on a Linux > machine? > > > > On Thu, Nov 17, 2016 at 10:30 AM, Keyes S.D. <[email protected]> > wrote: > > Dear all, > > I am encountering issues when trying to import large 8 bit RAW volumes to > Paraview 5.1.2 (64 bit). > > The file I wish to import has xyz dimensions of 2000x2000x1920 voxels > (Unsigned char), 7.5GB. I am running a machine with 192 GB RAM and 12 CPU > cores. > > I can import up to a 2000x2000x1000 crop of this file, but I encounter the > following message when I attempt larger z dimensions: > > ERROR: In C:\bbd\df0abce0\source-paraview\VTK\IO\Image\vtkImageReader2.cxx, > line 592 > > vtkPVImageReader (0000000007767F90): Initialize: Could not open file > \\cseg_2\ERC\SLS_201606\Myco_2\CT2PpR3h1_\Hyphalseg_Raw_ > 2000_2000_1920_8bit_SKELETON_FILT_PARAVIEW_TEST.raw > > > I cannot find an obvious memory ceiling within the paraview settings - any > ideas as to whether there is some effective upper limit to import file size? > > Sam > > Samuel D Keyes, MEng, PhD > New Frontiers Fellow > Bioengineering / μVIS Centre for Computed Tomography > University of Southampton > E: [email protected]<https://www.outlook.soton.ac.uk/owa/re > dir.aspx?SURL=dhUg3da8XR77LeItLDzJQwhAwmH0Wx6PHrXeKfxx2tOD3LCr737TCG0AYQBp > AGwAdABvADoAUwAuAEQALgBLAGUAeQBlAHMAQABzAG8AdABvAG4ALgBhAGMA > LgB1AGsA&URL=mailto%3aS.D.Keyes%40soton.ac.uk> > T: 07898720248 > _______________________________________________ > Powered by www.kitware.com > > Visit other Kitware open-source projects at http://www.kitware.com/opensou > rce/opensource.html > > Please keep messages on-topic and check the ParaView Wiki at: > http://paraview.org/Wiki/ParaView > > Search the list archives at: http://markmail.org/search/?q=ParaView > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/paraview > > > > > > > _______________________________________________ > > Powered by www.kitware.com > > > > Visit other Kitware open-source projects at > http://www.kitware.com/opensource/opensource.html > > > > Please keep messages on-topic and check the ParaView Wiki at: > http://paraview.org/Wiki/ParaView > > > > Search the list archives at: http://markmail.org/search/?q=ParaView > > > > Follow this link to subscribe/unsubscribe: > > http://public.kitware.com/mailman/listinfo/paraview > > > > _______________________________________________ > Powered by www.kitware.com > > Visit other Kitware open-source projects at http://www.kitware.com/ > opensource/opensource.html > > Please keep messages on-topic and check the ParaView Wiki at: > http://paraview.org/Wiki/ParaView > > Search the list archives at: http://markmail.org/search/?q=ParaView > > Follow this link to subscribe/unsubscribe: > http://public.kitware.com/mailman/listinfo/paraview > > -- Ken Martin PhD Chairman & CFO Kitware Inc. 28 Corporate Drive Clifton Park NY 12065 518 371 3971 This communication, including all attachments, contains confidential and legally privileged information, and it is intended only for the use of the addressee. Access to this email by anyone else is unauthorized. If you are not the intended recipient, any disclosure, copying, distribution or any action taken in reliance on it is prohibited and may be unlawful. If you received this communication in error please notify us immediately and destroy the original message. Thank you.
_______________________________________________ Powered by www.kitware.com Visit other Kitware open-source projects at http://www.kitware.com/opensource/opensource.html Please keep messages on-topic and check the ParaView Wiki at: http://paraview.org/Wiki/ParaView Search the list archives at: http://markmail.org/search/?q=ParaView Follow this link to subscribe/unsubscribe: http://public.kitware.com/mailman/listinfo/paraview
