Doriano Blengino wrote: > > Read carefully the documentation about the WRITE instruction - in the > first few lines you find the explanation. When you write something out, > you can specify the length of the data you write. If you don't specify > it, gambas will do it for you, and this is what you don't want. >
I did exactly that last night, and also found the String$ function which was helpful in creating the string with all the zeros. I can now create a zero-value 8 GB file in about 60 seconds. My roots are in QuickBASIC 4.5, but my GAMBAS skills are improving the more I use it thankfully. This forum is probably the most helpful forum in forum history as well. Doriano Blengino wrote: > > I don't know the algorithm you want to use on so many data, but if you > look deep at it, may be you find some shortcut to minimize read and > write from a file. You could read data in chunks, calculate intermediate > results in ram, and so on. If you try to randomly read 65537*65537 small > pieces of data, your program will take forever to run, because the > overhead of input/output will be overimposed on every two bytes. By > reading 4 bytes at a time instead of two, you will gain twice the speed, > and so on. If the calculus is heavy, then there will be a point of best > compromise between the heavyness of I/O and heavyness of calculus. If > the calculus is very-very-very heavy, then the I/O time will be > neglectable. > > Another nice thing could be to compress data to and from disk, if data > can be managed by blocks. You could even reduce data to 1/10, and this > would be a big improvement. > Those are accurate observations and my main devil at this point. Here's an explanation of the algorithm: http://www.gameprogrammer.com/fractal.html#diamond http://en.wikipedia.org/wiki/Diamond-square_algorithm Basically think of the file as a square grid of byte pairs (shorts), each representing an elevation point on a map. Each pair of bytes is read directly into a short for processing as necessary. Initially the four corner points are read and the algorithm is applied, generating the center point which is then written back to the file (figure b in the first link). The second part of the algorithm uses these five points plus four points initially outside the file to generate four more points (figure c in the first link). It's the same as the first part of the algorithm, but rotated 45 degrees and performed four times in different areas. After the two algorithm variations are applied they are repeated at half-scale across all the points, exponentially increasing the number of iterations per pass (1x1, 4x4, 16x16, 64x64, etc.). The only thing I can think to do to increase I/O efficiency without adding more overhead to the number crunching (which is already significant) would be to assign each read point to a small array of shorts so the points would not have to be read from disk more than once per pass. Another idea would be to wait for the subdivisions to become small enough that one could be loaded at once into a huge array of shorts (2 GB, for instance), then operate solely on that chunk until it was finished and commit it back to disk. Because of the second part of the algorithm's 45 degree angle I'd still have to read some bytes that were outside the array, however. That would also disrupt my nice nested FOR...NEXT loop I have going to control the iterations. I fear the bottom line is this app is going to bust my huevos and ultimately take a long time to run. :( For those reading this who still need to know how to zero-out a big data file, here's my code modified for general use: ' General declarations. DIM Zeros AS String ' Used for zeroing-out file. DIM counter AS Long ' Generic counter. ' Open file to zero out. somefile = OPEN "somefile" FOR INPUT OUTPUT CREATE ' Create 8 GB file full of zeros in 1 MB writes. Zeros = String$(1048576, Chr$(0)) FOR counter = 1 TO 8192 WRITE #somefile, Zeros, 1048576 NEXT ' Add additional length as needed in 1 byte writes. Zeros = Chr$(0) FOR counter = 1 TO 262146 WRITE #somefile, Zeros, 1 NEXT ----- Kevin Fishburne, Eight Virtues www: http://sales.eightvirtues.com http://sales.eightvirtues.com e-mail: mailto:sa...@eightvirtues.com sa...@eightvirtues.com phone: (770) 853-6271 -- View this message in context: http://old.nabble.com/fastest-way-to-%22zero-out%22-a-huge-binary-file-tp27290885p27299072.html Sent from the gambas-user mailing list archive at Nabble.com. ------------------------------------------------------------------------------ Throughout its 18-year history, RSA Conference consistently attracts the world's best and brightest in the field, creating opportunities for Conference attendees to learn about information security's most important issues through interactions with peers, luminaries and emerging and established companies. http://p.sf.net/sfu/rsaconf-dev2dev _______________________________________________ Gambas-user mailing list Gambas-user@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/gambas-user
