The documentation shows a constructor DataStream(). Is the only way to get a DataStream to read it from a file using readstream?
James On Thursday, January 9, 2014 11:22:18 AM UTC-5, John Myles White wrote: > > I just restored the DataStream functionality. Still need to revise its > documentation, but (assuming you’re on master for both DataArrays and > DataFrames) you can try: > > using DataFrames > ds = readstream(“data.csv”, nrows = 100) > > for minibatch in ds > show(minibatch, true) > end > > — John > > On Jan 9, 2014, at 6:59 AM, Iain Dunning <[email protected] <javascript:>> > wrote: > > > So we have readlines, which loads the file all at once into an array of > strings, and eachline, which allows us to iterate through a file line by > line. > > > > We have readcsv/readdlm, but don't have an iterating version (as far as > I know). This is a bit of a problem because it means I have to manually > parse CSV files that are too big to read in all at once, which is error > prone (e.g. quoted delimiter handling). Is there demand for this, and what > would it be called? > > > > Cheers, > > Iain > >
