Vincent Davis wrote: > I have a csv file I have to make some changes to before I read it into > pandas. Currently I open the csv read each row, make changes and save it > to a new file. Then read it into pandas with pandas.read_csv(). How do I > skip writing the file to disk? Using python3.5. > > This is what I am doing now. > > with open(infile,"r") as fin: > with open(outfile,"w") as fout: > writer=csv.writer(fout) > for row in csv.reader(fin): > #do stuff to the row > writer.writerow(row) > > df = pandas.csv_reader(outfile)
If you can implement "do stuff to the row" in pandas I'd recommend that. If not, the following should work (though I'm not a pandas expert): def preprocess(filename): with open(filename) as f: for row in csv.reader(f): # do stuff yield row rows = preprocess("pandas.csv") df = pandas.DataFrame.from_records(rows, columns=next(rows)) df = df.convert_objects(convert_numeric=True) If the csv file doesn't have a header row omit the columns=next(rows) argument. -- https://mail.python.org/mailman/listinfo/python-list