[Matplotlib-users] Dates interpretation on csv2rec
Dear all, I have a CSV file with the first column with timestamps: 0:00:00 01/01/2007, 0.000, 10, 0.000, 10, 0.000, 10: 00:00:00 02/01/2007, 0.000, 10, 0.000, 10, 0.000, 10 00:00:00 03/01/2007, 0.000, 10, 0.000, 10, 0.000, 10 ... 00:00:00 29/12/2009, 0.000, 10, 0.000, 10, 0.000, 10 00:00:00 30/12/2009, 0.000, 10, 0.000, 10, 0.000, 10 00:00:00 31/12/2009, 0.000, 10, 0.000, 10, 0.000, 10 As you can see, the format is hour:minute:second (nor relevant, always 00) day/month/year. When loaded with mlab.csv2rec, it automatically detects them as dates and creates datetime objects, but it is not consistent with the interpretation of the dates. When the day is < 13, it interprets it as month/day/year. It's loaded correctly otherwise (as day/month/year). I could pre-process the files and change the dates format, but imho it's not very elegant. Any suggestion about how to address this issue? Regards, Sergi -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Matplotlib-users mailing list Matplotlib-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Dates interpretation on csv2rec
> Dear all,
>
> I have a CSV file with the first column with timestamps:
>
> 0:00:00 01/01/2007, 0.000, 10, 0.000, 10,
> 0.000, 10:
> 00:00:00 02/01/2007, 0.000, 10, 0.000, 10,
> 0.000, 10
> 00:00:00 03/01/2007, 0.000, 10, 0.000, 10,
> 0.000, 10
> ...
> 00:00:00 29/12/2009, 0.000, 10, 0.000, 10,
> 0.000, 10
> 00:00:00 30/12/2009, 0.000, 10, 0.000, 10,
> 0.000, 10
> 00:00:00 31/12/2009, 0.000, 10, 0.000, 10,
> 0.000, 10
>
> As you can see, the format is hour:minute:second (nor relevant, always
> 00) day/month/year.
>
> When loaded with mlab.csv2rec, it automatically detects them as dates
> and creates datetime objects, but it is not consistent with the
> interpretation of the dates. When the day is < 13, it interprets it as
> month/day/year. It's loaded correctly otherwise (as day/month/year). I
> could pre-process the files and change the dates format, but imho it's
> not very elegant. Any suggestion about how to address this issue?
>
> Regards,
> Sergi
You could use numpy.genfromtxt together with its `converters` parameter:
converters : variable, optional
The set of functions that convert the data of a column to a value.
The converters can also be used to provide a default value
for missing data: ``converters = {3: lambda s: float(s or 0)}``.
Cheers,
A.
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
Re: [Matplotlib-users] Dates interpretation on csv2rec
On Fri, Jun 29, 2012 at 12:02 PM, Andreas Hilboll wrote:
>
> You could use numpy.genfromtxt together with its `converters` parameter:
>
> converters : variable, optional
> The set of functions that convert the data of a column to a value.
> The converters can also be used to provide a default value
> for missing data: ``converters = {3: lambda s: float(s or 0)}``.
I finally used:
def mkdate(text):
return(dt.datetime.strptime(text.decode('utf8'),\
'%H:%M:%S %d/%m/%Y'))
and
data = np.genfromtxt(\
os.path.join(sys.argv[1],f),\
delimiter=',',\
skip_header=4,\
usecols = (0, 1, 2),\
dtype=(object, float, float),\
converters={0: mkdate},\
autostrip=True,\
)
Thank you!
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Matplotlib-users mailing list
Matplotlib-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/matplotlib-users
