As others on this list, I've also been confused a bit by the prolific numpy
interfaces to reading text. Would it be an idea to create some sort of object
oriented solution for this purpose?
reader = np.FileReader('my_file.txt')
reader.loadtxt() # for backwards compat.; np.loadtxt could instantia
For convenience, here's a link to the mailing list thread on this topic
from a couple months ago:
http://thread.gmane.org/gmane.comp.python.numeric.general/47094 .
Drew
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/ma
Hey all,
I would like to gather concrete information about NumPy users and have some
data to look at regarding the user base and features that are of interest.
We have been putting together a survey that I would love feedback on from
members of this list. If you have time and are intere
Hi,
On Thu, Feb 23, 2012 at 2:56 PM, Pierre Haessig
wrote:
> Le 23/02/2012 20:08, Mark Wiebe a écrit :
>> +1, I think it's good for its name to correspond to the name in C/C++,
>> so that when people search for information on it they will find the
>> relevant information more easily. With a bunch
===
Announcing Theano 0.5
===
This is a major version, with lots of new features, bug fixes, and some
interface changes (deprecated or potentially misleading features were
removed).
Upgrading to Theano 0.5 is recommended for everyone, but you shoul
Hi there,
I'm having a problem building NumPy on Python 2.7.1 and OS X 10.7.3. Here is my
build log:
https://gist.github.com/1895377
Does anyone have any idea what might be happening? I get a very similar error
when compiling with clang.
Installing a binary really isn't an option for me due t
Le 23/02/2012 22:38, Benjamin Root a écrit :
> labmate/officemate/advisor is using Excel...
... or an industrial partner with its windows-based software that can
export (when it works) some very nice field data from a proprietary
Honeywell data logger.
CSV data is better than no data ! (and better
On Thu, Feb 23, 2012 at 4:20 PM, Erin Sheldon wrote:
> Excerpts from Wes McKinney's message of Thu Feb 23 16:07:04 -0500 2012:
>> That's pretty good. That's faster than pandas's csv-module+Cython
>> approach almost certainly (but I haven't run your code to get a read
>> on how much my hardware mak
On Thu, Feb 23, 2012 at 3:14 PM, Robert Kern wrote:
> On Thu, Feb 23, 2012 at 21:09, Gael Varoquaux
> wrote:
> > On Thu, Feb 23, 2012 at 04:07:04PM -0500, Wes McKinney wrote:
> >> In this last case for example, around 500 MB of RAM is taken up for an
> >> array that should only be about 80-90MB.
Excerpts from Wes McKinney's message of Thu Feb 23 16:07:04 -0500 2012:
> That's pretty good. That's faster than pandas's csv-module+Cython
> approach almost certainly (but I haven't run your code to get a read
> on how much my hardware makes a difference), but that's not shocking
> at all:
>
> In
On Thu, Feb 23, 2012 at 21:09, Gael Varoquaux
wrote:
> On Thu, Feb 23, 2012 at 04:07:04PM -0500, Wes McKinney wrote:
>> In this last case for example, around 500 MB of RAM is taken up for an
>> array that should only be about 80-90MB. If you're a data scientist
>> working in Python, this is _not g
> But why, oh why, are people storing big data in CSV?
Well, that's what scientist do :-)
Éric.
>
> G
> ___
> NumPy-Discussion mailing list
> NumPy-Discussion@scipy.org
> http://mail.scipy.org/mailman/listinfo/numpy-discussion
Un clavier azerty en vaut
On Thu, Feb 23, 2012 at 04:07:04PM -0500, Wes McKinney wrote:
> In this last case for example, around 500 MB of RAM is taken up for an
> array that should only be about 80-90MB. If you're a data scientist
> working in Python, this is _not good_.
But why, oh why, are people storing big data in CSV?
On Thu, Feb 23, 2012 at 3:55 PM, Erin Sheldon wrote:
> Excerpts from Wes McKinney's message of Thu Feb 23 15:45:18 -0500 2012:
>> Reasonably wide CSV files with hundreds of thousands to millions of
>> rows. I have a separate interest in JSON handling but that is a
>> different kind of problem, and
Excerpts from Wes McKinney's message of Thu Feb 23 15:45:18 -0500 2012:
> Reasonably wide CSV files with hundreds of thousands to millions of
> rows. I have a separate interest in JSON handling but that is a
> different kind of problem, and probably just a matter of forking
> ultrajson and having i
On Thu, Feb 23, 2012 at 3:31 PM, Éric Depagne wrote:
> Le jeudi 23 février 2012 21:24:28, Wes McKinney a écrit :
>>
> That would indeed be great. Reading large files is a real pain whatever the
> python method used.
>
> BTW, could you tell us what you mean by large files?
>
> cheers,
> Éric.
Reas
Le 23/02/2012 21:08, Travis Oliphant a écrit :
> I think loadtxt is now the 3rd or 4th "text-reading" interface I've seen in
> NumPy.
Ok, now I understand why I got confused ;-)
--
Pierre
signature.asc
Description: OpenPGP digital signature
___
Num
Le 23/02/2012 20:32, Wes McKinney a écrit :
> If anyone wants to get involved in this particular problem right
> now, let me know!
Hi Wes,
I'm totally out of the implementations issues you described, but I have
some million-lines-long CSV files so that I experience "some slowdown"
when loading tho
On Feb 23, 2012, at 2:19 PM, Neal Becker wrote:
> Pauli Virtanen wrote:
>
>> 23.02.2012 20:44, Francesc Alted kirjoitti:
>>> On Feb 23, 2012, at 1:33 PM, Neal Becker wrote:
>>>
Is mkl only used for linear algebra? Will it speed up e.g., elementwise
transendental functions?
>>>
>>> Ye
Excerpts from Wes McKinney's message of Thu Feb 23 15:24:44 -0500 2012:
> On Thu, Feb 23, 2012 at 3:23 PM, Erin Sheldon wrote:
> > I designed the recfile package to fill this need. It might be a start.
> Can you relicense as BSD-compatible?
If required, that would be fine with me.
-e
>
> > Exc
Le jeudi 23 février 2012 21:24:28, Wes McKinney a écrit :
>
That would indeed be great. Reading large files is a real pain whatever the
python method used.
BTW, could you tell us what you mean by large files?
cheers,
Éric.
> Sweet, between this, Continuum folks, and me and my guys I think we
On Thu, Feb 23, 2012 at 3:23 PM, Erin Sheldon wrote:
> Wes -
>
> I designed the recfile package to fill this need. It might be a start.
>
> Some features:
>
> - the ability to efficiently read any subset of the data without
> loading the whole file.
> - reads directly into a recarray,
On Thu, Feb 23, 2012 at 3:19 PM, Warren Weckesser
wrote:
>
> On Thu, Feb 23, 2012 at 2:08 PM, Travis Oliphant
> wrote:
>>
>> This is actually on my short-list as well --- it just didn't make it to
>> the list.
>>
>> In fact, we have someone starting work on it this week. It is his first
>> proje
Wes -
I designed the recfile package to fill this need. It might be a start.
Some features:
- the ability to efficiently read any subset of the data without
loading the whole file.
- reads directly into a recarray, so no overheads.
- object oriented interface, mimicking rec
On Thu, Feb 23, 2012 at 2:08 PM, Travis Oliphant wrote:
> This is actually on my short-list as well --- it just didn't make it to
> the list.
>
> In fact, we have someone starting work on it this week. It is his first
> project so it will take him a little time to get up to speed on it, but he
>
Pauli Virtanen wrote:
> 23.02.2012 20:44, Francesc Alted kirjoitti:
>> On Feb 23, 2012, at 1:33 PM, Neal Becker wrote:
>>
>>> Is mkl only used for linear algebra? Will it speed up e.g., elementwise
>>> transendental functions?
>>
>> Yes, MKL comes with VML that has this type of optimizations:
>
On Thu, Feb 23, 2012 at 3:08 PM, Travis Oliphant wrote:
> This is actually on my short-list as well --- it just didn't make it to the
> list.
>
> In fact, we have someone starting work on it this week. It is his first
> project so it will take him a little time to get up to speed on it, but he
This is actually on my short-list as well --- it just didn't make it to the
list.
In fact, we have someone starting work on it this week. It is his first
project so it will take him a little time to get up to speed on it, but he will
contact Wes and work with him and report progress to this l
Le 23/02/2012 20:08, Mark Wiebe a écrit :
> +1, I think it's good for its name to correspond to the name in C/C++,
> so that when people search for information on it they will find the
> relevant information more easily. With a bunch of NumPy-specific
> aliases, it just creates more hassle for ever
Hi,
23.02.2012 20:32, Wes McKinney kirjoitti:
[clip]
> To be clear: I'm going to do this eventually whether or not it
> happens in NumPy because it's an existing problem for heavy
> pandas users. I see no reason why the code can't emit structured
> arrays, too, so we might as well have a common li
23.02.2012 20:44, Francesc Alted kirjoitti:
> On Feb 23, 2012, at 1:33 PM, Neal Becker wrote:
>
>> Is mkl only used for linear algebra? Will it speed up e.g., elementwise
>> transendental functions?
>
> Yes, MKL comes with VML that has this type of optimizations:
And also no, in the sense that
On Feb 23, 2012, at 1:33 PM, Neal Becker wrote:
> Is mkl only used for linear algebra? Will it speed up e.g., elementwise
> transendental functions?
Yes, MKL comes with VML that has this type of optimizations:
http://software.intel.com/sites/products/documentation/hpc/mkl/vml/vmldata.htm
Also
Is mkl only used for linear algebra? Will it speed up e.g., elementwise
transendental functions?
___
NumPy-Discussion mailing list
NumPy-Discussion@scipy.org
http://mail.scipy.org/mailman/listinfo/numpy-discussion
dear all,
I haven't read all 180 e-mails, but I didn't see this on Travis's
initial list.
All of the existing flat file reading solutions I have seen are
not suitable for many applications, and they compare very unfavorably
to tools present in other languages, like R. Here are some of the
main is
On Thu, Feb 23, 2012 at 10:55 AM, Matthew Brett wrote:
> Hi,
>
> On Thu, Feb 23, 2012 at 10:45 AM, Mark Wiebe wrote:
> > On Thu, Feb 23, 2012 at 10:42 AM, Matthew Brett >
> > wrote:
> >>
> >> Hi,
> >>
> >> On Thu, Feb 23, 2012 at 10:11 AM, Pierre Haessig
> >> wrote:
> >> > Le 23/02/2012 17:28,
Hi,
On Thu, Feb 23, 2012 at 10:45 AM, Mark Wiebe wrote:
> On Thu, Feb 23, 2012 at 10:42 AM, Matthew Brett
> wrote:
>>
>> Hi,
>>
>> On Thu, Feb 23, 2012 at 10:11 AM, Pierre Haessig
>> wrote:
>> > Le 23/02/2012 17:28, Charles R Harris a écrit :
>> >> That's correct. They are both extended precisi
On Thu, Feb 23, 2012 at 10:42 AM, Matthew Brett wrote:
> Hi,
>
> On Thu, Feb 23, 2012 at 10:11 AM, Pierre Haessig
> wrote:
> > Le 23/02/2012 17:28, Charles R Harris a écrit :
> >> That's correct. They are both extended precision (80 bits), but
> >> aligned on 32bit/64bit boundaries respectively.
Hi,
On Thu, Feb 23, 2012 at 10:11 AM, Pierre Haessig
wrote:
> Le 23/02/2012 17:28, Charles R Harris a écrit :
>> That's correct. They are both extended precision (80 bits), but
>> aligned on 32bit/64bit boundaries respectively. Sun provides a true
>> quad precision, also called float128, while on
Le 23/02/2012 17:28, Charles R Harris a écrit :
> That's correct. They are both extended precision (80 bits), but
> aligned on 32bit/64bit boundaries respectively. Sun provides a true
> quad precision, also called float128, while on PPC long double is an
> odd combination of two doubles.
This is in
On 02/23/2012 09:47 AM, Dag Sverre Seljebotn wrote:
> On 02/23/2012 05:50 AM, Jaakko Luttinen wrote:
>> Hi!
>>
>> I was wondering whether it would be easy/possible/reasonable to have
>> classes for arrays that have special structure in order to use less
>> memory and speed up some computations?
>>
On 02/23/2012 05:50 AM, Jaakko Luttinen wrote:
> Hi!
>
> I was wondering whether it would be easy/possible/reasonable to have
> classes for arrays that have special structure in order to use less
> memory and speed up some computations?
>
> For instance:
> - symmetric matrix could be stored in almo
On Feb 23, 2012, at 10:26 AM, Matthew Brett wrote:
> Hi,
>
> On Thu, Feb 23, 2012 at 4:23 AM, Francesc Alted wrote:
>> On Feb 23, 2012, at 6:06 AM, Francesc Alted wrote:
>>> On Feb 23, 2012, at 5:43 AM, Nathaniel Smith wrote:
>>>
On Thu, Feb 23, 2012 at 11:40 AM, Francesc Alted
wrot
On Thu, Feb 23, 2012 at 5:23 AM, Francesc Alted wrote:
> On Feb 23, 2012, at 6:06 AM, Francesc Alted wrote:
> > On Feb 23, 2012, at 5:43 AM, Nathaniel Smith wrote:
> >
> >> On Thu, Feb 23, 2012 at 11:40 AM, Francesc Alted
> wrote:
> >>> Exactly. I'd update this to read:
> >>>
> >>> float9696
Hi,
On Thu, Feb 23, 2012 at 4:23 AM, Francesc Alted wrote:
> On Feb 23, 2012, at 6:06 AM, Francesc Alted wrote:
>> On Feb 23, 2012, at 5:43 AM, Nathaniel Smith wrote:
>>
>>> On Thu, Feb 23, 2012 at 11:40 AM, Francesc Alted
>>> wrote:
Exactly. I'd update this to read:
float96
Hi!
I was wondering whether it would be easy/possible/reasonable to have
classes for arrays that have special structure in order to use less
memory and speed up some computations?
For instance:
- symmetric matrix could be stored in almost half the memory required by
a non-symmetric matrix
- diago
2012/2/23 Vincent Schut
> On 02/22/2012 10:45 PM, Chao YUE wrote:
> > Hi all,
> >
> > Is anyone using some python geospatial package that can do jobs like
> > intersection, etc. the job is like you automatically extract a region
> > on a global map etc.
> >
> > thanks and cheers,
> >
> > Chao
>
On Feb 23, 2012, at 6:06 AM, Francesc Alted wrote:
> On Feb 23, 2012, at 5:43 AM, Nathaniel Smith wrote:
>
>> On Thu, Feb 23, 2012 at 11:40 AM, Francesc Alted
>> wrote:
>>> Exactly. I'd update this to read:
>>>
>>> float9696 bits. Only available on 32-bit (i386) platforms.
>>> float128 1
On Feb 23, 2012, at 5:43 AM, Nathaniel Smith wrote:
> On Thu, Feb 23, 2012 at 11:40 AM, Francesc Alted
> wrote:
>> Exactly. I'd update this to read:
>>
>> float9696 bits. Only available on 32-bit (i386) platforms.
>> float128 128 bits. Only available on 64-bit (AMD64) platforms.
>
> E
Le 23/02/2012 12:40, Francesc Alted a écrit :
>> However, I was surprised that float128 is not mentioned in the array of
>> > available types in the user guide.
>> > http://docs.scipy.org/doc/numpy/user/basics.types.html
>> > Is there a specific reason for this absence, or is just about visiting
>>
On Thu, Feb 23, 2012 at 11:40 AM, Francesc Alted wrote:
> Exactly. I'd update this to read:
>
> float96 96 bits. Only available on 32-bit (i386) platforms.
> float128 128 bits. Only available on 64-bit (AMD64) platforms.
Except float96 is actually 80 bits. (Usually?) Plus some padding...
On Feb 23, 2012, at 3:06 AM, Pierre Haessig wrote:
> Hi,
> Le 23/02/2012 02:24, Matthew Brett a écrit :
>> Luckily I was in fact using longdouble in the live code,
> I had never "exotic" floating point precision, so thanks for your post
> which made me take a look at docstring and documentation.
>
On 02/22/2012 10:45 PM, Chao YUE wrote:
> Hi all,
>
> Is anyone using some python geospatial package that can do jobs like
> intersection, etc. the job is like you automatically extract a region
> on a global map etc.
>
> thanks and cheers,
>
> Chao
Chao,
shapely would do this, though I found it
Hi,
Le 23/02/2012 02:24, Matthew Brett a écrit :
> Luckily I was in fact using longdouble in the live code,
I had never "exotic" floating point precision, so thanks for your post
which made me take a look at docstring and documentation.
If I got it right from the docstring, 'np.longdouble', 'np.lo
53 matches
Mail list logo
