On Thursday, November 24, 2005 at 8:06:34 PM UTC+5:30, bon...@gmail.com wrote:
> Bengt Richter wrote:
> > Depends on what you run ;-) Maybe how many times buggy apps die in unusual
> > ways.
> > Or worse, buggy drivers. Your car might have bad 4-wheel drive that you
> > never used,
> > so you
John, thanks for your reply. I will then use the files as input to generate
an index. So the
files are temporary, and provide some attributes in the index. So I do this
multiple times
to gather different attributes, merge, etc.
John Machin [EMAIL PROTECTED] wrote in message
news:[EMAIL
Jack wrote:
I need to process large amount of data. The data structure fits well
in a dictionary but the amount is large - close to or more than the size
of physical memory. I wonder what will happen if I try to load the data
into a dictionary. Will Python use swap memory or will it fail
PROTECTED] wrote in message
news:[EMAIL PROTECTED]
Jack wrote:
I need to process large amount of data. The data structure fits well
in a dictionary but the amount is large - close to or more than the size
of physical memory. I wonder what will happen if I try to load the data
into a dictionary
chunks.
In order to do this, I'll need to have a rule to partition the data into
chunks.
So this is more work in general.
kaens [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
On 5/25/07, Jack [EMAIL PROTECTED] wrote:
I need to process large amount of data. The data structure fits well
If swap memery can not handle this efficiently, I may need to partition
data to multiple servers and use RPC to communicate.
Dennis Lee Bieber [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
On Fri, 25 May 2007 11:11:28 -0700, Jack [EMAIL PROTECTED]
declaimed the following in
In [EMAIL PROTECTED], Jack wrote:
I have tens of millions (could be more) of document in files. Each of them
has other properties in separate files. I need to check if they exist,
update and merge properties, etc.
And this is not a one time job. Because of the quantity of the files, I
think
Jack wrote:
John Nagle [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
Jack wrote:
I need to process large amount of data. The data structure fits well
in a dictionary but the amount is large - close to or more than the size
of physical memory. I wonder what will happen if I try
On May 26, 6:17 pm, Jack [EMAIL PROTECTED] wrote:
I have tens of millions (could be more) of document in files. Each of them
has other
properties in separate files. I need to check if they exist, update and
merge properties, etc.
And then save the results where?
Option (0) retain it in memory
I'll save them in a file for further processing.
John Machin [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
On May 26, 6:17 pm, Jack [EMAIL PROTECTED] wrote:
I have tens of millions (could be more) of document in files. Each of
them
has other
properties in separate files. I need
On May 27, 11:24 am, Jack [EMAIL PROTECTED] wrote:
I'll save them in a file for further processing.
Further processing would be what?
Did you read the remainder of what I wrote?
--
http://mail.python.org/mailman/listinfo/python-list
I need to process large amount of data. The data structure fits well
in a dictionary but the amount is large - close to or more than the size
of physical memory. I wonder what will happen if I try to load the data
into a dictionary. Will Python use swap memory or will it fail?
Thanks.
--
http
In [EMAIL PROTECTED], Jack wrote:
I need to process large amount of data. The data structure fits well
in a dictionary but the amount is large - close to or more than the size
of physical memory. I wonder what will happen if I try to load the data
into a dictionary. Will Python use swap
Thanks for the replies!
Database will be too slow for what I want to do.
Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
In [EMAIL PROTECTED], Jack wrote:
I need to process large amount of data. The data structure fits well
in a dictionary but the amount
On May 25, 10:50 am, Jack [EMAIL PROTECTED] wrote:
I need to process large amount of data. The data structure fits well
in a dictionary but the amount is large - close to or more than the size
of physical memory. I wonder what will happen if I try to load the data
into a dictionary
Jack wrote:
Thanks for the replies!
Database will be too slow for what I want to do.
Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
In [EMAIL PROTECTED], Jack wrote:
I need to process large amount of data. The data structure fits well
On 5/25/07, Jack [EMAIL PROTECTED] wrote:
I need to process large amount of data. The data structure fits well
in a dictionary but the amount is large - close to or more than the size
of physical memory. I wonder what will happen if I try to load the data
into a dictionary. Will Python use
Larry Bates wrote:
Jack wrote:
Thanks for the replies!
Database will be too slow for what I want to do.
Marc 'BlackJack' Rintsch [EMAIL PROTECTED] wrote in message
news:[EMAIL PROTECTED]
In [EMAIL PROTECTED], Jack wrote:
I need to process large amount of data. The data structure fits
On Thu, 24 Nov 2005 05:22:18 GMT, Dennis Lee Bieber
[EMAIL PROTECTED] wrote:
On Wed, 23 Nov 2005 16:51:15 GMT, Manlio Perillo
[EMAIL PROTECTED] declaimed the following in
comp.lang.python:
So, it's seem to be a specific problem of Windows XP(?).
Pardon? I think the prior respondent said
Manlio Perillo wrote:
I have added a question mark...
However: did you have installed SP1 or SP2?
running XP without service packs is a really bad idea. you're not
telling me that you haven't updated, are you?
(the machines I tested on are all fully up to date)
/F
--
On Thu, 24 Nov 2005 10:24:02 +0100, Fredrik Lundh
[EMAIL PROTECTED] wrote:
Manlio Perillo wrote:
I have added a question mark...
However: did you have installed SP1 or SP2?
running XP without service packs is a really bad idea. you're not
telling me that you haven't updated, are you?
I
On Wed, 23 Nov 2005 18:18:34 +0100, Fredrik Lundh [EMAIL PROTECTED] wrote:
Manlio Perillo wrote:
print data
Traceback (most recent call last):
File xxx, line xxx, in ?
print data
IOError: [Errno 12] Not enough space
errno 12 is ENOMEM (that is, the system did not have enough
Bengt Richter wrote:
If windows has been running a long time (a few days or a week may be long ;-)
it
may get fragmented in some smallish memory arena reserved for special things
(I forgot
what versions, but I wouldn't be surprised if something still had s
specialized limit).
I know it
On 24 Nov 2005 03:22:26 -0800, [EMAIL PROTECTED] [EMAIL PROTECTED] wrote:
Bengt Richter wrote:
If windows has been running a long time (a few days or a week may be long
;-) it
may get fragmented in some smallish memory arena reserved for special things
(I forgot
what versions, but I
Bengt Richter wrote:
Depends on what you run ;-) Maybe how many times buggy apps die in unusual
ways.
Or worse, buggy drivers. Your car might have bad 4-wheel drive that you never
used,
so you thought your car reliable ;-)
Sure, no casual toys are allowed. But this is true in general,
Regards.
On my system:
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)]
on win32, Windows XP
I have this problem:
n = 61409 + 1
data = 'x' * n
print data
Traceback (most recent call last):
File xxx, line xxx, in ?
print data
IOError: [Errno 12] Not enough space
Manlio Perillo wrote:
Regards.
On my system:
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)]
on win32, Windows XP
I have this problem:
n = 61409 + 1
data = 'x' * n
print data
Traceback (most recent call last):
File xxx, line xxx, in ?
print data
That code works here.
Python2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
It's Windows XP, Pentium 4, unknown amount of RAM. I'm running python.exe in a
console window. It also worked in IDLE.
Jeff
pgptwrbVpG8CR.pgp
Description: PGP signature
--
Manlio Perillo wrote:
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)]
on win32, Windows XP
I have this problem:
n = 61409 + 1
data = 'x' * n
print data
Traceback (most recent call last):
File xxx, line xxx, in ?
print data
IOError: [Errno 12] Not enough
On Wed, 23 Nov 2005 14:59:45 +0100, Fredrik Lundh
[EMAIL PROTECTED] wrote:
Manlio Perillo wrote:
Python 2.4.1 (#65, Mar 30 2005, 09:13:57) [MSC v.1310 32 bit (Intel)]
on win32, Windows XP
I have this problem:
n = 61409 + 1
data = 'x' * n
print data
Traceback (most recent call last):
On Wed, 23 Nov 2005 07:48:30 -0600, [EMAIL PROTECTED] wrote:
That code works here.
Python2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
It's Windows XP, Pentium 4, unknown amount of RAM. I'm running python.exe in a
console window. It also worked in IDLE.
Jeff
Thanks.
Manlio Perillo wrote:
print data
Traceback (most recent call last):
File xxx, line xxx, in ?
print data
IOError: [Errno 12] Not enough space
errno 12 is ENOMEM (that is, the system did not have enough memory
to finish an operation).
However I think the error message is not a
On 2005-11-23, Manlio Perillo [EMAIL PROTECTED] wrote:
On Wed, 23 Nov 2005 07:48:30 -0600, [EMAIL PROTECTED] wrote:
That code works here.
Python2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
It's Windows XP, Pentium 4, unknown amount of RAM. I'm running python.exe in
a
Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
win32
No problems on Windows 2000 Pro with 512MB of RAM
--
http://mail.python.org/mailman/listinfo/python-list
Manlio Perillo wrote:
So, it's seem to be a specific problem of Windows XP(?).
Nobody can test it on a Windows 98/2000 machine?
works fine on assorted XP and 2000 boxes for me.
is the n = 61409 + 1 stuff necessary, or do you get the same
problem for all outputs over a given size ? even if
On Wed, 23 Nov 2005 18:21:45 +0100, Fredrik Lundh
[EMAIL PROTECTED] wrote:
Manlio Perillo wrote:
So, it's seem to be a specific problem of Windows XP(?).
Nobody can test it on a Windows 98/2000 machine?
works fine on assorted XP and 2000 boxes for me.
That's very strange.
is the n = 61409
36 matches
Mail list logo
