Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Chris Angelico
; interact with systems B and C and so on, and each machine knows what kind it > is, and especially if it knows which kind of machine left it something in > shared memory, there has to be a way to coordinate between them. If character > strings in ASCII or UTF8 or EBCDIC or some kind of

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Avi Gross via Python-list
something in shared memory, there has to be a way to coordinate between them. If character strings in ASCII or UTF8 or EBCDIC or some kind of raw are allowed, and used with just a few characters that can spell out numbers, I would think much is possible. If needed, perhaps a fixed size could be set

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Barry
> On 2 Feb 2022, at 18:19, Jen Kris via Python-list > wrote: > > It's not clear to me from the struct module whether it can actually > auto-detect endianness. It is impossible to auto detect endian in the general case. > I think it must be specified, just as I had to do with int.from_byte

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Dennis Lee Bieber
On Wed, 2 Feb 2022 19:16:19 +0100 (CET), Jen Kris declaimed the following: >It's not clear to me from the struct module whether it can actually >auto-detect endianness.  I think it must be specified, just as I had to do >with int.from_bytes().  In my case endianness was dictated by how the four

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Jen Kris via Python-list
y is also not. > > -Original Message- > From: Dennis Lee Bieber > To: python-list@python.org > Sent: Wed, Feb 2, 2022 12:30 am > Subject: Re: Data unchanged when passing data to Python in multiprocessing > shared memory > > > On Wed, 2 Feb 2022 00:40:22 +0100

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Jen Kris via Python-list
It's not clear to me from the struct module whether it can actually auto-detect endianness.  I think it must be specified, just as I had to do with int.from_bytes().  In my case endianness was dictated by how the four bytes were populated, starting with the zero bytes on the left.  Feb 1, 202

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Avi Gross via Python-list
m: Dennis Lee Bieber To: python-list@python.org Sent: Wed, Feb 2, 2022 12:30 am Subject: Re: Data unchanged when passing data to Python in multiprocessing shared memory On Wed, 2 Feb 2022 00:40:22 +0100 (CET), Jen Kris declaimed the following: > > breakup = int.from_bytes(byte_va

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Dennis Lee Bieber
On Wed, 2 Feb 2022 00:40:22 +0100 (CET), Jen Kris declaimed the following: > > breakup = int.from_bytes(byte_val, "big") >print("this is breakup " + str(breakup)) > >Python prints:  this is breakup 32894 > >Note that I had to switch from little endian to big endian.  Python is little >endian by

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-02 Thread Barry Scott
let me (and the > list) know. By using the struct module you can control for endian of the data. Barry > > Thanks. > > > Feb 1, 2022, 14:20 by ba...@barrys-emacs.org: > > On 1 Feb 2022, at 20:26, Jen Kris via Python-list > wrote: > > I am using multipro

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-01 Thread Jen Kris via Python-list
on-list >> wrote: >> >> I am using multiprocesssing.shared_memory to pass data between NASM and >> Python. The shared memory is created in NASM before Python is called. >> Python connects to the shm: shm_00 = >> shared_memory.SharedMemory(name='shm_obje

Re: Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-01 Thread Barry
> On 1 Feb 2022, at 20:26, Jen Kris via Python-list > wrote: > > I am using multiprocesssing.shared_memory to pass data between NASM and > Python. The shared memory is created in NASM before Python is called. > Python connects to the shm: shm_00 = > shared_mem

Data unchanged when passing data to Python in multiprocessing shared memory

2022-02-01 Thread Jen Kris via Python-list
I am using multiprocesssing.shared_memory to pass data between NASM and Python.  The shared memory is created in NASM before Python is called.  Python connects to the shm:  shm_00 = shared_memory.SharedMemory(name='shm_object_00',create=False).  I have used shared memory at other

Re: Help needed to create a Python extension library for an existing shared memory hash table library

2014-03-23 Thread Rustom Mody
On Sunday, March 23, 2014 6:37:11 PM UTC+5:30, Jens Thoms Toerring wrote: > Simon Hardy-Francis wrote: > > Hi Python fans, I just released my first open source project ever called > > SharedHashFile [1]. It's a shared memory hash table written in C. Some guy > > on Quora

Re: Help needed to create a Python extension library for an existing shared memory hash table library

2014-03-23 Thread Jens Thoms Toerring
Simon Hardy-Francis wrote: > Hi Python fans, I just released my first open source project ever called > SharedHashFile [1]. It's a shared memory hash table written in C. Some guy > on Quora asked [2] whether there's an extension library for Python coming > out. I would lik

Help needed to create a Python extension library for an existing shared memory hash table library

2014-03-22 Thread Simon Hardy-Francis
Hi Python fans, I just released my first open source project ever called SharedHashFile [1]. It's a shared memory hash table written in C. Some guy on Quora asked [2] whether there's an extension library for Python coming out. I would like to do one but I know little about Pyt

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-11 Thread sturlamolden
On 11 apr, 21:11, sturlamolden wrote: > import numpy as np > import sharedmem as sm > private_array = np.zeros((10,10)) > shared_array  = sm.zeros((10,10)) I am also using this to implement synchronization primitives (barrier, lock, semaphore, event) that can be sent over an instance of multipro

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-11 Thread sturlamolden
ively use multi-core CPUs. Then why not let NumPy use shared memory in this case? It is as simple as this: import numpy as np import sharedmem as sm private_array = np.zeros((10,10)) shared_array = sm.zeros((10,10)) I.e. the code is identical (more or less). Also, do you think shared memo

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-11 Thread John Nagle
On 4/10/2011 3:29 PM, sturlamolden wrote: On 10 apr, 18:27, John Nagle wrote: Unless you have a performance problem, don't bother with shared memory. If you have a performance problem, Python is probably the wrong tool for the job anyway. Then why does Python h

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-10 Thread sturlamolden
On 8 apr, 03:10, sturlamolden wrote: > That was easy, 64-bit support for Windows is done :-) > > Now I'll just have to fix the Linux code, and figure out what to do > with os._exit preventing clean-up on exit... :-( Now it feel dumb, it's not worse than monkey patching os._exit, which I should h

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-10 Thread sturlamolden
On 10 apr, 18:27, John Nagle wrote: >     Unless you have a performance problem, don't bother with shared > memory. > >     If you have a performance problem, Python is probably the wrong > tool for the job anyway. Then why does Python have a multiprocessing module? In m

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-10 Thread John Nagle
ork" (if you're on a *nix machine). See http://pythonwise.blogspot.com/2009/04/pmap.html for example ;) Unless you have a performance problem, don't bother with shared memory. If you have a performance problem, Python is probably the wrong tool for the job anyway.

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-10 Thread Miki Tebeka
> Now, I don't know that I actually HAVE to pass my neural network and > input data as copies -- they're both READ-ONLY objects for the > duration of an evaluate function (which can go on for quite a while). One option in that case is to use "fork" (if you're on a *nix machine). See http://pythonwi

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-10 Thread sturlamolden
On 9 apr, 22:18, John Ladasky wrote: > So, there are limited advantages to trying to parallelize the > evaluation of ONE cascade network's weights against ONE input vector. > However, evaluating several copies of one cascade network's output, > against several different test inputs simultaneously

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-09 Thread John Ladasky
have not updated the sharedmem arrays for two > years is that I have come to the conclusion that there are better ways > to do this (paricularly vendor tuned libraries). But since they are > mostly useful with 64-bit (i.e. large arrays), I'll post an update > soon. > > If you

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-09 Thread sturlamolden
al in terms of > time and memory usage. The expensive overhead in passing a NymPy array to multiprocessing.Queue is related to pickle/cPickle, not IPC or making a copy of the buffer. For any NumPy array you can afford to copy in terms of memory, just work with copies. The shared memory arrays I

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-09 Thread John Ladasky
On Apr 7, 6:10 pm, sturlamolden wrote: > On 8 apr, 02:38, sturlamolden wrote: > > > I should probably fix it for 64-bit now. Just recompiliong with 64-bit > > integers will not work, because I intentionally hardcoded the higher > > 32 bits to 0. > > That was easy, 64-bit support for Windows is do

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread sturlamolden
On 8 apr, 02:38, sturlamolden wrote: > I should probably fix it for 64-bit now. Just recompiliong with 64-bit > integers will not work, because I intentionally hardcoded the higher > 32 bits to 0. That was easy, 64-bit support for Windows is done :-) Now I'll just have to fix the Linux code, an

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread sturlamolden
On 8 apr, 02:03, sturlamolden wrote: > http://folk.uio.no/sturlamo/python/sharedmem-feb13-2009.zip > Known issues/bugs: 64-bit support is lacking, and os._exit in > multiprocessing causes a memory leak on Linux. I should probably fix it for 64-bit now. Just recompiliong with 64-bit integers will

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread sturlamolden
On 4 apr, 22:20, John Ladasky wrote: > https://bitbucket.org/cleemesser/numpy-sharedmem/src/3fa526d11578/shm... > > I've added a few lines to this code which allows subclassing the > shared memory array, which I need (because my neural net objects are > more than just

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread sturlamolden
; allocated before the Pool is started. And this only works on UNIX machines. > The > shared memory objects that shmarray uses can only be inherited. I believe > that's > what Sturla was getting at. It's a C extension that gives a buffer to NumPy. Then YOU changed how NumPy p

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread Robert Kern
On 4/7/11 1:39 PM, John Ladasky wrote: On Apr 7, 10:44 am, Robert Kern wrote: On 4/7/11 1:40 AM, John Ladasky wrote: On Apr 5, 10:43 am, Philip Semanchukwrote: And as Robert Kern pointed out, numpy arrays are also pickle-able. OK, but SUBCLASSES of numpy.ndarray are not, in my hands,

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread John Ladasky
On Apr 7, 10:44 am, Robert Kern wrote: > On 4/7/11 1:40 AM, John Ladasky wrote: > > > On Apr 5, 10:43 am, Philip Semanchuk  wrote: > >> And as Robert Kern pointed out, numpy arrays are also pickle-able. > > > OK, but SUBCLASSES of numpy.ndarray are not, in my hands, pickling as > > I would expect.

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread Robert Kern
On 4/7/11 1:40 AM, John Ladasky wrote: On Apr 5, 10:43 am, Philip Semanchuk wrote: And as Robert Kern pointed out, numpy arrays are also pickle-able. OK, but SUBCLASSES of numpy.ndarray are not, in my hands, pickling as I would expect. I already have lots of code that is based on such sub

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread Philip Semanchuk
On Apr 7, 2011, at 3:41 AM, John Ladasky wrote: > Following up to my own post... > > On Apr 6, 11:40 pm, John Ladasky wrote: > >> What's up with that? > > Apparently, "what's up" is that I will need to implement a third > method in my ndarray subclass -- namely, __reduce__. > > http://www.ma

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-07 Thread John Ladasky
Following up to my own post... On Apr 6, 11:40 pm, John Ladasky wrote: > What's up with that? Apparently, "what's up" is that I will need to implement a third method in my ndarray subclass -- namely, __reduce__. http://www.mail-archive.com/numpy-discussion@scipy.org/msg02446.html I'm burned o

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-06 Thread John Ladasky
Hello again, Philip, I really appreciate you sticking with me. Hopefully this will help someone else, too. I've done some more reading, and will offer some minimal code below. I've known about this page for a while, and it describes some of the unconventional things one needs to consider when s

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-05 Thread Philip Semanchuk
On Apr 5, 2011, at 12:58 PM, John Ladasky wrote: > Hi Philip, > > Thanks for the reply. > > On Apr 4, 4:34 pm, Philip Semanchuk wrote: >> So if you're going to use multiprocessing, you're going to use pickle, and >> you >> need pickleable objects. > > OK, that's good to know. But as Dan Str

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-05 Thread John Ladasky
Hi Philip, Thanks for the reply. On Apr 4, 4:34 pm, Philip Semanchuk wrote: > So if you're going to use multiprocessing, you're going to use pickle, and you > need pickleable objects. OK, that's good to know. > > Pickling is still a pretty > > vague progress to me, but I can see that you have

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-05 Thread Robert Kern
e sure that the shared arrays are allocated before the Pool is started. And this only works on UNIX machines. The shared memory objects that shmarray uses can only be inherited. I believe that's what Sturla was getting at. I'll take that back a little bit. Since the underlying shared memory

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-04 Thread Philip Semanchuk
On Apr 4, 2011, at 9:03 PM, Dan Stromberg wrote: > On Mon, Apr 4, 2011 at 4:34 PM, Philip Semanchuk wrote: > >> So if you're going to use multiprocessing, you're going to use pickle, and >> you need pickleable objects. >> > > http://docs.python.org/library/multiprocessing.html#sharing-state-be

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-04 Thread Dan Stromberg
On Mon, Apr 4, 2011 at 4:34 PM, Philip Semanchuk wrote: > So if you're going to use multiprocessing, you're going to use pickle, and > you need pickleable objects. > http://docs.python.org/library/multiprocessing.html#sharing-state-between-processes -- http://mail.python.org/mailman/listinfo/pyt

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-04 Thread Robert Kern
e duration of an evaluate function (which can go on for quite a while). So, I have also started to investigate shared-memory approaches. I don't know how a shared-memory object is referenced by a subprocess yet, but presumably you pass a reference to the object, rather than the whole object. Also

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-04 Thread Philip Semanchuk
> input data as copies -- they're both READ-ONLY objects for the > duration of an evaluate function (which can go on for quite a while). > So, I have also started to investigate shared-memory approaches. I > don't know how a shared-memory object is referenced by a subproces

Re: Multiprocessing, shared memory vs. pickled copies

2011-04-04 Thread Dan Stromberg
On Mon, Apr 4, 2011 at 1:20 PM, John Ladasky wrote: > When should one pickle and copy? When to implement an object in > shared memory? Why is pickling apparently such a non-trivial process > anyway? And, given that multi-core CPU's are apparently here to stay, > should it b

Multiprocessing, shared memory vs. pickled copies

2011-04-04 Thread John Ladasky
ich can go on for quite a while). So, I have also started to investigate shared-memory approaches. I don't know how a shared-memory object is referenced by a subprocess yet, but presumably you pass a reference to the object, rather than the whole object. Also, it appears that subprocesses also a

Re: Shared memory python between two separate shell-launched processes

2011-02-12 Thread Adam Skutt
On Feb 12, 3:29 am, John Nagle wrote: > >     If you're having trouble debugging a sequential program, > you do not want to add shared memory to the problem. No, you don't want to add additional concurrent threads of execution. But he's not doing that, he's just pre

Re: Shared memory python between two separate shell-launched processes

2011-02-12 Thread John Nagle
nch of unit tests for the little bits but something is unhappy when I put them all together...) If you're having trouble debugging a sequential program, you do not want to add shared memory to the problem. John Nagle -- http://mail.python.org/mailman/listinfo/python-list

Re: Shared memory python between two separate shell-launched processes

2011-02-11 Thread Miki Tebeka
it can > start up quickly and read from the shared memory instead of loading > the cache itself. You can fork and set "hard" breakpoint using "import pdb; pdb.set_trace()". Not sure how Emacs will handle that but you can use pdb from the terminal. For easy forking see http:

Re: Shared memory python between two separate shell-launched processes

2011-02-11 Thread Philip
On Feb 11, 6:27 am, Adam Skutt wrote: > On Feb 10, 9:30 am, "Charles Fox (Sheffield)" > wrote: > > > But when I look at posix_ipc and POSH it looks like you have to fork > > the second process from the first one, rather than access the shared > > memory

Re: Shared memory python between two separate shell-launched processes

2011-02-11 Thread Jean-Paul Calderone
> > > > > > So speed up the compile-test cycle I'm thinking about running a > > > > > completely separate process (not a fork, but a processed launched form > > > > > a different terminal) > > > > > Why _not_ fork?  Load up your

Re: Shared memory python between two separate shell-launched processes

2011-02-11 Thread Adam Skutt
On Feb 10, 9:30 am, "Charles Fox (Sheffield)" wrote: > > But when I look at posix_ipc and POSH it looks like you have to fork > the second process from the first one, rather than access the shared > memory though a key ID as in standard C unix shared memory.  Am I > missi

Re: Shared memory python between two separate shell-launched processes

2011-02-11 Thread Charles Fox (Sheffield)
ed launched form > > > > a different terminal) > > > > Why _not_ fork?  Load up your data, then go into a loop forking and > > > loading/ > > > running the rest of your code in the child.  This should be really > > > easy to > > >

Re: Shared memory python between two separate shell-launched processes

2011-02-10 Thread Jean-Paul Calderone
your data, then go into a loop forking and > > loading/ > > running the rest of your code in the child.  This should be really > > easy to > > implement compared to doing something with shared memory, and solves > > the > > problem you're trying to solv

Re: Shared memory python between two separate shell-launched processes

2011-02-10 Thread Charles Fox (Sheffield)
rate process (not a fork, but a processed launched form > > a different terminal) > > Why _not_ fork?  Load up your data, then go into a loop forking and > loading/ > running the rest of your code in the child.  This should be really > easy to > implement compared to doing some

Re: Shared memory python between two separate shell-launched processes

2011-02-10 Thread Jean-Paul Calderone
Load up your data, then go into a loop forking and loading/ running the rest of your code in the child. This should be really easy to implement compared to doing something with shared memory, and solves the problem you're trying to solve of long startup time just as well. It also protects you

Shared memory python between two separate shell-launched processes

2011-02-10 Thread Charles Fox (Sheffield)
y and read from the shared memory instead of loading the cache itself. But when I look at posix_ipc and POSH it looks like you have to fork the second process from the first one, rather than access the shared memory though a key ID as in standard C unix shared memory. Am I missing something? Are

Which multiprocessing methods use shared memory?

2010-07-28 Thread Kevin Ar18
The multiprocessing module has 4 methods for sharing data between processes: Queues Pipes Shared Memory Map Server Process Which of these use shared memory? I understand that the 3rd (Shared Memory Map) does, but what about Queues? Thanks, Kevin

Re: Which multiprocessing methods use shared memory?

2010-07-27 Thread John Nagle
On 7/27/2010 12:30 PM, MRAB wrote: Kevin Ar18 wrote: I'm not sure my previous message went through (I wasn't subscribe), so I'm gonna try again. The multiprocessing module has 4 methods for sharing data between processes: Queues Pipes Shared Memory Map Server Process Which of t

Re: Which multiprocessing methods use shared memory?

2010-07-27 Thread MRAB
Kevin Ar18 wrote: I'm not sure my previous message went through (I wasn't subscribe), so I'm gonna try again. The multiprocessing module has 4 methods for sharing data between processes: Queues Pipes Shared Memory Map Server Process Which of these use shared memory? I unde

Which multiprocessing methods use shared memory?

2010-07-27 Thread Kevin Ar18
I'm not sure my previous message went through (I wasn't subscribe), so I'm gonna try again. The multiprocessing module has 4 methods for sharing data between processes: Queues Pipes Shared Memory Map Server Process Which of these use shared memory? I understand that the 3rd

Re: ob_type in shared memory

2009-01-26 Thread Aaron Brady
On Jan 26, 9:20 pm, Aaron Brady wrote: > Hi, Mark, snip > On Jan 25, 9:47 pm, Mark Wooding wrote:> Aaron Brady > writes: > > > I am writing an extension using shared memory.  I need a data type > > > that is able to reassign its 'ob_type' field depen

Re: ob_type in shared memory

2009-01-26 Thread Aaron Brady
Hi, Mark, Do you mind if I approach you off the group about this? Aaron On Jan 25, 9:47 pm, Mark Wooding wrote: > Aaron Brady writes: > > I am writing an extension using shared memory.  I need a data type > > that is able to reassign its 'ob_type' field dependi

Re: ob_type in shared memory

2009-01-26 Thread Aaron Brady
'P' sees it.  When > > process 'Q' is looking at it, it needs to have an 'ob_type' that is > > 'Ta' as process 'Q' sees it.  If it referred to 'Ta' in process 'P' > > when 'Q' was calling it, 'Q&#x

Re: ob_type in shared memory

2009-01-25 Thread Mark Wooding
Aaron Brady writes: > I am writing an extension using shared memory. I need a data type > that is able to reassign its 'ob_type' field depending on what process > is calling it. That sounds scary! > Object 'A' is of type 'Ta'. When process 'P&#x

ob_type in shared memory

2009-01-25 Thread Aaron Brady
Hello, I am writing an extension using shared memory. I need a data type that is able to reassign its 'ob_type' field depending on what process is calling it. Object 'A' is of type 'Ta'. When process 'P' is looking at it, it needs to have an 'ob_typ

Re: Regarding shared memory

2008-10-30 Thread Aaron Brady
ms using socket programming,but i > am not able to put the data in shared memory and then access it. > > NOTE:I want to put the data in main memory only once(on the server > using server program) i.e. whenever client connects to the server it > should only access the data and not cr

Re: Regarding shared memory

2008-10-30 Thread James Mills
d server programs using socket programming,but i > am not able to put the data in shared memory and then access it. > > NOTE:I want to put the data in main memory only once(on the server > using server program) i.e. whenever client connects to the server it > should only access the

Regarding shared memory

2008-10-29 Thread gaurav kashyap
memory.Here the server calls a module that processes the data as per the client request and the returns some information to the client. I can create client and server programs using socket programming,but i am not able to put the data in shared memory and then access it. NOTE:I want to put the

Re: Using Python to shared memory resources between Linux and Windows

2008-08-27 Thread Diez B. Roggisch
Blubaugh, David A. schrieb: Diez, What you have said is extremely concerning. I am now using VMware. With Linux as the Master and windows as the guest operating system. I was wondering if you have ever had to develop a share memory resource between Linux and windows within a Vmware setup?

Re: FW: Using Python to shared memory resources between Linux and Windows

2008-08-26 Thread Emile van Sebille
Blubaugh, David A. wrote: Diez, What you have said is extremely concerning. I am now using VMware. With Linux as the Master and windows as the guest operating system. I was wondering if you have ever had to develop a share memory resource between Linux and windows within a Vmware setup?

FW: Using Python to shared memory resources between Linux and Windows

2008-08-26 Thread Blubaugh, David A.
will not be forgotten, David Blubaugh -Original Message- From: Diez B. Roggisch [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 26, 2008 4:54 PM To: python-list@python.org Subject: Re: Using Python to shared memory resources between Linux and Windows Blubaugh, David A. schrieb: >

RE: Using Python to shared memory resources between Linux and Windows

2008-08-26 Thread Blubaugh, David A.
will not be forgotten, David Blubaugh -Original Message- From: Diez B. Roggisch [mailto:[EMAIL PROTECTED] Sent: Tuesday, August 26, 2008 4:54 PM To: python-list@python.org Subject: Re: Using Python to shared memory resources between Linux and Windows Blubaugh, David A. schrieb: >

Re: Using Python to shared memory resources between Linux and Windows

2008-08-26 Thread Diez B. Roggisch
Blubaugh, David A. schrieb: To All, To whom else if I may ask? I was wondering if it was possible to utilize python to share a memory resource between a linux and windows system?? It should be stated that both the Linux (CENTOS 5) and windows are physically located on the same computer. Is

Using Python to shared memory resources between Linux and Windows

2008-08-26 Thread Blubaugh, David A.
To All, I was wondering if it was possible to utilize python to share a memory resource between a linux and windows system?? It should be stated that both the Linux (CENTOS 5) and windows are physically located on the same computer. Is any of this possible? Thanks, David Blubaugh

Re: mmap and shared memory

2008-02-15 Thread Ryan Smith-Roberts
On Feb 11, 3:41 pm, Matias Surdi <[EMAIL PROTECTED]> wrote: > Suppose I've a process P1, which generates itself a lot of data , for > example 2Mb. > Then, I've a process P2 which must access P1 shared memory and, > probably, modify this data. > To accomplish this, I&

Re: mmap and shared memory

2008-02-14 Thread Nikita the Spider
address in the process's > >>> address space at which the object you specify is to be mapped. > >>> > >>> As far as I know, the only way two unrelated processes can share > >>> memory via mmap is by mapping a file. An anonymous block is known >

Re: mmap and shared memory

2008-02-13 Thread Jeff Schwab
as I know, the only way two unrelated processes can share >>> memory via mmap is by mapping a file. An anonymous block is known >>> only to the process that creates it -- being anonymous, there's >>> no way for another process to refer to it. >> On POSIX sys

Re: mmap and shared memory

2008-02-13 Thread Nikita the Spider
hare > > memory via mmap is by mapping a file. An anonymous block is known > > only to the process that creates it -- being anonymous, there's > > no way for another process to refer to it. > > On POSIX systems, you can create a shared memory object without a file > u

Re: mmap and shared memory

2008-02-12 Thread Jeff Schwab
, there's > no way for another process to refer to it. On POSIX systems, you can create a shared memory object without a file using shm_open. The function returns a file descriptor. > However, if one process is forked from the other, the parent > can mmap an anonymous block and t

Re: mmap and shared memory

2008-02-12 Thread greg
Carl Banks wrote: > In C you can use the mmap call to request a specific physical location > in memory (whence I presume two different processes can mmap anonymous > memory block in the same location) Um, no, it lets you specify the *virtual* address in the process's address space at which the obj

Re: mmap and shared memory

2008-02-11 Thread Tim Roberts
Matias Surdi <[EMAIL PROTECTED]> wrote: >Suppose I've a process P1, which generates itself a lot of data , for >example 2Mb. >Then, I've a process P2 which must access P1 shared memory and, >probably, modify this data. >To accomplish this, I've been diggin

Re: mmap and shared memory

2008-02-11 Thread Carl Banks
On Feb 11, 6:41 pm, Matias Surdi <[EMAIL PROTECTED]> wrote: > Suppose I've a process P1, which generates itself a lot of data , for > example 2Mb. > Then, I've a process P2 which must access P1 shared memory and, > probably, modify this data. > To accomplish this, I&

mmap and shared memory

2008-02-11 Thread Matias Surdi
Suppose I've a process P1, which generates itself a lot of data , for example 2Mb. Then, I've a process P2 which must access P1 shared memory and, probably, modify this data. To accomplish this, I've been digging around python's mmap module, but I can't figure ho

Re: shared memory pointer

2007-09-11 Thread Tim Golden
thon charting ...] OK, so your C++ stuff dumps to memory. Using CreateFileMapping and MapViewOfFile? Or some other kind of global memory? In short, if your C++ can write stuff out to shared memory created using CreateFileMapping/MapViewOfFile with a first param of INVALID_HANDLE_VALUE and a tag

Re: shared memory pointer

2007-09-11 Thread Tim
block in shared memory so I don't have to wait for the data to be written to a disk. I have about 200 shared memory segments that the simulation creates. I did not know that my code would write to the hard drive because I use ctypes. Could you explain that. Even if I use mmap, how could I mma

Re: shared memory pointer

2007-09-11 Thread Tim Golden
Tim wrote: > I saw the mmap function in the shared memory example. I had some > concern with my large memory size being written to the disk drive. I > though it might slow down my application. The reason I am writting > this new code is because the existing method using a file. I thou

Re: shared memory pointer

2007-09-10 Thread Tim Golden
Tim wrote: > I reviewed the mmap function and I have a question. In the example > code below, what is the connection between the data in shared memory > and the mmap function. The fileno is zero. Why is it zero? The size > makes sense because there is 256 bytes in shared memory

Re: shared memory pointer

2007-09-10 Thread MC
Hi! I agree ; on windows mmap use Memory-Mapped-file, who use virtual memory. And shared memory use physical memory. The difference is OS an not Python -- @-salutations Michel Claveau -- http://mail.python.org/mailman/listinfo/python-list

Re: shared memory pointer

2007-09-10 Thread Tim
On Sep 10, 10:11 am, Tim Golden <[EMAIL PROTECTED]> wrote: > Tim wrote: > > Hello Everyone, > > > I am getting shared memory in python using the following. > > > szName = c_char_p(name) > > hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_VALUE,

Re: shared memory pointer

2007-09-10 Thread Tim
On Sep 10, 10:11 am, Tim Golden <[EMAIL PROTECTED]> wrote: > Tim wrote: > > Hello Everyone, > > > I am getting shared memory in python using the following. > > > szName = c_char_p(name) > > hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_VALUE,

Re: shared memory pointer

2007-09-10 Thread Tim Golden
Tim wrote: > Hello Everyone, > > I am getting shared memory in python using the following. > > szName = c_char_p(name) > hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_VALUE, > None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName)

shared memory pointer

2007-09-10 Thread Tim
Hello Everyone, I am getting shared memory in python using the following. szName = c_char_p(name) hMapObject = windll.kernel32.CreateFileMappingA(INVALID_HANDLE_VALUE, None, PAGE_READONLY, 0, TABLE_SHMEMSIZE, szName) if (hMapObject == 0): print "OpenKey: Could not

Re: Shared Memory Space - Accross Apps & Network

2007-05-25 Thread Irmen de Jong
Hendrik van Rooyen wrote: >>> Just to get the ball rolling, I'd suggest two things: >>> >>> Pyro -http://pyro.sf.net > > This is good advice, if you have the power to run it. What do you mean exactly by "the power to run it"? --Irmen -- http://mail.python.org/mailman/listinfo/python-list

Re: Shared Memory Space - Accross Apps & Network

2007-05-25 Thread Hendrik van Rooyen
From: "D.Hering" wrote: > On May 23, 4:04 am, Tim Golden <[EMAIL PROTECTED]> wrote: > > Robert Rawlins - Think Blue wrote: > > I did not see the original post either :-( ... > > > I've got an application that runs on an embedded system, the application > > > uses a whole bunch or dicts and othe

Re: Shared Memory Space - Accross Apps & Network

2007-05-24 Thread Cameron Laird
In article <[EMAIL PROTECTED]>, Tim Golden <[EMAIL PROTECTED]> wrote: . . . >PyLinda - http://www-users.cs.york.ac.uk/~aw/pylinda/ > >This implements the tuplespace paradigm. It's great >fun to use, but as far as I know this i

Re: Shared Memory Space - Accross Apps & Network

2007-05-23 Thread Paul Boddie
On 23 May, 11:48, "D.Hering" <[EMAIL PROTECTED]> wrote: > > Possibly, IPython's new interactive parallel environment is what you > are looking for:http://ipython.scipy.org/moin/Parallel_Computing See this and related projects on the python.org Wiki: http://wiki.python.org/moin/ParallelProcessing

Re: Shared Memory Space - Accross Apps & Network

2007-05-23 Thread Paul Boddie
On 23 May, 11:48, "D.Hering" <[EMAIL PROTECTED]> wrote: > > Possibly, IPython's new interactive parallel environment is what you > are looking for:http://ipython.scipy.org/moin/Parallel_Computing See this and related projects on the python.org Wiki: http://wiki.python.org/moin/ParallelProcessing

Re: Shared Memory Space - Accross Apps & Network

2007-05-23 Thread D.Hering
On May 23, 4:04 am, Tim Golden <[EMAIL PROTECTED]> wrote: > Robert Rawlins - Think Blue wrote: > > > I've got an application that runs on an embedded system, the application > > uses a whole bunch or dicts and other data types to store state and other > > important information. > > I'm looking to b

Re: Shared Memory Space - Accross Apps & Network

2007-05-23 Thread D.Hering
On May 23, 4:04 am, Tim Golden <[EMAIL PROTECTED]> wrote: > Robert Rawlins - Think Blue wrote: > > > I've got an application that runs on an embedded system, the application > > uses a whole bunch or dicts and other data types to store state and other > > important information. > > I'm looking to b

Re: Shared Memory Space - Accross Apps & Network

2007-05-23 Thread Gabriel Genellina
En Wed, 23 May 2007 05:04:10 -0300, Tim Golden <[EMAIL PROTECTED]> escribió: > I noticed that the post hadn't appeared on Google > Groups, at least. I read things via the mailing list; > is it possible your post hasn't made it across to > Usenet either? I read this thru the gmane news interfase

  1   2   >