Simulate `bash` behaviour using Python and named pipes.

Hi everybody, I am trying to understand how to use named pipes in python to launch external processes (in a Linux environment). As an example I am trying to imitate the behaviour of the following sets of commands is bash: mkfifo named_pipe ls -lah named_pipe cat named_pipe In Python I

Re: Simulate `bash` behaviour using Python and named pipes.

Hi Paul, first of all thanks for the help. I am aware of the first solutions, just now I would like to experiment a bit with using named pipes (I also know that the example is trivial, but it just to grasp the main concepts) You can also pass a file object to p1's stdout and p2's stdin if

Re: Simulate `bash` behaviour using Python and named pipes.

Hi MRAB, thanks for the reply! Opening the pipe for reading will block until it's also opened for writing, and vice versa. OK. In your bash code, 'ls' blocked until you ran 'cat', but because you ran 'ls' in the background you didn't notice it! Right. In your Python code,

Re: Simulate `bash` behaviour using Python and named pipes.

Hi Alister, Are you sure you are using the correct tool for the task? Yes. For two reasons: 1. I want to learn how to do this in Python :) 2. for an application I have in mind I will need to run external tools (not developed by me) and process the output using some tools that I have written in

Re: Simulate `bash` behaviour using Python and named pipes.

Thanks MRAB, You need to ensure that the pipe is already open at the other end. So I need to open the process that reads the pipe before writing in it? Why are you using a named pipe anyway? For some bug in ipython (see my previous email) I can't use subprocess.Popen and pipe in the

Re: Piping processes works with 'shell = True' but not otherwise.

thanks and what about python 2.7? In Python 3.3 and above: p = subprocess.Popen(..., stderr=subprocess.DEVNULL) P.s. sorry for the late reply, I discovered I don't receive notifications from google groups.. -- http://mail.python.org/mailman/listinfo/python-list

Re: Simulate `bash` behaviour using Python and named pipes.

Thanks this works (if you add shell=True in Popen). If I don't want to use shell = True, how can I redirect the stdout to named_pipe? Popen accepts an open file handle for stdout, which I can't open for writing because that blocks the process... os.mkfifo(named_pipe, 0777) ls_process =

Re: Simulate `bash` behaviour using Python and named pipes.

You're back to using separate threads for the reader and the writer. And how do I create separate threads in Python? I was trying to use the threading library without not too success.. -- http://mail.python.org/mailman/listinfo/python-list

Re: Simulate `bash` behaviour using Python and named pipes.

my_thread.join() Thanks! I managed to make it work using the threading library :) -- http://mail.python.org/mailman/listinfo/python-list

Using Pool map with a method of a class and a list

Hi guys, I would like to apply the Pool.map method to a member of a class. Here is a small example that shows what I would like to do: from multiprocessing import Pool class A(object): def __init__(self,x): self.value = x def fun(self,x): return self.value**x l = range(10)

Re: Using Pool map with a method of a class and a list

On Tuesday, 6 August 2013 18:12:26 UTC+1, Luca Cerone wrote: Hi guys, I would like to apply the Pool.map method to a member of a class. Here is a small example that shows what I would like to do: from multiprocessing import Pool class A(object): def __init__(self,x

Re: Using Pool map with a method of a class and a list

Hi Chris, thanks Do you ever instantiate any A() objects? You're attempting to call an unbound method without passing it a 'self'. I have tried a lot of variations, instantiating the object, creating lambda functions that use the unbound version of fun (A.fun.__func__) etc etc.. I have

Re: Using Pool map with a method of a class and a list

Hi Joshua thanks! I think you might not understand what Chris said. Currently this does *not* work with Python 2.7 as you suggested it would. op = map(A.fun,l) Yeah actually that wouldn't work even in Python 3, since value attribute used by fun has not been set. It was my mistake in the

Re: Using Pool map with a method of a class and a list

doesn't work neither in Python 2.7, nor 3.2 (by the way I can't use Python 3 for my application). Are you using Windows? Over here on 3.3 on Linux it does. Not on 2.7 though. No I am using Ubuntu (12.04, 64 bit).. maybe things changed from 3.2 to 3.3? from multiprocessing import Pool

Re: Using Pool map with a method of a class and a list

Thanks for the post. I actually don't know exactly what can and can't be pickles.. not what partialing a function means.. Maybe can you link me to some resources? I still can't understand all the details in your code :) -- http://mail.python.org/mailman/listinfo/python-list

Re: Using Pool map with a method of a class and a list

Thanks for the help Peter! def make_instancemethod(inst, methodname): return getattr(inst, methodname) This is just getattr -- you can replace the two uses of make_instancemethod with getattr and delete this ;). D'oh ;) --

Nested virtual environments

Dear all, is there a way to nest virtual environments? I work on several different projects that involve Python programming. For a lot of this projects I have to use the same packages (e.g. numpy, scipy, matplotlib and so on), while having to install packages that are specific for each project.

Re: Nested virtual environments

-wheel --no-index --find-links=/local/wheels -r /local/requirements.txt [1] https://wheel.readthedocs.org [2] http://www.pip-installer.org/en/latest/cookbook.html#building-and-installing-wheels 2013/8/9 Luca Cerone luca@gmail.com Dear all

How to keep cookies when making http requests (Python 2.7)

Hi everybody, I am trying to write a simple Python script to solve the riddle at: http://quiz.gambitresearch.com/ The quiz is quite easy to solve, one needs to evaluate the expression between the curly brackets (say that the expression has value val) and go to the web page:

Re: How to keep cookies when making http requests (Python 2.7)

I have used cookielib externally to urllib2. It looks like this: from urllib2 import urlopen, Request from cookielib import CookieJar cookies = CookieJar() r = Request(...) cookies.add_cookie_header(r) # set the cookies R = urlopen(r, ...) # make the request

Re: How to keep cookies when making http requests (Python 2.7)

Dear all, first of all thanks for the help. As for your remark, you are right, and I usually tend to post questions in a way that is detached from the particular problem I have to solve. In this case since I only have a limited knowledge of the cookies mechanism (in general, not only in

Re: How to keep cookies when making http requests (Python 2.7)

Let me make an additional remark however: you should not expect to get complete details in a list like this - but only hints towards a solution for your problem (i.e. there remains some work for you). Thus, I expect you to read the cookielib/cookiejar documentation (part of Python's

Re: How to keep cookies when making http requests (Python 2.7)

Thanks Dieter, With respect to cookie handling, you do everything right. There may be other problems with the (wider) process. Analysing the responses of your requests (reading the status codes, the response headers and the response bodies) may provide hints towards the problem.

Sphinx Doctest: test the code without comparing the output.

Dear all, I am writing the documentation for a Python package using Sphinx. I have a problem when using doctest blocks in the documentation: I couldn't manage to get doctest to run a command but completely ignoring the output. For example, how can I get a doctest like the following to run

Re: Sphinx Doctest: test the code without comparing the output.

Dear Steven, thanks for the help. I am aware that I might have used the SKIP directive (as I hinted in my mail). Even if the fine manual suggests to do so I don't agree with it, though. The reason is simple: SKIP as the name suggests causes the code not to be run at all, it doesn't ignore the

Re: Sphinx Doctest: test the code without comparing the output.

And if you ignore the output, the error won't be caught either. What's the difference? 1 + 1 #doctest:+IGNORE_OUTPUT (not a real directive) 1000 The difference is that in that case you want to check whether the result is correct or not, because you expect a certain result. In

Re: Sphinx Doctest: test the code without comparing the output.

but if you're using this for a tutorial, you risk creating a breed of novice programmers who believe their first priority is to stop the program crashing. Smoke testing is Hi Chris, actually my priority is to check that the code is correct. I changed the syntax during the development, and

Re: Sphinx Doctest: test the code without comparing the output.

That is not how doctest works. That test fails because its output is: ok.. is there a tool by which I can test if my code runs regardless the output? The only wild-card output that doctest recognises is ellipsis, and like all wild-cards, can match too much if you aren't careful. If

Re: Sphinx Doctest: test the code without comparing the output.

This makes no sense. If you ignore the output, the code could do ANYTHING and the test would still pass. Raise an exception? Pass. SyntaxError? Pass. Print 99 bottles of beer? Pass. if you try the commands, you can see that the tests fail.. for example .. doctest:: raise

Re: Sphinx Doctest: test the code without comparing the output.

On Sunday, 22 September 2013 14:39:07 UTC+1, Ned Batchelder wrote: On 9/22/13 12:09 AM, Luca Cerone wrote: Hi Chris, actually my priority is to check that the code is correct. I changed the syntax during the development, and I want to be sure that my tutorial is up to date

Re: Sphinx Doctest: test the code without comparing the output.

It won't be very good documenation any more but nothing stops you from examining the result in the next doctest and making yourself happy about it. x = input(indeterminate:) result = '{}'.format(x)) result.startswith(') and result.endswith(') True Hi Neil,

Re: Sphinx Doctest: test the code without comparing the output.

I don't know why but it seems that google groups stripped the indentation from the code. I just wanted to ensure you that in the examples that I have run the definition of myfunc contained correctly indented code! On Monday, 23 September 2013 15:45:43 UTC+1, Luca Cerone wrote: .. doctest

Re: Sphinx Doctest: test the code without comparing the output.

The docstring for doctest.DocTestRunner contains the example code I was looking for. Thanks, I will give it a try! -- Neil Cerutti -- https://mail.python.org/mailman/listinfo/python-list

Re: What's the best way to extract 2 values from a CSV file from each row systematically?

I'd really appreciate any suggestions or help, thanks in advance! Hi Alex if you know that you want only columns 3 and 5, you could also use list comprehension to fetch the values: import csv with open('yourfile.csv','rU') as fo: #the rU means read using Universal newlines cr =

Piping processes works with 'shell = True' but not otherwise.

Hi everybody, I am new to the group (and relatively new to Python) so I am sorry if this issues has been discussed (although searching for topics in the group I couldn't find a solution to my problem). I am using Python 2.7.3 to analyse the output of two 3rd parties programs that can be

Re: Piping processes works with 'shell = True' but not otherwise.

Can you please help me understanding what's the difference between the two cases? Hi guys has some of you ideas on what is causing my issue? -- http://mail.python.org/mailman/listinfo/python-list

Re: Piping processes works with 'shell = True' but not otherwise.

Could you provide the *actual* commands you're using, rather than the generic program1 and program2 placeholders? It's *very* common for people to get the tokenization of a command line wrong (see the Note box in http://docs.python.org/2/library/subprocess.html#subprocess.Popen for some

Re: Piping processes works with 'shell = True' but not otherwise.

Will it violate privacy / NDA to post the command line? Even if we can't actually replicate your system, we may be able to see something from the commands given. Unfortunately yes.. -- http://mail.python.org/mailman/listinfo/python-list

Re: Create a file in /etc/ as a non-root user

fd = open('/etc/file','w') fd.write('jpdas') fd.close() Hi Bibhu, that is not a Python problem, but a permission one. You should configure the permissions so that you have write access to the folder. However unless you know what you are doing it is discouraged to save your file in the

Re: Piping processes works with 'shell = True' but not otherwise.

That's because stdin/stdout/stderr take file descriptors or file objects, not path strings. Thanks Chris, how do I set the file descriptor to /dev/null then? -- http://mail.python.org/mailman/listinfo/python-list