date:20140512

Re: Using threads for audio computing?

2014-05-12 Thread Chris Angelico

On Mon, May 12, 2014 at 3:54 PM, lgabiot lgab...@hotmail.com wrote:
 So back to my original question: A Queue and two threads (producer/consumer)
 seems a good answer to my problem, or is there a better way to solve it?
 (again, I'm really a beginner, so I made up this solution, but really wonder
 if I do not miss a well known obvious much better idea).

Well, the first thing I'd try is simply asking for more data when
you're ready for it - can you get five seconds' of data all at once?
Obviously this won't work if your upstream buffers only a small
amount, in which case your thread is there to do that buffering; also,
if you can't absolutely *guarantee* that you can process the data
quickly enough, every time, then you need to use the queue to buffer
that.

But otherwise, it sounds like a quite reasonable way to do things.

ChrisA
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Using threads for audio computing?

2014-05-12 Thread lgabiot


Le 12/05/14 07:58, Chris Angelico a écrit :


Well, the first thing I'd try is simply asking for more data when
you're ready for it - can you get five seconds' of data all at once?
Obviously this won't work if your upstream buffers only a small
amount, in which case your thread is there to do that buffering; also,
if you can't absolutely *guarantee* that you can process the data
quickly enough, every time, then you need to use the queue to buffer
that.

But otherwise, it sounds like a quite reasonable way to do things.

ChrisA



Ok, thanks a lot!
--
https://mail.python.org/mailman/listinfo/python-list

Re: Using threads for audio computing?

2014-05-12 Thread Stefan Behnel

lgabiot, 12.05.2014 07:33:
 Le 11/05/14 17:40, lgabiot a écrit :
 
 I guess if my calculation had to be performed on a small number of
 samples (i.e. under the value of the Pyaudio buffer size (2048 samples
 for instance), and that the calculation would last less than the time it
 takes to get the next 2048 samples from Pyaudio, I wouldn't need the
 Queue and Thread system.
 But in my case where I need a large buffer, it might not work?
 Unless I ask pyaudio to feed me directly with 5 seconds chunks (instead
 of the usual buffer sizes: 1024, 2048, etc...), which I didn't try,
 because I hadn't though of it.
 
 I guess this solution might probably not work, since it would mean that the
 calculation should be quick enough so it wouldn't last longer than 1 sample
 (1/48000 s for instance), since while doing the calculation, no audio would
 be ingested (unless pyAudio possess some kind of internal concurrency system).
 Which leads me to think that a buffer (queue) and separate threads
 (producer and consumer) are necessary for this task.

This sounds like a use case for double buffering. Use two buffers, start
filling one. When it's full, switch buffers, start filling the second and
process the first. When the second is full, switch again.

Note that you have to make sure that the processing always terminates
within the time it takes to fill the other buffer. If you can't assure
that, however, you have a problem anyway and should see if there's a way to
improve your algorithm.

If the fill my buffer call in PyAudio is blocking (i.e. if it returns
only after filling the buffer), then you definitely need two threads for this.


 But AFAIK the python GIL (and in smaller or older computers that have only
 one core) does not permit true paralell execution of two threads.

Not for code that runs in the *interpreter, but it certainly allows I/O
and low-level NumPy array processing to happen in parallel, as they do not
need the interpreter.

Stefan


-- 
https://mail.python.org/mailman/listinfo/python-list

ANN: Cerridwen 1.0c4 (initial release)

2014-05-12 Thread leslie . polzer

Dear Python friends,

today I'm proud to be

ANNOUNCING: The initial release of a new package, named Cerridwen (1.0c4).

PyPI entry: https://pypi.python.org/pypi/cerridwen


What gives?

  The author perceives a lack of modern open source software
  providing high quality planetary data that is suitable for
  astronomical and astrological purposes.

  So, to start amending that situation I wrote Cerridwen.

  Cerridwen is a moon data API relying on pyswisseph and numpy
  to provide accurate and comprehensive data on the moon.

  With its help you will, for example, be able to get the moon's
  current tropical sign, the dates of the next new and full moons
  and its current absolute and relative diameter.

  Cerridwen offers a JSON HTTP API and comes a simple command-line
  tool. These are built on a library module that you can use to
  develop your own applications.


And you're invited too!

  Please try it if you're interested! :-)

  I've also set up a test server for the API here:

http://cerridwen.viridian-project.de/api/v1/moon

  You can load it in your browser too for a quick peek! Please
  note that the given information may be up to 10 seconds old,
  due to efficiency considerations in the current version of
  Cerridwen.

  All feedback and participation is welcome, including forks and
  patches. Cerridwen's code is on Github.


Need more information?

  Please do take a look at the PyPI page, it contains more detailed
  information. Also feel free to write to me at leslie.polzer SPAMPROTECT AT 
gmx.net.


Thanks for reading!

  With Cerridwen I'm trying to give something back to the Python
  community. This is my first publicly released Python package.

Cheers,

  Leslie
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Using threads for audio computing?

2014-05-12 Thread lgabiot


Le 12/05/14 08:13, Stefan Behnel a écrit :

This sounds like a use case for double buffering. Use two buffers, start
filling one. When it's full, switch buffers, start filling the second and
process the first. When the second is full, switch again.

Note that you have to make sure that the processing always terminates
within the time it takes to fill the other buffer. If you can't assure
that, however, you have a problem anyway and should see if there's a way to
improve your algorithm.

If the fill my buffer call in PyAudio is blocking (i.e. if it returns
only after filling the buffer), then you definitely need two threads for this.



But AFAIK the python GIL (and in smaller or older computers that have only
one core) does not permit true paralell execution of two threads.


Not for code that runs in the *interpreter, but it certainly allows I/O
and low-level NumPy array processing to happen in parallel, as they do not
need the interpreter.

Stefan



Thanks for your answer.

If I follow your explanations, I guess I have to review my understanding 
of python execution model (I have to admit it is quite crude anyway).


In my understanding, without threads, I would have two functions:
- get_audio() would get the 5 seconds of audio from Pyaudio
- process_audio() would process the 5 seconds of audio

the main code would be roughly executing this:
while(True)
get_audio()
process_audio()

so since the audio is a live feed (which makes a difference, say, with 
an audio file analyser program), the get_audio() part must take 5 
seconds to execute. (but most probably the processor stays still most of 
the time during the get_audio() part).

then once get_audio() is done, process_audio() begins.
Process_audio will take some time. If that time is greater that the 
times it takes for the next audio sample to arrive, I have a problem.

(which you already explained differently maybe with:
 If the fill my buffer call in PyAudio is blocking (i.e. if it returns
 only after filling the buffer), then you definitely need two threads 
for this.

)

So if I follow you, if the Pyaudio part is Non-blocking there would be 
a way to make it work without the two threads things. I'm back to the 
Pyaudio doc, and try to get my head around the callback method, which 
might be the good lead.





--
https://mail.python.org/mailman/listinfo/python-list

Re: parsing multiple root element XML into text

2014-05-12 Thread Peter Otten

Percy Tambunan wrote:

 On Friday, May 9, 2014 4:02:42 PM UTC+7, Chris Angelico wrote:
 On Fri, May 9, 2014 at 6:59 PM, Percy Tambunan percy.tambu...@gmail.com
 wrote:
 
  Hai, I would like to parse this multiple root element XML
 
 
 
 Easy fix might be to wrap it in root and /root, which will give
 
 you a new root. Would that help?
 
 
 
 ChrisA
 
 Thanks chris for the idea.
 Any suggestion to make it print like this:
 
 create enumdnsched 4.1.0.1.4.7.3.4.3.2.6.e164.arpa -set naptrFlags=nu
 create enumdnsched 5.1.0.1.4.7.3.4.3.2.6.e164.arpa -set naptrFlags=nu

[Stefan Behnel]

 ElementTree's XMLParser() can be use efficiently for this. Something like
 this should work:
 
 from xml.etree.ElementTree import XMLParser
 
 parser = XMLParser()
 parser.feed(b'root')
 parser.feed(real_input_data)
 parser.feed(b'/root')
 root = parser.close()
 
 for subtree in root:
 ...
 
Have you tried to integrate Stefan's example into your script? If so, what 
is the current state of your code, and what problems prevented you from 
completing it?

Expect help to fix these problems, but not necessarily a ready-to-run 
solution.

If you have not made an attempt yet look here for ideas on how you can use 
XPath to extract interesting data from the subtree:

https://docs.python.org/dev/library/xml.etree.elementtree.html#example


-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fortran (Was: The does Python have variables? debate)

2014-05-12 Thread Alain Ketterlin

Mark H Harris harrismh...@gmail.com writes:

 On 5/11/14 12:05 PM, Alain Ketterlin wrote:
 Julia is Matlab and  R, Python, Lisp, Scheme; all rolled together on
 steroids. Its amazing as a dynamic language, and its fast, like
 lightning fast as well as multiprocessing (parallel processing) at its
 core. Its astounding, really.

 Hmmm...

 Its number concept is unified,

 What exactly is unified? There is no implicit promotion between
 primitive types and BigInt/Float.


 The built-in math functions (extensive, by the way) just work, and
 they work consistently the way you might expect across types. Consider
 sqrt():
[...]
 You'll notice that I did not need to import anything to use sqrt(),
 and sqrt() takes all types and does something meaningful with them.

Sorry, i wasn't clear enough: doing something meaningful with them is
precisely where the problem is. Every single operation requires
multiple-dispatch (i.e., dynamically testing types, converting to a
common type, and selecting the version of sqrt to use). That's probably
more than the time it takes to actually perform the computation, a bit
like what happens with x+y on integers with Python, where only a
fraction of time is spent on adding integers.

When you are doing scientific computation, this overhead is
unacceptable, because you'll have zillions of computations to perform.

Julia provides a way to make things fast: typing. If you provide
explicit types, the dynamic typing part obviously disappears, and
the overhead is removed.

But then, you're not too far from Fortran, or C/C++.

 The following code will produce over 100,000 digits of π (pi) in less
 than 2 seconds on a low-end processor, like my mac mini dual core
 2Ghz:

You seem to be discovering the power of the libraries that are behind
all this (MPFR in that case)...

[...]
 But, like lisp, Julia's internal structures are lists, so, it can
 create and modify its own code on-the-fly. [...]

Sorry, I was comparing to Fortran, and it's use in scientific computing.
Self modifying code is totally irrelevant there.

-- Alain.
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Plotting multiple datasets with gnuplot

2014-05-12 Thread s . c . wouters

On Friday, October 9, 2009 12:12:54 PM UTC+2, Gabriel Genellina wrote:
 En Fri, 09 Oct 2009 06:36:45 -0300, Rob Garrett rgagarr...@gmail.com  
 escribiï¿oe:
 
  I'm trying to get gnuplot to display multiple data series on a single
  plot using gnuplot in python.  I've searched around and haven't found
  a solution to how to do this when I have a variable-length list of
  plots to add.
 
  For example, the following code will work:
 
  plotData1 = Gnuplot.PlotItems.Data(data1, title=title1)
  plotData2 = Gnuplot.PlotItems.Data(data2, title=title2)
  g.plot( plotData1, plotData2 )
 
  [I've removed the rest of the code for clarity]
 
  But how can I do the following instead:
 
  data = []
  ...
  # Populate data
  ...
  plots = []
  for dataSet in data:
plots.append(dataSet)
  g.plot(plots)
 
 g.plot(*plots) should work; it's like calling g.plot(plots[0], plots[1],  
 plots[2]...)
 
 See http://docs.python.org/reference/expressions.html#calls for the gory  
 details.
 
 -- 
 Gabriel Genellina

I just spend an hour searching the web for a similar problem and finally found 
your answer. Thank you, it works great!
-- 
https://mail.python.org/mailman/listinfo/python-list

Re: Fortran

2014-05-12 Thread Mark Lawrence

On 12/05/2014 03:28, Steven D'Aprano wrote:

On Mon, 12 May 2014 01:27:17 +0100, Mark Lawrence wrote:

On 12/05/2014 00:51, Steven D'Aprano wrote:

Cars are standardized -- there are basically two types, manuals and
automatics.

Sadly they can still go wrong due to modern engineering practices. In
my neck of the woods some years ago people were killed when standing at
a bus stop, because the car driver was desperately pressing down on the
automatic's brake

Perhaps (s)he should have steered the car away from the people. Or I
suppose the steering wheel failed as well?

The entire vehicle was perfectly sound after the event, including the
brake and accelerator controls. It was only during the accident that
the interference played havoc. Pure coincidence of course.

but EMI overrode the engine controls

EMI? The record company? Wow, I knew they were evil, but I didn't realise
they were *that* evil.

Ho, ho, ho :)

and the car
simply went faster. At least that is what the defence claimed at the
trial. With no expert on the prosecution to refute the claim not
guilty was the verdict.

Sounds like the prosecution were just going through the motions. They
should have either had an expert able to refute the claim, or not
prosecuted in the first place.

Can't do that if you don't know in advance what the defence is :(

Personally, I'm rather skeptical about claims of I kept pushing the
brake but the car just accelerated. There is a long and inglorious
history of people stepping on the wrong pedal when in a panic, or drunk,
or distracted. I've even done it myself. (I expect *every* driver
has, at some point.) And a not-quite-as-long but even more inglorious
history of lawyers inventing nonsense links between the brake pedal and
the accelerator in order to extort money from car manufacturers. E.g. see
Galileo's Revenge by Peter W Huber and the case of the mythical, but
amazingly profitable for the lawyers involved, Audi Sudden Acceleration
Syndrome.

For me personally, perhaps the most despicable part of the whole sordid
story was the case of Wende Gatts, who ran over Darlene Norris, causing
$300,000 in damages. Not only did she got off scot-free, thanks to the
junk science invented by her expert witness, but actual victim Norris
was ordered to pay Gatts' legal fees of $64K.

Of course, that was in the late 1980s, when even luxury cars still had
mechanical linkage between the user and the brakes. These days, when
nearly everything in the car is computer controlled, I wouldn't be
*quite* so skeptical. Nevertheless, chances are almost certain that by
far the majority of unexpected acceleration cases are PEBCAP errors.

1 2 >

1 - 100 of 178 matches

Mail list logo