Re: TechRepublicDEVELOPERCXO JPMorgan's Athena has 35 million lines of Python code, and won't be updated to Python 3 in time
On 2019-09-14 07:30, Gene Heskett wrote: On Saturday 14 September 2019 04:37:14 Larry Martell wrote: On Fri, Sep 13, 2019 at 1:37 PM Skip Montanaro wrote: https://www.techrepublic.com/google-amp/article/jpmorgans-athena-has -35-million-lines-of-python-code-and-wont-be-updated-to-python-3-in-t ime/ I doubt this is unusual, and presume JP Morgan is big enough to handle the change of status, either by managing security releases in-house or relying on third-party releases (say, Anaconda). When I retired from Citadel recently, most Python was still 2.7 (though the group I worked in was well on the way to converting to 3.x, and no new applications were written against 2.7). Bank of America has an enterprise-wide system called Quartz. I wouldn't be surprised if it was still running Python 2.7 (though I don't know for sure). Yes Quartz is 2.7. As I’ve said before here, I know a lot of companies running large apps in 2.7 and they have no intention of moving to 3. And I, Larry, have little doubt that the hackers have a hole into a 2.7 install, all squirreled away, and waiting until 2.7 security support goes away. It's the nature of the thing. They will get hacked. Its like asking if concrete will crack as you are watching it being poured, will is the wrong question, when is far more correct. And it will cost them trillions in the long haul. The courts, adjudicating damages, will not be kind to the foot dragger's who think they are saving money. History sure seems to be pointing in that direction recently. Its a puzzle to me, why so-called sane MBA's cannot understand that the best defense is spending money on the offense by updateing their in-house operating code. Or the OS under it. Is anyone interested in contacting these companies -- or the companies from which they buy cybersecurity insurance -- and inviting them to provide paid staff to maintain 2.7 and to offer further offer consulting services to help these clients inventory what they have and how much it would cost to migrate? For example, how much would it cost to write and maintain an emulator for 2.7.16 in 3.7.4? The Python Software Foundation does not want to maintain 2.7 for free anymore, but if there is sufficient demand, they should be thrilled to make a handsome profit off of it -- while providing high quality, good paying jobs for smart Pythonistas. As I'm thinking about it, the companies that provide cybersecurity insurance could be the best points of leverage for this, because they think about these kinds of things all the time. Insurance companies for decades and probably well over 100 years have required their commercial clients to employ night watch crews, who make the rounds of a facility collecting time stamps from different points in the facility, which they provide to insurer(s) in exchange for reduced rates -- on as a condition of getting insurance in the first place. This is conceptually and practically the same kind of thing. Spencer Graves Cheers, Gene Heskett -- https://mail.python.org/mailman/listinfo/python-list
Re: Your IDE's?
On 2019-03-25 18:55, Gene Heskett wrote: On Monday 25 March 2019 18:20:29 DL Neil wrote: On 26/03/19 10:38 AM, John Doe wrote: What is your favorite Python IDE? In case you are tempted to reply, neither of "John"'s supposed domains resolves (to a web site)/has been registered. -- Regards =dn your email agent is inventing links? There were none in the single msg I got from a john doe. Unless they were buried in the headers that kmail doesn't show me.. The original email was "From John Doe ", with a "Reply to John Doe ". "Doe.com" is a URL being advertised for sale for $150,000. "ping something.com" returns, "cannot resolve something.com: Unknown host". Clearly the original poster is playing games with us. Spencer Cheers, Gene Heskett -- https://mail.python.org/mailman/listinfo/python-list
Re: 3 random numbers
On 2019-01-14 23:29, caig...@gmail.com wrote: So I was given this question to be solved in Python 3 : Pick any 3 random ascending numbers and write out a loop function that prints out all 3 numbers. This was the code and solution presented to me. Can anyone understand it and explain it to me please ? I have looked at it but cannot see how it was derived and why the correct solution was printed. Thanks alot ! # any 3 ascending numbers , counter must start at 0. # 400 467 851 i = 0 x = 400 while x < 852: print(x) if i > 0: x = x + ((i + 4) * 67) + (i * 49) else: x = x + 67 i = i + 1 This sounds like a homework problem for a class. I don't know how this list treats such questions, but I suspect answering such questions may be discouraged. Hint: Read the documentation on "while" and then trace the iterations. Spencer -- https://mail.python.org/mailman/listinfo/python-list
Re: sampling from frequency distribution / histogram without replacement
On 2019-01-14 18:40, duncan smith wrote: On 14/01/2019 22:59, Gregory Ewing wrote: duncan smith wrote: Hello, Just checking to see if anyone has attacked this problem before for cases where the population size is unfeasibly large. The fastest way I know of is to create a list of cumulative frequencies, then generate uniformly distributed numbers and use a binary search to find where they fall in the list. That's O(log n) per sample in the size of the list once it's been set up. That's the sort of thing I've been thinking about. But once I'd found the relevant category I'd need to reduce its frequency by 1 and correspondingly update the cumulative frequencies. Alternatively, I could add an extra step where I selected a unit from the relevant category with probability equal to the proportion of non-sampled units from the category. I could maybe set up an alias table and do something similar. The other thing I was thinking about was iterating through the categories (ideally from largest frequency to smallest frequency), generating the numbers to be sampled from the current category and the remaining categories (using numpy.random.hypergeometric). With a few large frequencies and lots of small frequencies that could be quite quick (on average). Alternatively I could partition the categories into two sets, generate the number to be sampled from each partition, then partition the partitions etc. binary search style. I suppose I'll try the both the alias table + rejection step and the recursive partitioning approach and see how they turn out. Cheers. R has functions "sample" and "sample.int"; see "https://www.rdocumentation.org/packages/base/versions/3.5.2/topics/sample;. You can call R from Python, "https://sites.google.com/site/aslugsguidetopython/data-analysis/pandas/calling-r-from-python;. These are in the "base" package. I believe they have been an important part of the base R language almost since its inception and have been used extensively. You'd have to work really hard to do better, in my judgment. Spencer Graves DISCLAIMER: I'm primarily an R guy and only use Python when I can't find a sensible way to do what I want in R. Duncan -- https://mail.python.org/mailman/listinfo/python-list
Re: Good editor for python
People rave about Jupyter Notebooks, which reportedly allow you
to mix narrative with code describing what you are doing and why.
I primarily program in R, and RMarkdown Documents in RStudio
allow me to mix narrative with R and Python code. I explain what I'm
doing and why, then write "```{python}" ... "```" to encapsulate a
Python code snippet and "```{r}" ... "```" for an R code snippet. Or I
just use the Idle editor that comes with Python.
Someone suggested that Apache Zeppelin and / or BeakerX might be
able to do this also, but I've not tried or verified them.
Spencer Graves
On 2018-11-11 08:11, Andrew Z wrote:
If you do scripts - emacs/vi is the way to go.
If you need something more (like creating libraries, classes) go with
pycharm. It is a professionally made IDE.
Over past 2 years ive been trying to "downgrade" myself to something with
less belts and whistles, but come back to it all the time.
On the other hand , if you already use emacs - u should not need anything
else.
On Sun, Nov 11, 2018, 04:15 Olive
I am not a professional programmer but I use Python regularly for custom
scripts (and plot with matplotlib). I have just learned VBA for Excel: what
I found amazing was their editor: it is able to suggest on the spot all the
methods an object support and there is a well-integrated debugger. I wonder
if something similar exists for Python. For now I just use emacs with the
command line pdb. What do people use here? Ideally I would like to have
something that is cross platform Windows/Linux.
Olivier
--
https://mail.python.org/mailman/listinfo/python-list
--
https://mail.python.org/mailman/listinfo/python-list
Mixing R and Python in the same Jupyter Notebook and finding Python code within an RMarkdown document
Hello, All:
Two questions:
1. Is it feasible to mix R and Python code in the same
Jupyter notebook? If yes, can you please point me to an example?
2. How can one find Python code from within and R Markdown
document?
** "https://github.com/sbgraves237/radioMonitor;
includes "radioMonitor-init2018-10-11.Rmd" that shows it's possible to
mix R and Python snippets in the same R Markdown (*.Rmd) document.
However, "radioMonitor0_1.Rmd" calls 'system("py idle0_1.py")' in an R
snippet, which does what I want (namely recording 5 seconds of whatever
is connected to "audio in" or something similar on your computer and
writing it to "KKFI2018-10-12t13_16-5sec.wav".
Thanks,
Spencer Graves
--
https://mail.python.org/mailman/listinfo/python-list
Wikipedia on Python
Thanks to Léo El Amri and Thomas Jollans for their quick and helpful replies to my question about "Package creation documentation". Beyond that, I'd like to encourage people on this list to review the Wikipedia article on "Python (programming language)",[1] especially the claim that "a package is a Python module with an __path__ attribute", which I added on 2018-09-24 to help me understand the distinction. That Wikipedia article has averaged over 6,000 views per day over the past 3 years. Therefore, any improvements will benefit lots of people. If you have suggestions for how the article might be improved, you can post them to the "Talk" page associated with that article or send them to me. If you are "autoconfirmed" with the Wikimedia system, you can make the changes yourself. Thanks, Spencer Graves [1] https://en.wikipedia.org/wiki/Python_(programming_language) On 2018-10-16 11:14, Léo El Amri wrote: Hello Spencer, On 16/10/2018 17:15, Spencer Graves wrote: Where can I find a reasonable tutorial on how to create a Python package? IMO, the best documentation about this is the tutorial: https://docs.python.org/3/tutorial/modules.html#packages According to the Python 3 Glossary, "a package is a Python module with an __path__ attribute."[1] What you are looking at are the technical details of what a package is. Incidentally, if you follow the tutorial, everything will get in-place. I found "packaging.python.org", which recommends "Packaging Python Projects"[2] and "An Overview of Packaging for Python".[3] packaging.python.org is centered on "How to install and distribute Python packages (Or modules)" - Léo -- https://mail.python.org/mailman/listinfo/python-list
Package creation documentation?
Where can I find a reasonable tutorial on how to create a Python package? I've not created a Python package before, and I want to. Sadly, I'm having trouble finding reasonable documentation on how to do so. According to the Python 3 Glossary, "a package is a Python module with an __path__ attribute."[1] I found "packaging.python.org", which recommends "Packaging Python Projects"[2] and "An Overview of Packaging for Python".[3] I failed to find "__path__" in either. I've started a project on GitHub for this, which includes what I have so far toward building a Python package I want along with RStudio Rmarkdown Documents summarizing what I've tried so far.[4] What would you suggest I do to understand what I should do to create a "__path__ attribute" for this, and what I should do after that? I have decades of coding experience, but only a small portion of that was with Python, and that was roughly 7 years ago. Thanks, Spencer Graves [1] https://docs.python.org/3/glossary.html#term-package [2] https://packaging.python.org/tutorials/packaging-projects/ [3] https://packaging.python.org/overview/ [4] https://github.com/sbgraves237/radioMonitor -- https://mail.python.org/mailman/listinfo/python-list
Re: Overwhelmed by the Simplicity of Python. Any Recommendation?
On 2018-10-12 11:44, Rhodri James wrote: On 12/10/18 17:12, Rob Gaddi wrote: On 10/11/2018 11:29 PM, Kaan Taze wrote: Hi everyone, Since this is my first post to mail-list I'm kind of hesitant to ask this question here but as many of you spend years working with Python maybe some of you can guide me. What I trouble with is not a logical error that exist on a program I wrote. It's the Python itself. Well, I'm 22 years old CS student -from Turkey- and what they showed us at university was C Language and Java but I mainly use C in school projects etc. So it's been few months that I started to use Python for my personal side-projects. There are lots of resources to learn language. I do what I need to do with Python too but I was kinda shocked when I solve Python questions at Hackerrank. Even with list comprehensions you can implement in very smart way to get things done and easy. Iterations, string operations. The codes I see on the Internet using basics in a very clever way which I couldn't come up with the same solution if I tried to for some time. I do understand this ways but coming from ANSI C makes it hard to see this flexibility. I probably do things in a both inefficient and hard way in my projects. How do I get used to this? Is this just another "practice, practice, practice" situation? Anything you can recommend? All the best. Kaan. A) Yes, it's practice practice practice. B) Don't get hung up on finding the clever solution. Comprehensions and generators and lots of other things are great under some circumstances for making the code clearer and easier to read, but they too can become the hammer that makes everything look like a nail. The most important thing is that your code is logical, clean, and easy to understand. If it doesn't take full advantage of the language features, or if the performance isn't optimized to within an inch of its life, well so be it. I completely agree. I too have come from a background in C, and still do most of my day job in C or assembler. It took a while before I was writing idiomatic Python, never mind efficient Python (arguably I still don't, but as Rob says, who cares?). Don't worry about it; at some point you will discover that the "obvious" Python you are writing looks a lot like the code you are looking at now and thinking "that's really clever, I'll never be able to to that." I suggest two things: 1. Document your work as you do it in something like Jupyter Notebooks that were discussed in another recent thread. I use "R Markdown Documents" in RStudio. This allows me to mix Python code with text and code for other languages (including R, C, SQL, and others). I tried installing Jupyter using Ananconda Navigator and failed -- under both Windows 7 and macOS 10.14.[1] One consulting gig I had involved spending roughly a week creating an "R Markdown Document" mixing text with code and results analyzing a client's data, followed by months replying to questions by asking, "Did you look at p. ___ in the R Markdown Document I gave you" -- plus a few extensions to that document. An article in The Atlantic last April claimed, "The scientific research paper is obsolete" and is being replaced by Jupyter Notebooks.[2] I'd like to see a serious comparison of "R Markdown Documents" with "Jupyter Notebooks": The latter may be better, but I was unable to even started with them after two days of effort. 2. Find a reasonable "Introduction to Python" on the web. Others on this list should be able to suggest several. I just found "https://docs.python.org/2/tutorial/introduction.html". A web search for "an introduction to Python" identified several others. I'd also be interested in reference(s) others might suggest for "creating python packages". Hope this helps. Spencer Graves [1] RStudio offers a free "Desktop" version, which I have used routinely for the past three years. It's available at "www.rstudio.com/products/rstudio/download". Creating "R Markdown Documents" (File > "New File" > "R Markdown..." in RStudio) made a dramatic improvement in my productivity in many ways similar to those described by Paul Romer in "https://paulromer.net/jupyter-mathematica-and-the-future-of-the-research-paper;. (I also experimented with File > "New File" > "R Notebook" in RStudio and encountered bazaar errors I could not understand -- and no benefits that I could see that would push me to spend more time trying to get past the problems I encountered. I've used "R Markdown Documents" extensively for three years -- with R -- and I found it easy to use with Python once I learned I could do that. See "https://bookdown.org/yihui/rmarkdown;. [2] https://www.theatlantic.com/science/archive/2018/04/the-scientific-paper-is-obsolete/556676/ -- https://mail.python.org/mailman/listinfo/python-list
Problems installing RStudio from Anaconda Navigator
I clicked "Install" for "RStudio 1.1.456" in Anaconda Navigator 1.9.2 under Windows 7 Home Premium SP1. It said "Installing application RStudio" for over 30 minutes. It looked stuck. Then I read a GitHub post that said, "On the environments tab create a new R/Python3.5 environment", then install RStudio into that new environment.[1] So I killed the RStudio install and looked at the "Environments tab". I'm new to Anaconda Navigator, and I didn't know what I was looking at so decided to ask here. Suggestions? Thanks, Spencer Graves p.s. I do not currently have RStudio installed on that machine. On a macOS 10.13.6 machine, I have have RStudio 1.2.792 on a macOS 10.13.6. Anaconda Navigator 1.9.2 on my Mac fails to recognize the existing and more recent installation of RStudio 1.2.792 there. Should I, e.g., try to "Import" that existing RStudio installation? [1] https://github.com/conda/conda/issues/4204 -- https://mail.python.org/mailman/listinfo/python-list
Re: Multiple problems with Python 3.7 under Windows 7 Home Premium
Thanks very much to all who replied. This problems was solved as follows: First I downloaded "PyAudio‑0.2.11‑cp37‑cp37m‑win_amd64.whl" from "https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio; as suggested by MRAB and Terry Reedy. Then I ran "python -m pip install PyAudio‑0.2.11‑cp37‑cp37m‑win_amd64.whl". Then I ran the script at "https://stackoverflow.com/questions/35344649/reading-input-sound-signal-using-python#35390981;. That produced "output.wav" containing 5 seconds of sound recorded from a radio connected to "audio in" on my Windows 7 machine. Thanks again. Spencer Graves On 2018-09-21 23:33, Terry Reedy wrote: On 9/21/2018 8:57 PM, MRAB wrote: On 2018-09-22 01:02, Michael Torrie wrote: On 09/21/2018 07:22 AM, Spencer Graves wrote: PYTHON - M PIP INSTALL PYAUDIO "python -m pip install pyaudio" stopped with 'error: Microsoft visual C++14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools;. You're going to want to wait until binary wheels of pyaudio for Python 3.7 and Windows 64-bit land in the PyPI repository. Unless you're already a visual studio user and have the where with all to build it from source. Python 3.7 is a recent release, so I'm not surprised that some packages don't yet have binary wheels in the repository. Christoph Gohlke's site appears to have a binary release for Python 3.7: https://www.lfd.uci.edu/~gohlke/pythonlibs/#pyaudio As I remember, he was building binaries for the 300 packages on the site well before the 3.7.0 release. -- https://mail.python.org/mailman/listinfo/python-list
Multiple problems with Python 3.7 under Windows 7 Home Premium
Hello: I'm having a series of problems getting Python 3.7 to work on a machine running Windows 7 Home Premium with SP1. WEBINSTALL.EXE: "python-3.7.0-amd64-webinstall.exe" stopped seemingly before it started. I can try it again and give you a more precise error message if you want, but I want to describe some of the other problems I've had first. "python-3.7.0-amd64.exe" acted like it installed properly, until I started using it. IDLE seemed to work for me on some simple tasks, but I wanted to try PyAudio, and I've so far not been able to make that work. PYTHON - M PIP INSTALL PYAUDIO "python -m pip install pyaudio" stopped with 'error: Microsoft visual C++14.0 is required. Get it with "Microsoft Visual C++ Build Tools": http://landinghub.visualstudio.com/visual-cpp-build-tools;. That web address "http://landinghub...; gave "404 NOT FOUND". However, I found elsewhere Microsoft Visual C++ Build Tools" and seemed to get them installed. Then "python -m pip install pyaudio" stopped with "error: command 'C:\\Program Files (x86)\\Microsoft Visual Studio\\BuildTools\\VC\\Tools\\MSVC\\14.15.26726\\bin\\HostX86\\x64\\c1.exe' failed with exit status 2". GCCWINBINARIES Searching for "... exit status 2" led me to a suggestion to use "https://github.com/develersrl/gccwinbinaries". That failed with "No Python installation found in the registried. It won't be possible to install the bindings." WINDOWS APP CERTIFICATION KIT Somehow, I got a notice to try "Windows App Certification Kit 10.0.17134.12". I tried that with "program installer: python-3.7.0-amd64.exe" and "command: python". This moved along, then generated a popup saying "Modify setup", then stopped. After waiting about 12 minutes, I clicked the "x" in the upper right. Then it moved again for a time until what looked like the same popup appeared during "Installed programs / In progress ...". After clicking "x" in the upper right, I got, "Testing cannot continue unless an application is successfully installed, including a discoverable shortcut and an entry in add / remove programs." I repeated "Windows App Certification Kit", except that this time when the "Modify setup" window appeared, I clicked "repair". It came to that same point and gave the same error. Suggestion? Thanks, Spencer -- https://mail.python.org/mailman/listinfo/python-list
