Re: How can I count word frequency in a web site?

2015-11-30 Thread ryguy7272
On Sunday, November 29, 2015 at 7:49:40 PM UTC-5, ryguy7272 wrote: > I'm trying to figure out how to count words in a web site. Here is a sample > of the link I want to scrape data from and count specific words. > http://finance.yahoo.com/q/h?s=STRP+Headlines > > I only want t

Re: How can I count word frequency in a web site?

2015-11-30 Thread ryguy7272
On Sunday, November 29, 2015 at 9:51:46 PM UTC-5, Laura Creighton wrote: > In a message of Sun, 29 Nov 2015 21:31:49 -0500, Cem Karan writes: > >You might want to look into Beautiful Soup > >(https://pypi.python.org/pypi/beautifulsoup4), which is an HTML > >screen-scraping tool. I've never used

Re: How can I count word frequency in a web site?

2015-11-29 Thread ryguy7272
luck, > Cem Karan > > On Nov 29, 2015, at 7:49 PM, ryguy7272 wrote: > > > I'm trying to figure out how to count words in a web site. Here is a > > sample of the link I want to scrape data from and count specific words. > > http://finance.yahoo.com/q/h?s=STRP+Headlines &

How can I count word frequency in a web site?

2015-11-29 Thread ryguy7272
I'm trying to figure out how to count words in a web site. Here is a sample of the link I want to scrape data from and count specific words. http://finance.yahoo.com/q/h?s=STRP+Headlines I only want to count certain words, like 'fraud', 'lawsuit', etc. I want to have a way to control for

Re: Does Python allow variables to be passed into function for dynamic screen scraping?

2015-11-28 Thread ryguy7272
On Saturday, November 28, 2015 at 5:28:55 PM UTC-5, Laura Creighton wrote: > In a message of Sat, 28 Nov 2015 14:03:10 -0800, ryguy7272 writes: > >I'm looking at this URL. > >https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names > > > >If I hit F12 I can see tags s

Does Python allow variables to be passed into function for dynamic screen scraping?

2015-11-28 Thread ryguy7272
I'm looking at this URL. https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names If I hit F12 I can see tags such as these: https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names; r = requests.get(url) soup=BeautifulSoup(r.content,"lxml") #set up a function to parse the "soup" for each

Re: Does Python allow variables to be passed into function for dynamic screen scraping?

2015-11-28 Thread ryguy7272
On Saturday, November 28, 2015 at 8:59:04 PM UTC-5, Steven D'Aprano wrote: > On Sun, 29 Nov 2015 09:03 am, ryguy7272 wrote: > > > I'm looking at this URL. > > https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names > > Don't screen-scrape Wikipedia. Just don't. T

Re: Screen scraper to get all 'a title' elements

2015-11-25 Thread ryguy7272
t; > HTH, > > -Grobu- > > > On 25/11/15 21:55, MRAB wrote: > > On 2015-11-25 20:42, ryguy7272 wrote: > >> Hello experts. I'm looking at this url: > >> https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names > >> > >>

Re: Screen scraper to get all 'a title' elements

2015-11-25 Thread ryguy7272
On Wednesday, November 25, 2015 at 6:34:00 PM UTC-5, Grobu wrote: > On 25/11/15 23:48, ryguy7272 wrote: > >> re.findall( r'\<a[^>]+title="(.+?)"', html ) > [ ... ] > > Thanks!! Is that regex? Can you explain exactly what it is doing? > > Also, it seems

Re: Screen scraper to get all 'a title' elements

2015-11-25 Thread ryguy7272
On Wednesday, November 25, 2015 at 3:42:21 PM UTC-5, ryguy7272 wrote: > Hello experts. I'm looking at this url: > https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names > > I'm trying to figure out how to list all 'a title' elements. For instance, I > see the following: >

Screen scraper to get all 'a title' elements

2015-11-25 Thread ryguy7272
Hello experts. I'm looking at this url: https://en.wikipedia.org/wiki/Wikipedia:Unusual_place_names I'm trying to figure out how to list all 'a title' elements. For instance, I see the following: Accident Ala-Lemu Alert Apocalypse Peaks So, I tried putting a script together to get 'title'.

How can I export data from a website and write the contents to a text file?

2015-11-18 Thread ryguy7272
I'm trying the script below, and it simple writes the last line to a text file. I want to add a '\n' after each line is written, so I don't overwrite all the lines. from bs4 import BeautifulSoup import urllib2 var_file = urllib2.urlopen("http://www.imdb.com/chart/top;) var_html =

Re: How can I export data from a website and write the contents to a text file?

2015-11-18 Thread ryguy7272
On Wednesday, November 18, 2015 at 11:58:17 AM UTC-5, Chris Angelico wrote: > On Thu, Nov 19, 2015 at 3:37 AM, ryguy7272 <ryanshu...@gmail.com> wrote: > > text_file = open("C:/Users/rshuell001/Desktop/excel/Text1.txt", "wb") > > z = str(

Re: How can I export data from a website and write the contents to a text file?

2015-11-18 Thread ryguy7272
On Wednesday, November 18, 2015 at 12:41:19 PM UTC-5, ryguy7272 wrote: > On Wednesday, November 18, 2015 at 12:21:47 PM UTC-5, Denis McMahon wrote: > > On Wed, 18 Nov 2015 08:37:47 -0800, ryguy7272 wrote: > > > > > I'm trying the script below... > > > > The pr

Re: How can I export data from a website and write the contents to a text file?

2015-11-18 Thread ryguy7272
On Wednesday, November 18, 2015 at 12:21:47 PM UTC-5, Denis McMahon wrote: > On Wed, 18 Nov 2015 08:37:47 -0800, ryguy7272 wrote: > > > I'm trying the script below... > > The problem isn't that you're over-writing the lines (although it may > seem that way to you), the p

Re: How can I export data from a website and write the contents to a text file?

2015-11-18 Thread ryguy7272
On Wednesday, November 18, 2015 at 12:04:16 PM UTC-5, ryguy7272 wrote: > On Wednesday, November 18, 2015 at 11:58:17 AM UTC-5, Chris Angelico wrote: > > On Thu, Nov 19, 2015 at 3:37 AM, ryguy7272 <> wrote: > > > text_file = open("C:/Users/rshuell001/Desktop

Re: Automation of Windows app?

2015-08-30 Thread ryguy7272
On Tuesday, March 31, 2015 at 9:14:38 PM UTC-4, alex23 wrote: On 23/03/2015 1:43 PM, Michael Torrie wrote: As near as I can tell the standard go-to utility for this is a program called AutoIt. https://www.autoitscript.com/site/autoit/ Nothing to do with Python, and its scripting language

Can I download XML data from the web and save, in as CSV or TXT delimitation?

2015-08-19 Thread ryguy7272
I'm trying to get R to download the data from here: http://www.usda.gov/oce/commodity/wasde/report_format/latest-July-2015-New-Format.xml # install and load the necessary package install.packages(XML) library(XML) # Save the URL of the xml file in a variable xml.url -

Re: Can I download XML data from the web and save, in as CSV or TXT delimitation?

2015-08-19 Thread ryguy7272
On Wednesday, August 19, 2015 at 8:21:50 AM UTC-4, Laura Creighton wrote: In a message of Wed, 19 Aug 2015 04:57:44 -0700, ryguy7272 writes: I'm trying to get R to download the data from here: http://www.usda.gov/oce/commodity/wasde/report_format/latest-July-2015-New-Format.xml # install

Re: Can I download XML data from the web and save, in as CSV or TXT delimitation?

2015-08-19 Thread ryguy7272
On Wednesday, August 19, 2015 at 1:14:44 PM UTC-4, Petite Abeille wrote: On Aug 19, 2015, at 7:01 PM, Denis McMahon wrote: Downloading xml from the web is easy writing csv or txt is easy The tricky bit is converting the xml you have into the csv or text data you want.

Re: JSON Object to CSV file

2015-08-19 Thread ryguy7272
On Wednesday, June 17, 2015 at 11:00:24 AM UTC-4, kbtyo wrote: I would like to have this JSON object written out to a CSV file so that the keys are header fields (for each of the columns) and the values are values that are associated with each header field. Is there a best practice for

Re: JSON Object to CSV Question

2015-08-19 Thread ryguy7272
On Thursday, June 18, 2015 at 2:59:11 AM UTC-4, kbtyo wrote: Good Evening Everyone: I would like to have this JSON object written out to a CSV file so that the keys are header fields (for each of the columns) and the values are values that are associated with each header field. Is there

Run Python Script; Nothing Happens

2015-07-29 Thread ryguy7272
I am using Spyder Python 2.7. I'm running this sample code. import numpy as np import numpy.random as npr import matplotlib.pyplot as plt S0 = 100 r = 0.05 sigma = 0.25 T = 30 / 365. I = 1 ST = S0 * np.exp((r - 0.5 * sigma ** 2) * T + sigma * np.sqrt(T) * npr.standard_normal(I)) R_gbm =

Re: How to Calculate NPV?

2015-07-29 Thread ryguy7272
On Wednesday, July 29, 2015 at 10:21:35 AM UTC-4, ryguy7272 wrote: On Wednesday, July 29, 2015 at 9:59:10 AM UTC-4, ryguy7272 wrote: I am using Spyder Python 2.7. I'm running this sample code. import scipy as sp cashflows=[50,40,20,10,50] npv=sp.npv(0.1,cashflows) round(npv,2

Re: How to Calculate NPV?

2015-07-29 Thread ryguy7272
On Wednesday, July 29, 2015 at 9:59:10 AM UTC-4, ryguy7272 wrote: I am using Spyder Python 2.7. I'm running this sample code. import scipy as sp cashflows=[50,40,20,10,50] npv=sp.npv(0.1,cashflows) round(npv,2) Now, I'm trying to get the NPV, and I don't see any obvious way to get

How to Calculate NPV?

2015-07-29 Thread ryguy7272
I am using Spyder Python 2.7. I'm running this sample code. import scipy as sp cashflows=[50,40,20,10,50] npv=sp.npv(0.1,cashflows) round(npv,2) Now, I'm trying to get the NPV, and I don't see any obvious way to get it. The author of the book that I'm reading gets 144.56. I think that's

Re: Run Python Script; Nothing Happens

2015-07-29 Thread ryguy7272
On Wednesday, July 29, 2015 at 9:19:19 AM UTC-4, ryguy7272 wrote: I am using Spyder Python 2.7. I'm running this sample code. import numpy as np import numpy.random as npr import matplotlib.pyplot as plt S0 = 100 r = 0.05 sigma = 0.25 T = 30 / 365. I = 1 ST = S0 * np.exp((r - 0.5

Error: valueError: ordinal must be = 1

2015-07-27 Thread ryguy7272
Hello experts. I'm working in Python Anaconda Spyder. I'm reading a book called 'Python for Finance' and I'm trying to run this sample code: import numpy as np import pandas as pd import pandas.io.data as web sp500 = web.DataReader('^GSPC', data_source='yahoo', start='1/1/2000',

Re: Can I copy/paste Python code?

2015-07-21 Thread ryguy7272
On Monday, July 20, 2015 at 10:50:09 PM UTC-4, ryguy7272 wrote: I'm trying to copy some Python code from a PDF book that I'm reading. I want to test out the code, and I can copy it, but when I paste it into the Shell, everything is all screwed up because of the indentation. Every time I

Re: Is there a way to install ALL Python packages?

2015-07-21 Thread ryguy7272
On Monday, July 20, 2015 at 10:57:47 PM UTC-4, ryguy7272 wrote: I'd like to install ALL Python packages on my machine. Even if it takes up 4-5GB, or more, I'd like to get everything, and then use it when I need it. Now, I'd like to import packages, like numpy and pandas, but nothing

Re: Is there a way to install ALL Python packages?

2015-07-21 Thread ryguy7272
On Monday, July 20, 2015 at 10:57:47 PM UTC-4, ryguy7272 wrote: I'd like to install ALL Python packages on my machine. Even if it takes up 4-5GB, or more, I'd like to get everything, and then use it when I need it. Now, I'd like to import packages, like numpy and pandas, but nothing

Can I copy/paste Python code?

2015-07-20 Thread ryguy7272
I'm trying to copy some Python code from a PDF book that I'm reading. I want to test out the code, and I can copy it, but when I paste it into the Shell, everything is all screwed up because of the indentation. Every time I paste in any kind of code, it seems like everything is immediately

Re: Can't Install Pandas

2015-07-20 Thread ryguy7272
On Sunday, July 19, 2015 at 11:05:46 PM UTC-4, ryguy7272 wrote: Hello experts. I odwnloaded Pandas, and put it here. C:\Python34\Scripts\pandas-0.16.2 Then, I ran this in what most people call the c-prompt, but I call it the 'Python 3.4.3 Shell' C:\Python34\Scripts\pandas-0.16.2 pip

Is there a way to install ALL Python packages?

2015-07-20 Thread ryguy7272
I'd like to install ALL Python packages on my machine. Even if it takes up 4-5GB, or more, I'd like to get everything, and then use it when I need it. Now, I'd like to import packages, like numpy and pandas, but nothing will install. I figure, if I can just install everything, I can simply

Re: Can't Install Pandas

2015-07-19 Thread ryguy7272
On Sunday, July 19, 2015 at 11:05:46 PM UTC-4, ryguy7272 wrote: Hello experts. I odwnloaded Pandas, and put it here. C:\Python34\Scripts\pandas-0.16.2 Then, I ran this in what most people call the c-prompt, but I call it the 'Python 3.4.3 Shell' C:\Python34\Scripts\pandas-0.16.2 pip

Can't Install Pandas

2015-07-19 Thread ryguy7272
Hello experts. I odwnloaded Pandas, and put it here. C:\Python34\Scripts\pandas-0.16.2 Then, I ran this in what most people call the c-prompt, but I call it the 'Python 3.4.3 Shell' C:\Python34\Scripts\pandas-0.16.2 pip install 'setup.py' It seems like everything ran fine, so I try this.

Trying to import numpy

2015-07-06 Thread ryguy7272
I'm trying to use numpy. I get this error: import numpy as np Traceback (most recent call last): File pyshell#1, line 1, in module import numpy as np ImportError: No module named numpy I followed the instructions here. https://pip.pypa.io/en/latest/installing.html In the c-prompt, I

Re: very weird pandas behavior

2014-12-22 Thread ryguy7272
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: I downloaded pandas and put it in my python directory, then, at the C-prompt, I ran this: pip install pandas It looks like everything downloaded and installed fine. Great. Now, in Python Shell, I enter this: import

Re: very weird pandas behavior

2014-12-21 Thread ryguy7272
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: I downloaded pandas and put it in my python directory, then, at the C-prompt, I ran this: pip install pandas It looks like everything downloaded and installed fine. Great. Now, in Python Shell, I enter this: import

Re: very weird pandas behavior

2014-12-21 Thread ryguy7272
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: I downloaded pandas and put it in my python directory, then, at the C-prompt, I ran this: pip install pandas It looks like everything downloaded and installed fine. Great. Now, in Python Shell, I enter this: import

Re: very weird pandas behavior

2014-12-21 Thread ryguy7272
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: I downloaded pandas and put it in my python directory, then, at the C-prompt, I ran this: pip install pandas It looks like everything downloaded and installed fine. Great. Now, in Python Shell, I enter this: import

Re: very weird pandas behavior

2014-12-21 Thread ryguy7272
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: I downloaded pandas and put it in my python directory, then, at the C-prompt, I ran this: pip install pandas It looks like everything downloaded and installed fine. Great. Now, in Python Shell, I enter this: import

Re: very weird pandas behavior

2014-12-21 Thread ryguy7272
On Saturday, December 20, 2014 10:46:40 AM UTC-5, ryguy7272 wrote: I downloaded pandas and put it in my python directory, then, at the C-prompt, I ran this: pip install pandas It looks like everything downloaded and installed fine. Great. Now, in Python Shell, I enter this: import

very weird pandas behavior

2014-12-20 Thread ryguy7272
I downloaded pandas and put it in my python directory, then, at the C-prompt, I ran this: pip install pandas It looks like everything downloaded and installed fine. Great. Now, in Python Shell, I enter this: import pandas as pd I get this error. Traceback (most recent call last): File

import math error

2014-11-16 Thread ryguy7272
When I type 'import math', it seems like my Python recognizes this library. Great. When I try to run the following script, I get an error, which suggests (to me) the math library is not working correctly. Script: import math def main(): print This program finds the real solutions to a

import graphics library; causes error

2014-11-16 Thread ryguy7272
These libraries drive me totally nuts. Sorry, just had to get it out there. Anyway, I open the cmd window, and typed this: 'easy_install python graphics'. So, it starts up and runs/downloads the appropriate library from the web. I get confirmation (in the cmd window) that it finishes, then I

Re: import graphics library; causes error

2014-11-16 Thread ryguy7272
On Sunday, November 16, 2014 3:39:45 PM UTC-5, ryguy7272 wrote: These libraries drive me totally nuts. Sorry, just had to get it out there. Anyway, I open the cmd window, and typed this: 'easy_install python graphics'. So, it starts up and runs/downloads the appropriate library from

Meetup in NYC?

2014-10-25 Thread ryguy7272
Are there any Python professionals in or around NYC who can meetup for an hour or two to help me with a few things? I've been trying to run various Python scripts for a few months now, and I'm not having any success with this stuff at all. Basically, everything built into Python works perfectly

No Error; No Output...Nothing

2014-10-21 Thread ryguy7272
Hey everyone, I'm trying to run this code. import os import pickle #import urllib2 from urllib.request import urlopen #import cookielib import http.cookiejar import re import time import numpy as np #import pylab as pl # symbol - USDEUR=X - problem that the server sometimes returns 0.0 def

Re: No Error; No Output...Nothing

2014-10-21 Thread ryguy7272
On Tuesday, October 21, 2014 5:44:33 PM UTC-4, ryguy7272 wrote: Hey everyone, I'm trying to run this code. import os import pickle #import urllib2 from urllib.request import urlopen #import cookielib import http.cookiejar import re import time import numpy as np

Quick Question About Setting Up Pytz

2014-10-18 Thread ryguy7272
I downloaded PYTZ and put it here. C:\Python27\pytz Now, in the cmd window, I typed this: C:\Python27\pytz\setup.py A text file opens and nothing else happens. I thought it was supposed to install the PYTZ library. What am I doing wrong? --

Question about PANDAS

2014-10-18 Thread ryguy7272
I'm trying to install Pandas. I went to this link. https://pypi.python.org/pypi/pandas/0.14.1/#downloads I downloaded this: pandas-0.14.1.win32-py2.7.exe (md5) I have Python27 installed. So, I run the executable and re-run my Python script and I get the same error as before. Traceback

Re: Quick Question About Setting Up Pytz

2014-10-18 Thread ryguy7272
On Saturday, October 18, 2014 3:55:02 PM UTC-4, ryguy7272 wrote: I downloaded PYTZ and put it here. C:\Python27\pytz Now, in the cmd window, I typed this: C:\Python27\pytz\setup.py A text file opens and nothing else happens. I thought it was supposed to install the PYTZ

How to debug Python IDLE?

2014-10-15 Thread ryguy7272
I'm wondering how to debug code in IDLE Python 3.4. I found this. http://www.cs.uky.edu/~paulp/CS115F11/notes/debug/debug.html That looks pretty helpful, but mine is nothing like that. All my controls are greyed out. The debugger does basically...nothing. All I get is messages like this.

Re: How to debug Python IDLE?

2014-10-15 Thread ryguy7272
On Wednesday, October 15, 2014 3:23:22 PM UTC-4, Terry Reedy wrote: On 10/15/2014 3:09 PM, ryguy7272 wrote: I'm wondering how to debug code in IDLE Python 3.4. I found this. Use 3.4.2, which has an important bugfix for debugger. http://www.cs.uky.edu/~paulp/CS115F11/notes

Re: How to debug Python IDLE?

2014-10-15 Thread ryguy7272
On Wednesday, October 15, 2014 3:10:05 PM UTC-4, ryguy7272 wrote: I'm wondering how to debug code in IDLE Python 3.4. I found this. http://www.cs.uky.edu/~paulp/CS115F11/notes/debug/debug.html That looks pretty helpful, but mine is nothing like that. All my controls are greyed out

Question About Running Python code

2014-10-15 Thread ryguy7272
I'm trying to run this script (using IDLE 3.4) #!/usr/bin/env python import urllib2 import pytz import pandas as pd from bs4 import BeautifulSoup from datetime import datetime from pandas.io.data import DataReader SITE = http://en.wikipedia.org/wiki/List_of_S%26P_500_companies; START =

Import Doesn't Import

2014-10-15 Thread ryguy7272
So sorry everyone. I've posted here several times today. This is VERY frustrating. So, I'm reading this link. https://docs.python.org/2/howto/urllib2.html Fetching URLs The simplest way to use urllib2 is as follows: import urllib2 response = urllib2.urlopen('http://python.org/') html =

Re: Import Doesn't Import

2014-10-15 Thread ryguy7272
On Wednesday, October 15, 2014 8:40:40 PM UTC-4, ryguy7272 wrote: So sorry everyone. I've posted here several times today. This is VERY frustrating. So, I'm reading this link. https://docs.python.org/2/howto/urllib2.html Fetching URLs The simplest way to use urllib2

I'm looking to start a team of developers, quants, and financial experts, to setup and manage an auto-trading-money-making-machine

2014-10-14 Thread ryguy7272
I'm looking to start a team of developers, quants, and financial experts, to setup and manage an auto-trading-money-making-machine #1) Setup a VM; must be a Windows machine (maybe Azure) #2) Load configure the software (Matlab, Python, R, Excel, and SQL Server) #3) Develop and test code (Matlab

Re: I'm looking to start a team of developers, quants, and financial experts, to setup and manage an auto-trading-money-making-machine

2014-10-14 Thread ryguy7272
On Tuesday, October 14, 2014 9:38:55 AM UTC-4, Johann Hibschman wrote: Marko Rauhamaa ma...@pacujo.net writes: ryguy7272 ryanshu...@gmail.com: I'm looking to start a team of developers, quants, and financial experts, to setup and manage an auto-trading-money-making-machine

Is there an easy way to control indents in Python

2014-10-14 Thread ryguy7272
I'm just learning Python. It seems like indents are EXTREMELY important. I guess, since there are no brackets, everything is controlled by indents. Well, I'm reading a couple books on Python now, and in almost all of the examples they don't have proper indents, so when I copy/paste the code

Re: How to install and run a script?

2014-10-12 Thread ryguy7272
On Sunday, October 12, 2014 4:18:19 PM UTC-4, ryguy7272 wrote: I'm an absolute noob to Python, although I have been programming in several other languages for over 10 years. I'm trying to install and run some scripts, and I'm not having much success. I followed the steps

problem with pytz

2014-10-12 Thread ryguy7272
I just tried running the code from here. http://www.thealgoengineer.com/2014/download_sp500_data/ I got this error. Traceback (most recent call last): File C:/Python27/stock_data.py, line 4, in module import pytz ImportError: No module named pytz So, I got here and download the files.