Re: [Imdbpy-help] Timing SQLITE creation
On Jan 13, Mike Castle dalg...@gmail.com wrote: I also just tested with PRAGMA journal_mode = OFF; Good - I'll update the code and documentation ASAP (and submit it to you, to check that I've understood everything). But, are you doing something at the beginning with preserving current ids? Not exactly (actually, at least); the only information saved and restored between two runs are imdbID (collected when IMDbPY have to retrieve from the web the real imdbID for a movie/person/..., and stored in the database for future faster accesses). They are not matched against the 'id' columns of their respective tables, but on the whole set of information (title, imdbIndex, year, ...) and so this would not a problem. Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- This SF.net email is sponsored by: SourcForge Community SourceForge wants to tell your story. http://p.sf.net/sfu/sf-spreadtheword ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] BUG: 4.0-dev - Newline Missing after Genres and incorrect plot summary
On Feb 22, Chris Thompson chris.thomp...@stormfolio.net wrote: a) the output of get_first_movie appears to be missing a newline at the end of the genres and before Director: b) the plot summary is incorrect Hi! Thank you very much for your bug reports - I've fixed the Movie.summary method (it wasn't updated after some recent changes to IMDbPY). There are some warnings, maybe this is related? No, you don't have the lxml package installed, and so the html is parsed using BeautifulSoup (pure python). It's just a warning and not a real problem (except that everything will be a lot slower). I'm still trying to understand what can cause the problem you reported in your other email. Thank you again - I've added your name to the credits. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy and imdb licences/access terms
On Mar 02, Andre Eliatamby andre.eliata...@gmail.com wrote: What I would like to know is: - are my assumptions above correct? Mostly; see also the other recent (and old) threads in imdbpy-devel. - Does imdbpy have a special license from imdb to provide this package? No. I honestly can't even remember if I ever asked (maybe, maybe not: I started the project in 2004). For sure at some point (years ago) I questioned the developers of other related libraries; many (most) never asked, some had tried. The results can be summarized in: - no reply at all. - no, you can't: buy this (license for X thousands of data, missing the point of the request). - yes you can (very few). As far as I can remember there was a discussion about this kind of software in debian-devel, and their opinion was that they are ok for inclusion in Debian (it goes without saying that they are more interested in the license of the code, so this may not be conclusive). The fact is (as usual: I'm not a lawyer) that similar terms _as far as I know_ aren't clearly enforceable (I'm not sure about the USA laws, however). IMDbPY fetches the data the same way your browser (or your cache, or your crap-filtering proxy) does. After that, _for your own usage_ I'm pretty sure you can do whatever you like (your how usage completely excludes redistribution and making any kind of profit, no matter how indirect, that's for sure). Moreover: keep in mind that IMDbPY is not (and never will be) tailored as a mass-scraper bot: it can handle only single (and serial) requests, and for the way it's structured using it on the whole database is impossible. In fact, it's as nice as it can be with the IMDb server. - Do I need to get special permission for imdb to actually use this package (or at least the query functions)? You can use IMDbPY to access the plain text data files (putting them in a SQL database: see README.sqldb). If you plan to use 'http'/'mobile', obviously a permission won't hurt, but as long as you use it for your own personal non-commercial usage, I can't see any problem. It goes without saying that if you plan to create a public service with it (or make money out of it in any other way), you can stop right now. :-) Again: as said other times, I'm genuinely convinced that using it non-commercially is legal, but... I'm not a lawyer. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDbPY 4.0 and IMDbPYKit 1.0
On Mar 13, Jean-Yves Avenard jyaven...@gmail.com wrote: Doesn't build on any of my machines (Ubuntu 8.10 and CentOS 5.3) :( The kind of things you don't want to hear 12 hours after a release. ;-) imdb/parser/common/cutils.c:63:20: error: Python.h: No such file or directory Looks like you're missing the python-dev package, used to compile an (optional, but suggested) C module. Let me know if it solves the problem. As an extreme measure, you can add the --without-cutils command line argument to exclude the cutils.c module, but... Bye, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Get a movie's keyword
On Mar 15, Friedrich Weber f...@samurai-x.org wrote:
Hi all!
Hi!
is it possible to obtain a movie's keywords
(just a list of the keywords listed at, for example,
http://www.imdb.com/title/tt0087544/keywords)? I couldn't find
anything about this in the sources, but maybe I am overlooking
something :)
Maybe the documentation? ;-)
See README.package, especially the 'INFORMATION SETS' section.
Example:
from imdb import IMDb
i = IMDb('http')
nausicaa = i.get_movie('0087544')
# The update method is the key for what you need.
i.update(nausicaa, 'keywords')
print nausicaa['keywords']
The same information is available in every data access system ('sql',
'local', 'mobile', ...)
By the way, if you really need _only_ these information and you
already have the movieID (maybe from a i.search_movie call), you can
save some time/bandwidth asking only for these information:
from imdb import IMDb
i = IMDb('http')
nausicaa = i.get_movie('0087544', info='keywords')
# Here the nausicaa Movie instance doesn't contains many information:
# the movie doesn't even have a title!
print nausicaa['keywords']
HTH,
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://erlug.linux.it/~da/
--
Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are
powering Web 2.0 with engaging, cross-platform capabilities. Quickly and
easily build your RIAs with Flex Builder, the Eclipse(TM)based development
software that enables intelligent coding and step-through debugging.
Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] MPAA Ratings
On Mar 15, Chris Spencer chriss...@gmail.com wrote: I had looked for the key in Movie.py and get_movie_infoset() in __init__.py, but I didn't see mpaa. There isn't, because it's not a separated info set: it's one of the keys you get, with the 'main' info set (the one with 'cast', 'director' and so on). It's not guaranteed that retrieving an info set will add _a single_ key to the instance: there can be more than one key in the returned information. Bye! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] BUG: 4.0-dev - Newline Missing after Genres and incorrect plot summary
On Mar 25, H. Turgut Uyar u...@itu.edu.tr wrote: I was browsing the archives and noticed that this was not answered. Completely missed that part. :-) The 1.3.6 version of python-lxml does not seem to have an html module. It's pretty old, it was released in October 2007. Maybe we should set the dependency info in the setup.py file so that a release with version greater than 2.1 will be selected. For sure it won't hurt. :-) PS: in a matter of days, I hope to add top 250/bottom 100 links to imdbpykit. Whata about i18n? I see a file for English and one for Turkish; I can add Italian. Is there an easy way to translate the XML generated by the XSL file? Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] With importing to MySQL I get these and other warnings and further.
On Apr 13, Atul Kulkarni atulskulka...@gmail.com wrote:
/usr/bin/imdbpy2sql.py:1278: Warning: Incorrect string value: '\xC2\x92s
ch...' for column 'info' at row 1164
/usr/bin/imdbpy2sql.py:1006: Warning: Incorrect string value: '\xC2\x80$'
for column 'title' at row 35842
Will they have any impact on my query?
I can't tell, since I can't figure the strange errors you got.
Or better: looks like a problem with garbage in the plain text
data files or a misconfiguration about charset/collation of your
MySQL, but... who knows? That chars don't make much sense. :-/
I am getting some errors like
[...]
TypeError: int() argument must be a string or a number, not 'NoneType'
That's really bad and should never happen.
It's as if the 'id' column in the 'title' table contains some
NULL value, but this can't be.
If possible, you can try this:
- update to 4.0 (I guess you're using 3.9 or older).
- run imdbpy2sql.py appending this:
21 | tee OUTPUT.txt
so that you can record its output in OUTPUT.txt.
- show me the code you're running (or a short example useful to
reproduce the problem), if possible using movie titles and not
movieIDs.
I.e.: i.get_movie(1234) is not that useful to debug the sql
data access system; it's much better:
movie = i.search_movie('The Movie Title')[0]
i.update(movie)
Thank you very much!
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://erlug.linux.it/~da/
--
This SF.net email is sponsored by:
High Quality Requirements in a Collaborative Environment.
Download a free trial of Rational Requirements Composer Now!
http://p.sf.net/sfu/www-ibm-com
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Upcoming US Air Times.
On Apr 19, Andre LeBlanc andreplebl...@gmail.com wrote:
Thats actually not the data that I am looking for. I need the date
AND TIME that the episode is airing, not the original air date.
on the page that I linked to you can see that 13x07 is airing at
10PM on wednesday.
Hmmm... no: no sign of it in my page. :-)
You can see what I got here:
http://imdbpy.sourceforge.net/tt0121955_episodes.html
Are you from the United States? I suppose IMDb customize the
pages based on the country of the viewer.
By the way, remember that IMDbPY accesses the IMDb server using its own
user (it sends its own cookies) and uses the 'akas' web server; i.e.:
http://akas.imdb.com/title/tt0121955/tvschedule
but when I do this:
i.update(ep, 'airing') ep['airing']
I get a KeyError, there is no 'airing' info for that episode.
I see... this is due to the fact that, for episodes, the table
containing the 'tv schedule' is different. :-/
You can look at:
http://akas.imdb.com/title/tt0121955/tvschedule
http://akas.imdb.com/title/tt1397945/tvschedule
respectively for the series and the episode.
So you are right; using the previous example, this won't work:
import imdb
i = imdb.IMDb('http') # or 'mobile'
m = i.get_movie('0121955')
i.update(m, 'episodes')
episode = m['episodes'][13][7]
i.update(episode, 'airing')
print episode['airing']
I'll see if it's possible to support both formats, but it could
take some time.
In the meantime, maybe you can parse the list of aired episodes
(taking it from the series) and look specifically at the episode
you're interested in.
Thanks for the bug report, I'll add your name to the credits. :-)
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://erlug.linux.it/~da/
--
Stay on top of everything new and different, both inside and
around Java (TM) technology - register by April 22, and save
$200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco.
300 plus technical and hands-on sessions. Register today.
Use priority code J9JMT32. http://p.sf.net/sfu/p
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] problems with the plain text data files
On Apr 21, Davide Alberani davide.alber...@gmail.com wrote: So, let's wait: usually they reply in a short time. Aaargh! I didn't notice it before; from: http://imdb.com/help/search?domain=helpdesk_faqindex=1file=titleformat Articles now remain where they occur in the title. This was a policy change in April 2009. Can I say it? A very awful choice. One thing is to _diplay_ the title in The Title format on the web page, but storing them losing the notion of what the article is, looks really bad to me. :-( This will affect 'http' and 'mobile', too. Probably nothing major (the code to handle both formats is already present, after all), but it will require some time to fix it everywhere. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Stay on top of everything new and different, both inside and around Java (TM) technology - register by April 22, and save $200 on the JavaOne (SM) conference, June 2-5, 2009, San Francisco. 300 plus technical and hands-on sessions. Register today. Use priority code J9JMT32. http://p.sf.net/sfu/p ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] RuntimeError: dictionary changed size during iteration
On May 01, Andre LeBlanc andreplebl...@gmail.com wrote: I got this error using the latest SVN, seems to only affect that one movie but I haven't tried many. [...] RuntimeError: dictionary changed size during iteration My fault: I made a bad assumption iterating over a dictionary. :-/ It's fixed in the SVN. Thank you very much for the bug report. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] installer problems (WAS: IMDbPY 4.1 and IMDbPYKit 1.1)
On May 03, Davide Alberani davide.alber...@gmail.com wrote: There are problems with the installer of both IMDbPY and IMDbPYKit. IMDbPYKit 1.1.1 was released today to fix the problem. As said, IMDbPY 4.1 localization won't work out-of-the-box. You can use the SVN version, or follow these simple steps, courtesy of H. Turgut Uyar: -- cd PATH-TO-YOUR/site-packages mv IMDbPY-4.1-py2.5-linux-i686.egg imdbpy.zip mkdir IMDbPY-4.1-py2.5-linux-i686.egg cd IMDbPY-4.1-py2.5-linux-i686.egg/ unzip ../imdbpy.zip rm ../imdbpy.zip cd imdb/locale/ cp PATH-TO-IMDBPY-SOURCES/imdb/locale/*po . python rebuildmo.py -- Pardon for all the troubles, -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Register Now Save for Velocity, the Web Performance Operations Conference from O'Reilly Media. Velocity features a full day of expert-led, hands-on workshops and two days of sessions from industry leaders in dedicated Performance Operations tracks. Use code vel09scf and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Cast - lead or supporting role?
On May 04, Roman Gaufman hacke...@gmail.com wrote:
Is there no way to get a list of just actors mentioned on the main
page for the movie?
If you're fetching the data from the web, you can use other
data access systems (beyond the default http - which uses the
combined page as its main source of information) to fetch
less data: both mobile and httpThin uses the maindetails
page.
httpThin uses the same parser of http (it just fetches the
main details page) while mobile is a different parser, tailored
for systems with little CPU/bandwidth resources.
E.g.:
from imdb import IMDb
ia = IMDb('httpThin') # or mobile
movie = ia.search_movie('12 Angry Men')[0] # gets the first result.
ia.update(movie) # updates the information.
print len(movie['cast']) # will print 12, and not 16 as http.
It goes without saying that some information present in combined
are not present in maindetails, so it's up to you to decide if
you need them or not.
HTH,
--
Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47]
http://erlug.linux.it/~da/
--
Register Now Save for Velocity, the Web Performance Operations
Conference from O'Reilly Media. Velocity features a full day of
expert-led, hands-on workshops and two days of sessions from industry
leaders in dedicated Performance Operations tracks. Use code vel09scf
and Save an extra 15% before 5/3. http://p.sf.net/sfu/velocityconf
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Data too long for column 'phonetic_code' at row 88
On May 10, Davide Alberani davide.alber...@gmail.com wrote: In the next days, I'll try with MySQL 5.1 and the latest IMDb data. Everything is fine for me, after a fast test with a partial set of data (5% of every plain text data file) and MySQL 5.1.34 with MyISAM tables (standard configuration from Debian/sid). I'm almost sure there is a problem in your configuration. HTH, -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- The NEW KODAK i700 Series Scanners deliver under ANY circumstances! Your production scanning environment may not be a perfect world - but thanks to Kodak, there's a perfect scanner to get the job done! With the NEW KODAK i700 Series Scanner you'll get full speed at 300 dpi even with all image processing features enabled. http://p.sf.net/sfu/kodak-com ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql.py, PostgreSQL, UTF8, invalid byte sequence
On Oct 06, Vitaly Pashkov ad...@fluda.net wrote:
You can get my copy here: http://fluda.net/personal/movies.list.gz
It's identical to the one I have.
imdb= select id, title from title order by random() limit 5;
id| title
-+-
1147146 | (#3.11)
1010589 | (2007-05-21)
The above ones are not symptoms of problems: many episodes of
tv series are identified only by their #SEASON.EPISODE number
or by their airing date (when a title is missing, obviously).
Got it!
[...]
Auf der grünen Wiese
Not exactly the nasty title I expected - whatever it means. :-)
There's nothing wrong with it, and there are many more umlauts
in that list - this is probably the first occurrence that creates
problems when the data are flushed to the database (the data are
temporarily stored in a Python dictionary, and so they are not
flushed to the db in order).
/usr/lib/python2.5/site-packages/IMDbPY-4.2-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py:628:
UnicodeWarning: Unicode equal comparison failed to convert both
arguments to Unicode - interpreting them as being unequal
if ts[-1].lower() in _articles:
Try this simple script, to see if it creates any problem:
#!/usr/bin/env python
import imdb
utf8_title = 'Auf der gr\xc3\xbcnen Wiese'
print utf8_title in imdb.utils._articles
If it replies False without raising warnings of exceptions,
I think the problem is not in Python itself but in the psycopg2
module or in the configuration of PostgreSQL.
BUT! I also tried to run it without my modification at line 1427 and with
modified title_soundex. It stopped at the other line:
[...]
/usr/lib/python2.5/site-packages/IMDbPY-4.2-py2.5.egg/EGG-INFO/scripts/imdbpy2sql.py,
line 1030, in _runCommand
CURS.executemany(self.sqlstr, self.converter(dataList))
psycopg2.DataError: byte sequence invalid for encoding UTF8: 0xc333
HINT: This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by client_encoding.
It's more or less expected: the data is processed without glitches,
but everything explodes when CURS.executemany tries to dump it into
the database.
To summarize how imdbpy2sql.py works: it reads the plain text data
files (which are mostly in iso-8859-1 encoding), convert them to
utf-8 for internal usage (for a series of more or less good reasons)
and uses a cursor provided by the db access module (psycopg2, in this
case) to store them (again, passing the strings as utf-8).
Your change forces imdbpy2sql.py to use _unicode_ representation of
titles; the UnicodeWarning you get is because it compares a unicode
string to a list of utf-8 encoded strings (imdb.utils._articles).
As a temporary solution you can convert _articles to a list of unicodes,
but I can't consider this a real fix. Put this line somewhere at the
top of the script - hoping it will not break something else ;-) :
_articles = [x.decode('utf8') for x in _articles]
Why psycopg2 or your PostgreSQL don't play nicely with utf-8 strings
is beyond my imagination. :-)
Another small test (this _could_ exclude psycopg2, even if it can
always be a matter of how it's initialized by SQLObject/SQLAlchemy):
===
#!/usr/bin/env python
import psycopg2
utf8_title = 'Auf der gr\xc3\xbcnen Wiese'
connection = psycopg2.connect(database='imdb', user='UNAME', password='PWD')
curs = connection.cursor()
curs.execute(INSERT INTO title (title, kind_id) VALUES (%s, 1);,
(utf8_title,))
connection.commit()
===
In my installation, it works.
As you can see, debugging these strange interactions between
charsets/encodings and various modules, libraries and database
engines is a real pain. :-(
Thanks for your time!
--
Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47]
http://erlug.linux.it/~da/
--
Come build with us! The BlackBerryreg; Developer Conference in SF, CA
is the only developer event you need to attend this year. Jumpstart your
developing skills, take BlackBerry mobile applications to market and stay
ahead of the curve. Join us from November 9#45;12, 2009. Register now#33;
http://p.sf.net/sfu/devconf
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] changes to imdb.com
On Nov 17, H. Turgut Uyar u...@itu.edu.tr wrote: Yes, it returns this: Good - even if it's making me crazy. ;-) You get what I see on my local system. Please, can you try it again in a hour or so (or tomorrow) and see if it's still ok (I've made some last-minute changes to the code and the account)? And if possible, can you try to install it (I tried it with Python 2.6, and it works, on my system)? I've tested it in clean virtual environments (2.6) both with lxml and without lxml and got the same result. It's the same on my system (Python 2.5) where it works. On a remote system, where IMDbPY is updated from the SVN and installed locally (on a clean directory), it still doesn't work. :-( The funny thing is that both my system and the remote one fetch pages that are served for the IMDbPYweb user account (I can see it in the HTML), and so I can't figure out why the HTML served to the remote system contains the sortable lists. It should, since I've set up the IMDbPYweb IMDb account to _not_ use the sortable lists, showing information about people. Do I need to set something in imdbpy.cfg for this or is it already handled by the code? No - it should be fine without any changes. I suspect there's something veeery wrong somewhere - but I can't find what - keep in mind that the remote system I'm talking about is clean, with no transparent proxies or strange things like these, and hosted on a more-than-trusted provider. Thanks! -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.3
IMDbPY 4.3 is out, to fix the last problems with some changes to the IMDb website. There're also some cool new features, like the ability to guess the language of a movie title. As usual, it can be downloaded from here: http://imdbpy.sf.net/ Since IMDb is introducing a new style for page about persons, soon we'll change the parsers to support it. -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Ending with error code 17 after traceback.
On Nov 19, jaw knee johnn...@gmail.com wrote: joh...@ears:/750/Code/Projects/MySQL/IMDB$ sudo imdbpy2sql.py -o sqlobject -d /750/Code/Projects/MySQL/IMDB/ -u mysql://root:passw...@localhost/imdb -c . [...] raise InternalError(ErrorMessage(e)) sqlobject.dberrors.InternalError: Can't create/write to file '/750/tmp/mysql/#sql_46a1_0.MYI' (Errcode: 17) While I'm a little confused by the fact that the error seems to be generated by SQLObject, it looks like this is a permission problem on your filesystem. Do you have read/write access to /750/tmp/mysql/ and to your working directory (for the -c . argument)? Does your MySQL works correctly (e.g., using phpmyadmin)? HTH, -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] get_imdbMovieId implementation ?
On Nov 26, Jonas Geiregat jo...@geiregat.org wrote:
When trying out the method get_imdbMovieId() it doesn't return an
object. It does return the same string I entered as param.
ai.get_imdbMovieID(Snatch)
'Snatch'
Is this method not implemented yet ?
It is, but it serves another purpose. :-)
See documentation about it in README.package, README.devel and
in the code itself.
Basically, it takes a movieID specific to the used data access
system (i.e. http, httpThin, mobile or sql) and convert
it to the unique imdbID used on the imdb.com web site.
For http/httpThin and mobile, it does nothing since they
use the same ID (i.e., the movieID _is_ the imdbID).
But sql internally uses IDs that are valid only locally (to your
installation); this means that a query to the imdb.com site must
be issued, to convert it to the real imdbID (beware that it can
return None).
Generally speaking, it's better to use the get_imdbID method, which
takes a Movie/Person/Character/Company instance, and not a
movieID/personID/characterID/companyID.
E.g.:
from imdb import IMDb
ia = IMDb('sq', uri='mysql://USER:p...@localhost/imdb') # my local DB.
tu = ia.search_movie(u'The Untouchables (1987)')[0] # First item.
print tu.movieID # will print 558195, valid only for my database.
print ia.get_imdbID(tu) # will print '0094226', that is the ID used
# by imdb.com
# The same thing, with the less flexible/smart get_imdbMovieId:
print ia.get_imdbMovieId(tu.movieID)
HTH,
--
Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47]
http://erlug.linux.it/~da/
--
Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day
trial. Simplify your report design, integration and deployment - and focus on
what you do best, core application coding. Discover what's new with
Crystal Reports now. http://p.sf.net/sfu/bobj-july
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.5
Released in a little rush, here it is 4.5: http://imdbpy.sourceforge.net/ This fixes (I really do hope...) the recent issues with the imdb.com web server and contains other fixes and improvements (most notably, the ability to decouple the creation of CSV files from their insertion in a SQL database, using the imdbpy2sql.py script). As usual, report any bug and share your ideas for improvements! Enjoy, -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPY 4.5
On Feb 28, Davide Alberani davide.alber...@gmail.com wrote: Released in a little rush, here it is 4.5: http://imdbpy.sourceforge.net/ You may have noticed that 4.5.1 is already out. :-) -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdb column questions
On Mar 19, Michael Liu mikel...@gmail.com wrote:
So in short, if i understand it all correctly, the IMDB has in
their own database: the real IMDB id, whether something is adult,
whether something is on Amazon/Blockbuster.
More or less: the imdbIDs are used only on the site, and
amazon/blockbuster information are not in the plain text data files.
If a movie is for mature viewers, you can guess it searching Adult
in their genre, and this can be done in 'sql', too (in fact, using
the do_adult_search method you can exclude adult movies from your
movie searches - if you already have a Movie instance, all you
need is a simple 'Adult' in movie.get('genres') chech)
However, none of this is shared by them, and the only way to get any
of them is to use a script to do a title search and scrape the pages?
You don't need to write it: IMDbPY already does everything you need.
Just use something like:
from imdb import IMDb
ia = IMDb('http') # or 'mobile' for a faster but less complete parser.
... here you do your searches and you fetch movies, persons, ...
In the example above, the movieIDs you'll use will be _the_ imdbIDs;
no need to mention that now you're NOT accessing your local database. :-)
There can be a little mess if you start mixing information from the
web with info from the sql database, but that's all.
Oh, by the way, information about products on Amazon can also be
found in 'sales' and 'dvd' info sets, I think (see README.package,
section 'INFORMATION SETS') and maybe other pages, I don't recall
for sure.
Bye,
--
Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47]
http://www.mimante.net/
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY: build objects from XML dumps
I love Mercurial. :-)
Have fun with this fork (soon it will be merged):
http://bitbucket.org/alberanid/imdbpy_import_xml/overview/
New feature: since some time IMDbPY can dump its object into
XML strings/files; now you can use this XML to recreated IMDbPY
objects (instances of classes Movie, Person, Character and Company).
I've not tested it extensively, and for sure it will fail here
and there (and the code is not exactly nice to look at).
You can test it this way (no output is a good sign ;-):
from imdb import IMDb
from imdb.helpers import parseXML
ia = IMDb('http')
matrix = ia.get_movie('0133093')
xml = matrix.asXML()
matrix_rebuilt = parseXML(xml)
for key in sorted(matrix.keys()):
origValue = matrix[key]
rebuiltValue = matrix_rebuilt.get(key)
if origValue != rebuiltValue:
print 'ERROR', key
print origValue
print rebuiltValue
print ''
--
Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47]
http://www.mimante.net/
--
Download Intel#174; Parallel Studio Eval
Try the new software tools for yourself. Speed compiling, find bugs
proactively, and fine-tune applications for parallel performance.
See why Intel Parallel Studio got high marks during beta.
http://p.sf.net/sfu/intel-sw-dev
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Retrieving a List of Movies in a Given Year
On Mar 27, Chris Spencer chriss...@gmail.com wrote: How would you use IMDBPy to retrieve a list of movies within a given year? It's not possible, at this time (and I'm not sure it's a feature I want to add). Anyway, if you're using the plain text data files (see README.sqldb, otherwise), you can scan your database for the 'production_year' column in the 'title' table, after that you can get the 'id's and use IMDbPY to retrieve all the information about these movies. HTH, -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql.py and 8-bit bytestrings
On May 13, Dustin Wyatt dustin.wy...@gmail.com wrote: Hi! imdbpy2sql.py -d C:\test\imdb_db\text_files -u sqlite:///C:\test\imdb_db\db.db --sqlite-transactions -o sqlalchemy I think it's a better idea to use the UNIX path separator (/), even on Windows. See the 'Microsoft Windows paths' section in the README.sqldb file of the documentation (one of these days I'll fix every troubles with the Windows' path separator...) CURS.executemany(self.sqlstr, self.converter(dataList)) sqlite3.ProgrammingError: You must not use 8-bit bytestrings unless you use a te xt_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode str ings. I see - that's a new feature introduced in the SQLite module shipped with Python 2.6. Thank you very much for the report: I wasn't aware of it: I've already fixed it in the Mercurial repository: http://imdbpy.sf.net/?page=download#hg By they way, I keep noticing other problems with SQLite and SQLAlchemy. I see a 'cannot commit - no transaction is active' error trying to executing the 'COMMIT;' statement (--sqlite-transactions option). The script goes on but... who know what's going on. Anyway, thank you again for the bug report: I've added your name to the credits. Bye! -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] a person photo
On Jun 15, Karim Ihab ki...@connectmetv.com wrote:
Dear all,
Wrong address: it should go to imdbpy-help@lists.sourceforge.net :-)
But it didn't include any photos or posters or any thing that I can use can
u help!
Person instances (can) have the 'headshot' keyword:
person.get('headshot')
Will give you
'http://ia.media-imdb.com/images/M/MV5BMTQzNjU3MDczN15BMl5BanBnXkFtZTYwNzY2Njc4._V1._SX94_SY140_.jpg'
You can get a larger version of the image with a function in
the 'helpers' module. E.g.:
import imdb.helpers
print imdb.helpers.fullSizeCoverURL(person)
It works for Movie instances, too.
Bye!
--
Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47]
http://www.mimante.net/
--
ThinkGeek and WIRED's GeekDad team up for the Ultimate
GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the
lucky parental unit. See the prize list and enter to win:
http://p.sf.net/sfu/thinkgeek-promo
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.6 released
IMDbPY 4.6 is available: http://imdbpy.sf.net/ In this release, a lot of minor fixes and some overall improvements. Sorry for the long delay - we really need a release manager. :-) Have fun! -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] TV series with unknown episode numbers
On Jun 27, Mike Castle dalg...@gmail.com wrote:
I assume that the bit that processes XML or whatever would be the
ideal place to have a counter to keep track of it?
Sure, or - even better - rethink the data structure: after all,
maybe we don't really need a dictionary of dictionaries to store
seasons/episodes.
The current choice was probably made to accomodate things like
unknown season or episode 0, but maybe we don't need these
pieces of information and move to a list of lists or a dictionary
of lists.
But I was busy and didn't have time to figure out your entire
code base. :-
Feel free to poke around, clone/fork the repository, and ask if
you need help!
For a personal fix, I ended up using just an integer
[...]
I've not yet looked at yours, though.
It does the same. :-)
I suppose on of the problems with this is that there is no good
way to guarantee an order.
That's why there are the sortedSeasons and sortedEpisodes
functions in the 'helpers' module (but they can't guarantee
to return the same order you've read on the web page).
Hmmm... I've to think more carefully about it, but right now
I'm inclined to use a dictionary of lists (keeping the 'season'
a key in a dictionary - when possible an integer - and the list
of episodes as its values).
E.g., for movie['episodes']:
{1: [Movie1.1, Movie1.2, ...],
2: [Movie2.1, Movie2.2, ...],
'unknown': [MovieX.1, MovieX.2, ...]}
Where MovieX.Y obviously are Movie instance, like now.
What do you think?
Thanks for your help!
--
Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47]
http://www.mimante.net/
--
This SF.net email is sponsored by Sprint
What will you do first with EVO, the first 4G phone?
Visit sprint.com/first -- http://p.sf.net/sfu/sprint-com-first
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Better imdbID support for imdbpy2sql
On Wed, Jul 28, 2010 at 8:41 PM, Alexmipego alexmip...@gmail.com wrote: For the project I've in mind I really need to have as many imdbid values mapped as possible. During research, and checking the raw files myself, I found that many people ask for it but it's kinda impossible for imdbpy2sql to do better than it does at guessing ids. More or less. :-) The basic problem is that the imdbIDs are not distributed in the plain text data files. My solution is based on the fact that searching imdb for the raw names (in the movies.list file) returns an exact match almost aways. That means, overtime, some applications will end up getting the true id of a movie but there is no way for imdbpy2sql/database to recover the original raw title. I'm not sure to have understood your point. What's the advantage of the MD5 sum, over the normal title? I mean: if the title changes, also its MD5 will change and you will not be able to find the imdbID. When changes in titles, new titles, etc... would occur it would simply fail gracefully and over time those new hash-imdbid codes could be made available. Well, it may works and it's easy to implement, but it means that you need a central repository for this hash table. Since I (as IMDbPY) don't want to provide it, most of the users will use none or create their own. By the way it's not clear to me when you want to ask the hash table for an imdbID: when the imdbpy2sql.py script runs (but this will have a heavy impact on performances, I fear) or when a single item (movie, person, character or company) is requested. Let me know what you think. The changes to support a MD5 column are just 2-3 lines iirc and it shouldn't cause any problems to anyone, yet it would allow for this type of feature to be implemented even if outside the imdbpy code base Yup - I see your point on this, and I'll take it on consideration. -- Davide Alberani d...@mimante.net [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- The Palm PDK Hot Apps Program offers developers who use the Plug-In Development Kit to bring their C/C++ apps to Palm for a share of $1 Million in cash or HP Products. Visit us here for more details: http://p.sf.net/sfu/dev2dev-palm ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Better imdbID support for imdbpy2sql
On Jul 30, Alexmipego alexmip...@gmail.com wrote: Once the MD5 column was present on the distribution everyone would be compatible with my solution. You ask about when and where this matching would be done, and the beauty of this (for you) is that it can be done completely outside imdbpy2sql. I admit you've more or less persuaded me. ;-) If I'll have time, this week or the next one, I'll look at what can be done. Btw, I initially thought that I could simply assume the ID you assign to each title is sequential, so I could simply assume the first line in the CSV was ID 1 and the last would simply match. However, I found out that the final sql table has more rows than the raw file and that means something is either wrong or some extra processing is done. That's due to the fact that not every file in the set of the plain text data files is updated at the same time, and so there can be incongruences and even errors here and there. So, every time imdbpy2sql is asked to handle a movie title (or a person or company name, for instance), it checks if it's already known or create it otherwise. This means that imdbpy2sql will run nicely even if the movies.list.gz file is removed. Bye, -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- The Palm PDK Hot Apps Program offers developers who use the Plug-In Development Kit to bring their C/C++ apps to Palm for a share of $1 Million in cash or HP Products. Visit us here for more details: http://p.sf.net/sfu/dev2dev-palm ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDb redesign: call for help
On Sep 19, Davide Alberani davide.alber...@gmail.com wrote: So... is anyone out there willing to help and be in charge of one or more parsers? I forgot to mention how I arranged the development of the new parsers: the old account (automatically used by IMDbPY) was changed to use the old set of web pages (mostly: the ones about people still needs to be fixed), so it can't be used to develop the new parsers. I've then created a new fork of IMDbPY on bitbucket, which uses a new account set to refer to the new web pages; this repository can be clone by here: http://bitbucket.org/alberanid/imdbpy_parsers2010/ Once you have cloned this repository, you can install this version on your system (or in a virtualenv) and modify it to fix the parsers. You can test each page as you wish; there's also a more comprehensive (well, more or less...) set of tests: http://bitbucket.org/alberanid/imdbpy-testsuite Specifically in the http-mobile directory. The steps: - download from http://erlug.linux.it/~da/erlugtmp/imdbpy_p.tar.gz a more-or-less correct set of .p files (dumps of IMDbPY objects taken when the parsers were in a good state) and untar it in the http-mobile directory. - fetch the new .html web pages with ./test_parser.py -f - run the tests with ./test_parser.py -t 21 | less - spot a problem (missing information or something like that), change the parsers and re-run the tests until the problem is not fixed. :-) In the 'standalone/' directory there is a separate test for each file (the ones labeled *lxml* are faster than the *bsoup* ones. Keep in mind that it's normal to see errors about things like changes in the number of votes, or new cast/companies informations; what really matters is that the parser - from one run to the other - doesn't lose complete sets of information (and that no crap ends up in the strings, movie titles and so on). If a key is completely missing the test_parser.py script will report it in the lists of key that are only in the expected or in the received information. If this was not clear enough, feel free to ask me anything! -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDb redesign: call for help
On Sun, Sep 26, 2010 at 4:31 PM, H. Turgut Uyar u...@itu.edu.tr wrote: I'll try to help. I have quite lot of work these days but I'll get to the parsers as soon as I can. As usual, thank you! :-) I hope to have time to check to at least the main problems about people's pages within this week. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy 4.6 can't fetch person head shot (and who add IMDb to the names ?)
On Tue, Sep 28, 2010 at 10:50 PM, Israel Fruchter israel.fruch...@gmail.com wrote: I fixed my two issues (Person name headshot), Great! Thank you very much. I'll have time to check the patch and commit it to Mercurial only tomorrow. By the way, anyone should feel free to fork the IMDbPY repository on Bitbucket (specifically http://bitbucket.org/alberanid/imdbpy_new_search_parsers/ ), commit his changes and ask for a pull. BTW, I really like the parser, it's a good scraper, Most of the credit should go to H. Turgut Uyar, who wrote that wonderful DOM/XPath-based parser. tried building it myself, imdb page killed lxml, elementtree and BeautifulSoup, Strange: we're based on lxml (falling back to BeautifulSoup if lxml is not installed). you think they are writing an ill-formed html deliberately to keep us scrapers away ?? Hmmm... I fear that, like every big portal, they have to cope with so many browsers and environment that a nice and standard HTML can't work. :-) Thanks again! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy 4.6 can't fetch person head shot (and who add IMDb to the names ?)
On Wed, Sep 29, 2010 at 12:45 PM, H. Turgut Uyar u...@itu.edu.tr wrote: Is it that fork or the one called imdbpy_parsers2010? No, pardon: my mistake. The correct project to fork is imdbpy_parsers2010 : http://bitbucket.org/alberanid/imdbpy_parsers2010 Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Suppressing lxml warning
On Sun, Oct 31, 2010 at 9:15 PM, Dustin Wyatt dustin.wy...@gmail.com wrote:
What's the best way to suppress the warning about not having lxml?
Probably the best way is to set the 'useModule' parameter, initialiazing
the IMDb instance.
Something like:
from imdb import IMDb
ia = IMDb('http', useModule='beautifulsoup')
It can also be set in the imdbpy.cfg configuration file (it should be copied
in your home directory as an hidden file, or in /etc)
Anyway, it would be a good idea to have a way to turn off all the
warnings from the code/configuration options.
HTH,
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
Nokia and ATT present the 2010 Calling All Innovators-North America contest
Create new apps games for the Nokia N8 for consumers in U.S. and Canada
$10 million total in prizes - $4M cash, 500 devices, nearly $6M in marketing
Develop with Nokia Qt SDK, Web Runtime, or Java and Publish to Ovi Store
http://p.sf.net/sfu/nokia-dev2dev
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDb redesign: IMDbPY status
Hi all, I've just pulled into the 'imdbpy' Mercurial repository some changes made in the 'imdbpy_parsers2010' repository. Some things will work now, other are still to be done. Persons' biographies work, and so most of the other information. The 'httpThin' parsers broken, and I have no plan to fix it (it was never really useful: let's consider it gone). I'm not too sure about the parsers for Characters and Companies, but they seem to be ok. The development will continue in the 'imdbpy_parsers2010': I plan to work on the 'mobile' parsers. As usual, any help is welcome. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Centralized Desktop Delivery: Dell and VMware Reference Architecture Simplifying enterprise desktop deployment and management using Dell EqualLogic storage and VMware View: A highly scalable, end-to-end client virtualization framework. Read more! http://p.sf.net/sfu/dell-eql-dev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Mysql foreign keys not created using innodb
On Thu, Oct 28, 2010 at 5:00 PM, laurent l.ver...@libertysurf.fr wrote: Hi everybody, Hi! I'm very sorry for the time it took to approve your mail: I didn't notice it in the list of pending messages. I installed imbdpy and downloaded the imdb files to install a mysql database of it. I ran ./imdbpy2sql.py -d /mnt/win_h/video/imdb_data/ --mysql-innodb -u 'mysql://user:passw...@localhost/base' and everything went fine. It took about 5 hours to complete. At the end, i got the innodb tables created but the foreign keys weren't created. Here is the end of the logs: [...] # TIME createForeignKeys() : 49min, 26sec (wall) 0min, 0sec (user) 0min, 0sec (system) EXECUTING END:FOR_EVERY_TABLE:ALTER TABLE %(table)s ENGINE=InnoDB;... EXECUTING ALTER TABLE aka_name ENGINE=InnoDB;... DONE! # TIME END command : 0min, 35sec (wall) 0min, 0sec (user) 0min, 0sec (system) Well, that's odd: it seems that the script _did_ something. I don't know if i did something wrong but i suspect that creating the foreign keys before having tables in innodb format is the problem because in myisam, we can't have foreign keys. Well, I assume you're right: probably the previous step just created some normal indexes, instead of foreign keys in the MyISAM tables. Right now i'm generating the foreign keys 1 by 1 so i have some time to report the problem :) Wooops. :-) To tell the truth foreign keys are not required, for IMDbPY, so you can live without them. Anyway, I've modified the imdbpy2sql.py script in the Mercurial repository [1] so that it would modify the tables (from MyISAM to InnoDB) _before_ the creation of the indexes and foreign keys. I've not tested it, but it should work. PS: thank you very much - I've added your name to the credits. +++ [1] http://bitbucket.org/alberanid/imdbpy/ -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Increase Visibility of Your 3D Game App Earn a Chance To Win $500! Tap into the largest installed PC base get more eyes on your game by optimizing for Intel(R) Graphics Technology. Get started today with the Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs. http://p.sf.net/sfu/intelisp-dev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Fwd: IMDB Analysis
On Sun, Nov 28, 2010 at 2:50 AM, Derek Ditch derek.ditch...@gmail.com wrote:
I'm working on a project that analyzes graph structures using a modified
version of PageRank for a sample data set, I'm considering IMDB, using
imdbpy,
Hi!
Your project sounds extremely cool. :-)
So, it doesn't look like imdbpy has the ability built-in to iterate through
all movies, or movies of a specific genre, nor of actors. So I suppose I
will create a new method of the Movie class and implementation within the
sql parser to return all results.
You're right: by itself IMDbPY doesn't have the ability to iterate over a
large subset of the IMDb database, and I don't think it's a feature we
should integrate too much; keep in mind that as a principle IMDbPY tries
to be transparent regarding its access to the information: such a feature
would be too specific of the SQL database, and impossible - or at least
legally dubious - to implement for the HTTP access.
Obviously a more or less separated package/framework to work on bunches
of items extracted from the SQL database would be more than welcome and,
to tell the truth, a way to express more complex searches on the SQL database
could be a really nice and useful feature. :-)
Basically, it works this way: each items which must be uniquely
identified (Person,
Movie, Character and Company instances) uses the 'id' primary key column of
its database table as ID (as you may have noticed, the ID used for a movie in
the SQL database is _not_ the imdbID used by IMDb on its web site, since the
latter are not included in the plain text data file).
So the best approach would be to access the SQL database in the same way
IMDbPY does: since we're amazingly cool ;-) we didn't settle on one ORM, but
we transparently support both SQLObject (we use its semantic, in our code)
and SQLAlchemy.
Their interface is abstracted in the dbschema.py, alchemyadapter.py and
objectadapter.py (beware: there's a certain amount of black magic involved :-)
To use them, the process is somewhat manual, and could probably be more
automated; to import what you need, see the __init__ method of the
IMDbSqlAccessSystem class in the sql/__init__.py file (very similar code can
be fund in the imdbpy2sql.py script, around line 277.
After that, as said, you can access the database using the created objects
(the ones returned by the getDBTables function) using the SQLObject syntax.
With these object, you can create complex queries on the database; once
you have the list of IDs you're interested in (or a generator of IDs), you can
use a normal imdb.IMDb('sql', ...) instance to access every information you
need using IMDbPY.
It goes without saying that it's possible that the information that you need
are somewhat limited, and such a solution could be too much: maybe you
can give up using the ORM abstraction and even IMDbPY, working directly
on the database.
Let me know how you decide to proceed, and if you need any help - as ideas
or clarification on the internals of IMDbPY: unfortunately right now I don't
have any time to write code :-/
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
Increase Visibility of Your 3D Game App Earn a Chance To Win $500!
Tap into the largest installed PC base get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Fwd: IMDB Analysis
On Thu, Dec 2, 2010 at 7:15 PM, Derek Ditch derek.ditch...@gmail.com wrote:
So, I've been able to begin building a graph reflecting relationships among
actors, but I've reached the point where it's huge iterations, so I'm
parallelizing it (using pp). The issue I'm having is that since I'm running
queries manually, I have to use sqlobject objects directly (i.e. CastInfo,
MovieInfoIdx, etc). I can do this in IPython by using from imdb.parser.sql
import *, and then running my script (even though my script imports the
same).
When I run the script standalone, I get errors telling me that CastInfo
isn't defined. I see that these are exported into globals in the SQL imdb
access class __init__, so, how can I ensure I'm importing these correctly?
That instances/classes are not created until you run the getDBTables
function of the proper module (objectadapter or alchemyadapter).
The easiest way to do it:
import imdb
ia = imdb.IMDb('sql', 'sqlite:///tmp/imdb.db')
print imdb.parser.sql.CastInfo
As you can see, after the imdb.IMDb object is instanced, you can access
the CastInfo (and other) class in the imdb.parser.sql module.
It goes without saying that you've to replace 'sqlite:///tmp/imdb.db' with
the correct credentials to access your database. :-)
HTH,
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
Increase Visibility of Your 3D Game App Earn a Chance To Win $500!
Tap into the largest installed PC base get more eyes on your game by
optimizing for Intel(R) Graphics Technology. Get started today with the
Intel(R) Software Partner Program. Five $500 cash prizes are up for grabs.
http://p.sf.net/sfu/intelisp-dev2dev
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.7 released
Released after a long delay, and despite that in a hurry, IMDbPY 4.7 can be found here: http://imdbpy.sf.net/ This is a transitional release, after the recent redesign of the IMDb pages. A new account is used and the new pages are parsed; for sure there are still many bugs; please read the README.redesign file for other details. Please contribute to the development with fixes and bug reports. Enjoy! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Proxy issue
On Mon, Jan 24, 2011 at 7:15 PM, David Kaufman david.kauf...@gmx.de wrote:
File G:\My Dropbox\Entwicklung\diribeo\imdb\parser\http\__init__.py,
line 237, in http_error_default
'proxy': self.get_proxy()}
IMDbDataAccessError: {'url':
'http://akas.imdb.com/title/tt0898266/epdate', 'error type':
'http_error_default', 'errcode': 500, 'headers': httplib.HTTPMessage
instance at 0x03838FD0, 'proxy': '', 'errmsg': 'Internal Server
Error'}
This actually already should have been fixed:
http://www.mail-archive.com/imdbpy-help@lists.sourceforge.net/msg00204.html
Hi!
From a fast check (I'm looking at a tcpdump), my headers seems to be
correct.
Are you using it in a strange environment of some kind, which could
rewrite the headers adding its own User-agent?
Which version of Python are you using, and what's the operating system?
If you can, you can try to look at what passes on the network, with
wireshark or something.
Bye,
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)!
Finally, a world-class log management solution at an even better price-free!
Download using promo code Free_Logger_4_Dev2Dev. Offer expires
February 28th, so secure your free ArcSight Logger TODAY!
http://p.sf.net/sfu/arcsight-sfd2d
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDbPY 4.7 released
On Wed, Jan 26, 2011 at 8:56 AM, H. Turgut Uyar u...@itu.edu.tr wrote: On 01/23/2011 05:12 PM, Davide Alberani wrote: Please contribute to the development with fixes and bug reports. Will development continue on the main fork or on the imdbpy_parsers2010 fork? Good question. :-) If there're no reason to do otherwise, I'd prefer to move the future development in the main repository (removing imdbpy_parsers2010). -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Filmografie
On Mon, Feb 7, 2011 at 1:47 AM, Sereno Polveroso serenopolver...@hotmail.com wrote: Thanks, now it's ok! Good! I have only one more question: I've seen that IMDb shows you immediatly the films italian akas, for example if you search Tarantino in filmography it shows you Bastardi senza gloria instead of the english title. Is there a way to obtain this in the field 'director', for example in the program you have send me. Or i have always to uodate the film and search the key 'akas' and then search the italian aka?? I fear there's no easy way. Some time ago it was possible, setting IMDbPY to point to the italian.imdb.com site, but now that site is an alias for imdb.it, which has slightly different web pages. You can do some tests changing the imdbURL_base variable in the imdb/__init__.py file, but expect some problems. Once you have all the information about a movie (after the 'update'), you can use the guessLanguage method of a Movie instance, to get its language. With that, one can more or less easily write a function to extract the AKA in the original language. To tell the truth we can also check for some special notes in the AKA list (e.g.: original title or something like that). Hmmm... this is a feature we can take into consideration for a future release. :-) -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPY User Support
On Sun, Feb 20, 2011 at 16:46, Te Kim tk...@ucsc.edu wrote: My group is using a SQL database to make these calls. Is this a bug or could there possibly be something wrong with our database? What you're trying to do should work. Are you sure to have downloaded and processed all the available plain text data files? Please download each available file (and be sure that they're not corrupted) and - if possible - keep track of the output of the imdbpy2sql.py script. In a Unix environment, use something like: imdbpy2sql.py -d /path/to/imdb_files/ -u db::/connection_params 21 | tee debug.txt Notice that you can somewhat speed-up the process creating temporary CSV files (see the README.sqldb for complete information - essentially you need to add a '-c /path/to/a/tmp/directory/' argument to the command line). In addition, I am unable to find ratings or votes in the database. The query: select * from movie_info mi join info_type it on mi.info_type_id = it.id where it.info = 'runtimes' group by mi.info; yields results, as well as in the case where it.info = 'genres'. However, if I change it.info to 'votes', 'cast', or 'rating', I get 0 rows. Well, 'cast' information are not there: they are in the cast_info table. Votes and rating should be present. What could be the reason for the missing information? Hard to tell: or you don't have some information, or something has gone wrong importing it. Unfortunately I'll be unable to do deeper tests on the database access system for another day or two, so in the meanwhile try to collect as much information as possible to spot the problem. HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDb IDs aren't really necessary for names and titles
On Tue, Mar 29, 2011 at 07:13, Michael Grier mr.michael.gr...@gmail.com wrote: I found this out by using the moviedb-3.24 programs. If you url encode the entire original movie line from movies.list... for example, on an episode: Yes, it's the old search system and more or less what it used to be the Exact Primary search feature of IMDb. Until some years ago, using a special key in the query, it was possible to do a strict search for a match (getting back the searched page if and only if an exact match existed, or an error page otherwise, if I recall correctly). By now, that key is gone: at the time, IMDbPY used it to convert from title/names to IDs. What's wrong with your query is that it doesn't work for titles or names containing what we call an imdbIndex (the roman number after the name/title, to distinguish between homonyms). E.g., try to search for Gibson Mel (I): http://www.imdb.com/Name?Gibson+Mel+%28I%29 it doesn't present you the page of Mel Columcille Gerard Gibson, but just a list of people named Mel Gibson or something similar. Now, the message of Petite Abeille seems to contradict me somehow, so I suppose I've to do some tests. :-) Bye! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Create and publish websites with WebMatrix Use the most popular FREE web apps or write code yourself; WebMatrix provides all the features you need to develop and publish your website. http://p.sf.net/sfu/ms-webmatrix-sf ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Mon, Apr 11, 2011 at 18:35, darklow dark...@gmail.com wrote: File ./imdbpy2sql.py, line 1194, in _toDB CURS.executemany(self.sqlstr, self.converter(l)) psycopg2.DataError: invalid byte sequence for encoding UTF8: 0xc320 HINT: This error can also happen if the byte sequence does not match the encoding expected by the server, which is controlled by client_encoding. Any suggestions? I found similar topic, but there were also no solutions. Yes, I've had other reports about this bug. Seems to be related to some garbage in the actors.list.gz file. I hope to have time to investigate the problem within a week or two. Thanks for the bug report! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Forrester Wave Report - Recovery time is now measured in hours and minutes not days. Key insights are discussed in the 2010 Forrester Wave Report as part of an in-depth evaluation of disaster recovery service providers. Forrester found the best-in-class provider in terms of services and vision. Read this report now! http://p.sf.net/sfu/ibm-webcastpromo ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Mon, Apr 11, 2011 at 18:35, darklow dark...@gmail.com wrote:
File ./imdbpy2sql.py, line 1194, in _toDB
CURS.executemany(self.sqlstr, self.converter(l))
psycopg2.DataError: invalid byte sequence for encoding UTF8: 0xc320
HINT: This error can also happen if the byte sequence does not match the
encoding expected by the server, which is controlled by client_encoding.
Hi all,
I'm writing regarding the recent 0xc320 problem with IMDbPY.
The above notice is extremely interesting, and should be investigated:
how can it be that 0xc320 is not UTF8 encodable?
It should work; from the Python prompt:
unichr(0xc320).encode('utf8')
'\xec\x8c\xa0'
Anyway, as a very fast and dirty fix (the main problem is probably some
crap in the data files), try this: after line 1181 of imdbpy2sql.py, add:
k = k.replace('\xec\x8c\xa0', '')
So that the nearby lines will become:
try:
k = k.replace('\xec\x8c\xa0', '')
t = analyze_name(k)
except IMDbParserError:
Please be aware that this fix was not tested at all, but I'm
almost sure that, at the above point, 'k' is a string encoded in utf8.
Anyway, beside the garbage theory, I have another idea
about the source of the error, but I have to verify it later...
Bye, and let me know if it works!
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Wed, Apr 13, 2011 at 08:46, darklow dark...@gmail.com wrote:
Maybe someone knows some fast dirty fix at least how to skip such invalid
byte sequence strings while there are no official fix, so i can finish the
import?
Can we detect invalid byte characters?
Hi again,
actually my problem is that I'm unable to reproduce this bug. :-)
Using Postgresql and SQLObject, my run goes on smooth.
I have downloaded the 'actors.list.gz' file today, so it's possible that some
garbage was removed.
Anyway, the previously proposed solution was obviously flawed, since
the problem was on _character_ names.
So, let's edit again the imdbpy2sql.py file and change the lines around 1540
so that they become:
movieid = CACHE_MID.addUnique(title)
if role is not None:
roles = filter(None, [x.strip() for x in role.split('/')])
for role in roles:
role = role.replace('\xec\x8c\xa0', '') # TEMPORARY FIX
cid = CACHE_CID.addUnique(role)
sqldata.add((pid, movieid, cid, note, order))
Maybe this will help... who knows? :-)
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
Benefiting from Server Virtualization: Beyond Initial Workload
Consolidation -- Increasing the use of server virtualization is a top
priority.Virtualization can reduce costs, simplify management, and improve
application availability and disaster protection. Learn more about boosting
the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Sun, Apr 17, 2011 at 14:04, darklow dark...@gmail.com wrote: Updated this morning to latest data files, no change and unfortunately this fix also doesn't work. Hmm... to debug a problem like this without being able to reproduce, is extremely difficult. :-/ This error started when we uninstalled imdbpy (left all the dependency libs) and started run it without installation. Maybe there is some kind of problem and some kind of hidden unicode dependencies? Maybe you can try to run without installation, jus from source? Have you some very good reason to do so? :-) Can't you try to purge every reference to IMDbPY left on the system (search for the scripts in /usr/bin/ and /usr/local/bin/ and be sure that import imdb fails, at the python prompt) and see if the problem is solved, after IMDbPY 4.7 is reinstalled? If you have problems locating the IMDbPY package, just open the Python prompt and: import imdb print imdb Also every time i start the script i receive two warnings: 2011-04-17 11:13:37,398 WARNING [imdbpy.parser.sql.aux] /data/web/imdb/imdbpy4.7-159671/imdb/parser/sql/__init__.py:125: Unable to import the cutils.ratcliff function. Searching names and titles using the sql data access system will be slower. This will force IMDbPY to use some pure-python fall-back functions. It's entirely possible that there are some bug in these functions, even if a run without cutils.so is running fine, for me (so far). IMPORTING psyco... FAILED (not a big deal, everything is alright...) That's not a problem for sure. Right now, my first guess is that somewhere, after the *.list files ar read and turned into utf-8 encoded strings, the imdbpy2sql.py script does Something Very Wrong(tm) to a string (like cutting it at a certain place, ending up cutting a single utf-8 encoded char in two: this could explain the error). I've tried the conversion suggested by Petite Abeille, and it works fine. Please, could you cut a small piece (few kilobytes) of the actors.list file, and attach it (no cut-and-paste)? It goes without saying that you should chose a portion where you see (or guess are) the strange chars :-) Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Mon, Apr 18, 2011 at 08:53, darklow dark...@gmail.com wrote: We have Debian linux on our server and our sysadmin allows only stable packs. However latest version of imdbpy has these md5 checksum that are quite important in our situation, that is why i have to run it from source. Ehhh... what about a virtual machine or - even easier - virtualenv [0] Thanks for the file, I hope to look at it within a day or two. +++ [0] http://pypi.python.org/pypi/virtualenv -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Mon, Apr 18, 2011 at 09:30, Davide Alberani davide.alber...@gmail.com wrote: Thanks for the file, I hope to look at it within a day or two. Ok: the file is correctly encoded in iso8859-1, as expected, and contains no garbage. Using it as the only input for imdbpy2sql.py (putting the attached file in a directory by itself), I can run the script with no errors (besides the expected warnings about missing files). I'm using the version from the Mercurial repository, without the cutils.so library. Please, if you can't install IMDbPY in your system, consider the use of virtualenv. Having tried that, I have to recommend you to double check the settings of your Postgresql server for some kind of incoherences about encodings and collations. HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ actors.list.gz Description: GNU Zip compressed data -- Benefiting from Server Virtualization: Beyond Initial Workload Consolidation -- Increasing the use of server virtualization is a top priority.Virtualization can reduce costs, simplify management, and improve application availability and disaster protection. Learn more about boosting the value of server virtualization. http://p.sf.net/sfu/vmware-sfdev2dev___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Wed, Apr 20, 2011 at 14:08, darklow dark...@gmail.com wrote: Still no luck :/ maybe the problem is in some environmental variables or settings, which on installed version are present, but running from source are missing or incorrect? Seems unlikely to me. What about this, i printed out some variables: print sys.stdout.encoding - UTF-8 print sys.stdin.encoding - UTF-8 print sys.getdefaultencoding(); - ascii Is it ok that sys.getdefaultencoding(); == ascii ? These are fine. I've reproduced - at the best of my capabilities - your environment: - no IMDbPY installed in the system. - IMDbPY from source (the latest version in the Mercurial repository), setting the PYTHONPATH environment variable to point to the source directory. - the cutils C module was not compiled. - the last actors.list.gz file. - postgres 8.4; my database was created with these settings: CREATE DATABASE imdb WITH OWNER = postgres ENCODING = 'UTF8' TABLESPACE = pg_default LC_COLLATE = 'it_IT.utf8' LC_CTYPE = 'it_IT.utf8' CONNECTION LIMIT = -1; I've run it with your and other portions of the actors.list.gz file, and everything went fine. Now... if I were you, I'd: - create a virtualenv environment with: virtualenv --no-site-packages - install in it IMDbPY, using easy_install or pip (the executable in your virtualenv, I mean) so that you'll have all the correct dependecies available. - run the imdbpy2sql.py within your virtualenv. If it still fails: - check your postgres settings. - try using SQLite (just for a test) - see notes in README.sqldb HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Fulfilling the Lean Software Promise Lean software platforms are now widely adopted and the benefits have been demonstrated beyond question. Learn why your peers are replacing JEE containers with lightweight application servers - and what you can gain from the move. http://p.sf.net/sfu/vmware-sfemails ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Sun, Apr 24, 2011 at 20:03, Thomas Stewart tho...@stewarts.org.uk wrote: I've just had a try using sqlite with fresh lists and on my Debian system and I get this: thomas@ikaite:~$ /tmp/imdbpy2sql.py -d /home/thomas/Desktop/imdb/lists -u sqlite:///home/thomas/Desktop/imdb/imdb.db --sqlite-transactions IMPORTING psyco... DONE! [...] CURS.executemany(self.sqlstr, self.converter(dataList)) pysqlite2.dbapi2.ProgrammingError: You must not use 8-bit bytestrings unless you use a text_factory that can interpret 8-bit bytestrings (like text_factory = str). It is highly recommended that you instead just switch your application to Unicode strings. This specific bug (a bad interaction between SQLObject and SQLite) should be fixed in the version in the Mercurial repository; isn't it? -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Fulfilling the Lean Software Promise Lean software platforms are now widely adopted and the benefits have been demonstrated beyond question. Learn why your peers are replacing JEE containers with lightweight application servers - and what you can gain from the move. http://p.sf.net/sfu/vmware-sfemails ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Sun, Apr 24, 2011 at 21:03, darklow dark...@gmail.com wrote: I tried reinstalling all installed dependancies and run from clean sources, but no luck. I tried to run scripts with SQLAlchemy instead of SQLObject, but same error, so the problem is not there. Perfect - these tests are really important to spot the problem. Every test takes about 1h, because error takes place in Actors Cast list. Wait: I'll read the rest of your mails tomorrow, but this can help you to do things faster: you don't need the other files at all. Simply put the actors.list.gz file in a directory by itself, and run imdbpy2sql.py with this directory as -d argument. You can even use a shorter version of actors.list.gz, just remember to leave the lines at the begin and at the end (various separators are used to identify where the data begin), like I did with the actors.lists.gz file that I attached some days ago. In the 'docs/goodies' directory you'll find the 'reduce.sh' script, which takes a whole directory of *.list.gz files and reduce them to 1% of their length. It makes me think that problem might be somewhere in cutting line in peaces, maybe it does something wrong, cuts some good unicode character into peaces and so invalid byte sequence appears. My guess, too... it's just that I can't see where it happens... :-/ Thanks for your tests! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Fulfilling the Lean Software Promise Lean software platforms are now widely adopted and the benefits have been demonstrated beyond question. Learn why your peers are replacing JEE containers with lightweight application servers - and what you can gain from the move. http://p.sf.net/sfu/vmware-sfemails ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Sun, Apr 24, 2011 at 22:44, darklow dark...@gmail.com wrote: Yes i can confirm - Script version 4.6 works perfectly on same server with same files. And i think by this we come closer to solution. Excellent! (well, it still baffles me why I'm absolutely unable to reproduce the problem on my system, but that's another story...) Maybe this helps to identify the problem, this is what we did on our server. (Remember, we are doing this copying because there are only stable versions for Debian on server allowed, but we need those md5 hashes from 4.7 version) I'll look at your setup tomorrow. I'll surely sound pedantic, but... seriously: why you don't use a virtualenv environment? It's easy to install and doesn't require root privileges. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Fulfilling the Lean Software Promise Lean software platforms are now widely adopted and the benefits have been demonstrated beyond question. Learn why your peers are replacing JEE containers with lightweight application servers - and what you can gain from the move. http://p.sf.net/sfu/vmware-sfemails ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Tue, Apr 26, 2011 at 09:36, darklow dark...@gmail.com wrote: Thanks, let me know if you have any ideas, how to fix the problem... Eh... As usual, right now I'm really busy. :-( I looked at virtualenv documentation, i didn't understand how to use it, Ok, let's try: - download virtualenv from http://pypi.python.org/pypi/virtualenv#downloads - tar xvfz virtualenv-1.6.tar.gz - cd virtualenv-1.6 - python virtualenv.py --no-site-packages ~/myvenv - cd ~/myvenv - . ./bin/activate # notice the initial dot - pip install formencode # bug with the dependencies. :( - pip install IMDbPY # or download from the Mercurial repository and run 'python setup.py install' The most important step is the activation of the virtualenv: your prompt should change to something like (myvenv)$ to denote that your virtualenv is active. Now, always from inside the virtualenv, you can run the imdbpy2sql.py script: everything was installed locally to your ~/myvenv/ directory (the local python interpreter is in ~/myvenv/bin/python). If you need to deactivate the virtualenv, simply run the deactivate command. HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Thu, Apr 28, 2011 at 22:52, darklow dark...@gmail.com wrote: However last command pip install IMDbPY didn't succeeded so well, looks like i got exactly the same error, that another user reported some days ago in the same discussion and he has also UTF-8 encoding problem: Sure: you don't have the python-dev package installed in your system. :-/ A per-user installation is possible, but a little tricky... By running python setup.py install I receive the same error. I also tried latest version (4.8dev20110425) but got same error. Using the latest version sources, run (after you've activated your virtualenv!): python setup.py install --without-cutils Maybe this explains the problem why the script doesn't handle UTF-8 at first place - some strange incapabilities with cutils.c I've run some tests without the compiled C module, so I think this is not the cause, but at this point... who knows. :-) -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Mon, May 2, 2011 at 08:47, darklow dark...@gmail.com wrote: Thank you for your patience and guiding through the tests, i really glad we finally found the problem and fixed it. Yep, even if it took a little too long. :-) Just curious, why only me and one another user encountered this problem, but when you run the same tests, you didn't see the error? :) It may have something to do with the use python library to connect to Postgres. Maybe some libraries handle gracefully this kind of error; I have to check better the versions installed on my system and on the virtualenv I've used to reproduce the bug. In fact the right thing to do in such cases is to raise an exception (like in our case); other databases - or libraries to connect to databases - like MySQL simply ignore with a warning these errors (not a great idea). -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- WhatsUp Gold - Download Free Network Management Software The most intuitive, comprehensive, and cost-effective network management toolset available today. Delivers lowest initial acquisition cost and overall TCO of any competing solution. http://p.sf.net/sfu/whatsupgold-sd ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Key Error when trying to access show['episodes']
On Mon, May 9, 2011 at 09:26, Fruch israel.fruch...@gmail.com wrote:
Found the problem:
in DOMHTMLEpisodesParser
file: \imdb\parser\http\movieParser.py
lines: 1703 - 1704
should change to:
if key.startswith('filter-season-'):
season_key = key[len('filter-season-'):]
Wasn't it already fixed in the mercurial repository [1]?
Thanks for your help!
+++
[1] http://bitbucket.org/alberanid/imdbpy/
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
WhatsUp Gold - Download Free Network Management Software
The most intuitive, comprehensive, and cost-effective network
management toolset available today. Delivers lowest initial
acquisition cost and overall TCO of any competing solution.
http://p.sf.net/sfu/whatsupgold-sd
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign
On Tue, Jul 19, 2011 at 16:57, Alberto Malagoli albem...@gmail.com wrote: Hello, I am an habitual user of IMDbPY, I use it for my projects, and compared with other similar tools I must say this is the best one! Thanks! Unfortunately in the last months it's affected by many problems, due to the IMDb redesign and my total lack of free time. :-( So, as an homage to the hard work you do on it, I would like to contribute to the project a bit by proposing a new style for the website. Ehi, thank you for the offer! Any help is greatly appreciated. Do you have any ideas for the layout and the technologies to use? Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign
On Thu, Jul 21, 2011 at 09:25, Alberto Malagoli albem...@gmail.com wrote: By the way, could you add it to the list of programs using IMDbPY? Done! Thank you! Yes, I have some ideas, but I want to think about them a bit more, than I'll send you some mockups.. But I want to inform you that in the next days I will be on holiday, so I'll start working on it the next week! eheheh, we're in no hurry. Have a nice holiday. :-) I also would like to ask you if want to add or remove something from the current website, I mean some parts or pages, some text, Don't know... the altar tab can go away; the polls can be replaced with something more functional than the ones I've written. Besides that, it's just informative text. I don't have any preferences about how it's organized (ok: no frames and no blinking texts ;-) p.s. Anyway, I'm italian too :) Noticed. :-) Bye! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- 10 Tips for Better Web Security Learn 10 ways to better secure your business today. Topics covered include: Web security, SSL, hacker attacks Denial of Service (DoS), private keys, security Microsoft Exchange, secure Instant Messaging, and much more. http://www.accelacomm.com/jaw/sfnl/114/51426210/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Imdbpy
On Tue, Jul 26, 2011 at 02:25, Zach Anderson zpanderso...@gmail.com wrote: I would love to use imdbpy in an add-on that I'm writing for HTPC program XBMC. The only issue that I'm having is that I'd prefer to only have 1 include file. Is this possible? I guess you can try to create an egg. From the source tree, you have to run the command: python ./setup.py bdist_egg After that you can install the egg on the target system with something like: easy_install name_of_the.egg I'm not sure that the current setup can suit your needs. Maybe you can try to set the zip_safe parameter in the setup.py file to True, instead of False. By the way, there's a known problem with the dependencies of SQLObjects, which depends on FormEncode. If the installation fails, try to run it again. If it fails again, manually install FormEncode before (or exclude it, if you don't need it). HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign
On Fri, Jul 29, 2011 at 16:51, Alberto Malagoli albem...@gmail.com wrote: http://dl.dropbox.com/u/921872/IxD/IMDbPY%20logo.png Cool! I like the second of the second row, but I'm not sure if it can work on a light/white background... Ok.. I decided to join some menus, for a simpler navigation, renaming some of them. Ok, good for me. I still don't know how to replace the polls.. Which technology did you used to write the current website? It was manually coded by me in PHP, using a MySQL database. Since I'm out of ideas for new polls subjects, I think we can also leave it out, for the moment! :-) Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign
On Sun, Jul 31, 2011 at 12:22, Alberto Malagoli albem...@gmail.com wrote: Don't worry, I would like to use the same colors I used on that mockup for the header on top of the page. Do you like them? Hmmm... have I missed a link, here? :-) I've seen only the logos. Anyway, my taste for colors is not really meaningful, so I'm fine with anything you think is ok. ;-) -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] Getting information about genres only
On Sun, Jul 31, 2011 at 09:58, Saravanan
saravanan.thirumuruganat...@mavs.uta.edu wrote:
I am trying to write some Ubuntu Unity lens using IMDBPy.
Great! Let me know when it's done!
For this purpose, I need only genre information. I took a look at the code and
did not find any get_movie_genre kinda function. So currently, I am
using ia.update(x,main) . Is there anyway to just the genre information?
There's no way to get only the 'genres' key; you're doing right, asking for
only the 'main' info set (after that, you can access the 'genres' key of the
Movie object).
If you're worried about performances, you can consider using the
'mobile' data access system (i.e.: ia = imdb.IMDb('mobile') ) and
leaving the rest of your code untouched.
It should be slightly faster, but... I notice right now that it fails
to parse the 'genres' key. I have to fix it. :-)
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
Got Input? Slashdot Needs You.
Take our quick survey online. Come on, we don't ask for help often.
Plus, you'll get a chance to win $100 to spend on ThinkGeek.
http://p.sf.net/sfu/slashdot-survey
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] Getting information about genres only
On Sun, Jul 31, 2011 at 13:38, Davide Alberani davide.alber...@gmail.com wrote: It should be slightly faster, but... I notice right now that it fails to parse the 'genres' key. I have to fix it. :-) Ok, fixed in the Mercurial repository and added your name to the credits. Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign - first proposal
On Tue, Aug 23, 2011 at 12:29, Alberto Malagoli albem...@gmail.com wrote: Anyway, let me know if you like it Hi! Yeah, I like it a lot, thanks! Nice design: very clean. - links in the middle of the page are like fast paths to the most used pages. Is there a guests counter in the current website, so we could know which pages are currently the most visited ones? There are google analytics and piwik; do you need some stats? By the way, in the last week a lot of bugs were fixed, and so I think we can be ready for a news release... let's say before the end of September. :-) It would be cool to have the new web site, for it. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign - first proposal
On Tue, Aug 23, 2011 at 15:04, ronaldhatcher ronaldhatc...@mac.com wrote: was wondering what the changes of moving from hg to git would be? Not much, not only for the fact that I slightly prefer mercurial. :-D No, seriously: they are both wonderful tools, and both github and bitbucket are great services. The differences from each other are very few, compared to the differences with svn. Did I already mentioned that I prefer mercurial? ;-) I personally find github excellent and since most of my other work is in git so I think it would be great. But IMDbPY already has a github official repository: https://github.com/alberanid/imdbpy I keep it in sync with the mercurial repository on bitbucket [0] using the hg-github plugin [1]. So if you want to fork the project on github and develop using git, feel free! +++ [0] http://bitbucket.org/alberanid/imdbpy/ [1] http://hg-git.github.com/ -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Get a FREE DOWNLOAD! and learn more about uberSVN rich system, user administration capabilities and model configuration. Take the hassle out of deploying and managing Subversion and the tools developers use with it. http://p.sf.net/sfu/wandisco-d2d-2 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign - first proposal
On Wed, Aug 24, 2011 at 21:27, Alberto Malagoli albem...@gmail.com wrote: There are google analytics and piwik; do you need some stats? Yes, I would like to know which pages are the most visited one. So I will add these pages on the home as fast links.. Ok, I'll send you the data. That's great! I saw IMDB pages are chaged in the last few weeks, so the last fixes takes into account these changements? Mosto of them, I hope. I also would like to ask you if there are any plans about a Python 3 porting... Not sure about it: it also depends on the porting of our dependencies, like SQLAlchemy/SQLObject. - I cannot understend the difference, under the downloads page, between official packages and unofficial packages. May I merge them into one or not? If you want, yes. The difference is that the official ones were made by the package managers of the distribution, the other by me, so they may not be fully compliant with the policies of the distribution. - Is the mobile version still under development? Where can I put it, under a development page or under the downloads page? No, the Symbian interface is no longer developed or supported. :-/ I fixed some problems on the home page, and added some new stuff, so check it: http://dl.dropbox.com/u/921872/IxD/IMDbPY/index.html I also created the License page: Good - I link them! Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- EMC VNX: the world's simplest storage, starting under $10K The only unified storage solution that offers unified management Up to 160% more powerful than alternatives and 25% more efficient. Guaranteed. http://p.sf.net/sfu/emc-vnx-dev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign - first proposal
On Mon, Aug 29, 2011 at 23:21, Alberto Malagoli albem...@gmail.com wrote: No, the Symbian interface is no longer developed or supported. :-/ Mhh, ok... I'm still thinking about where I could put it, but I have no ideas.. I guess it can can removed. After all, there's no way that we'll return to work on a Symbian. :-) I'm working also on new pages like the downloads one. Here's the last version: http://dl.dropbox.com/u/921872/IxD/IMDbPY/downloads.html From now on, you can start navigating the website through the menu on top. Take a look at the home page also (I added some new stuff). Wow! This site is awesome! It's really wonderful. Taking a look at the documentation part, I have to say that by now is a bit confusing. Contents should be organized in a different way. I don't want to say that it has to be changed now, but a reorganization of the documentation could be a necessary future work. True, I've always created a new text document every time there was something new, but they are too many and too long. :-( By the way, if you need access to the sourceforge web server, let me know. What's your username on sourceforge? -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Special Offer -- Download ArcSight Logger for FREE! Finally, a world-class log management solution at an even better price-free! And you'll get a free Love Thy Logs t-shirt when you download Logger. Secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsisghtdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Limit imdbpy2sql to only movies - KIND_ID=1
On Fri, Sep 2, 2011 at 09:09, darklow dark...@gmail.com wrote: Since i dont need other movie types than Movies, which KIND_ID=1 so it means i could narrow down from 1 985 650 to 560 606 titles, which would greatly improve performance for some of my scripts. So i could get rid of ~1.4 million titles, it means event more million names. Hmmm... I have to think about and look at the code, but I fear it's nearly impossible. The information about the kind of a movie is not taken from a single file. Instead, each time a title is found (in any of the parsed files), it's analyzed (and amongst other things its kind is detected) and checked to see if we already known this title and if it has an titleID already given (otherwise, a new progressive one is assigned). The whole process is automated, and I don't think it would be easy to exclude only a certain kind of title, sorry. This way of working makes possible to import only a single file (e.g. special-effects-companies.list.gz) and incidentally makes the movies.list.gz file almost useless. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Special Offer -- Download ArcSight Logger for FREE! Finally, a world-class log management solution at an even better price-free! And you'll get a free Love Thy Logs t-shirt when you download Logger. Secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsisghtdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign - first proposal
On Thu, Sep 8, 2011 at 16:46, Alberto Malagoli albem...@gmail.com wrote: Sorry if I haven't posted any news or reply for a while but I had some problems with my computer, which is to the doctor by now.. :( No problem at all... I'm as busy as hell, at the moment. :-/ Mhh, I don't think I need it by now, anyway my username is alpert_ (with an underscore at the end). Perfect. Once you're ready, you can give me your public ssh key, so that you can directly upload files to the web server (if you need instructions, just ask). Bye! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Why Cloud-Based Security and Archiving Make Sense Osterman Research conducted this study that outlines how and why cloud computing security and archiving is rapidly being adopted across the IT space for its ease of implementation, lower cost, and increased reliability. Learn more. http://www.accelacomm.com/jaw/sfnl/114/51425301/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Problem with importing to MySQL database
On Tue, Sep 20, 2011 at 12:50, K Hein khei...@gmail.com wrote: It throws the following error. It seems like the exception handler code tries to insert a duplicated record into cast_info table with existing primary key. Yep, that's the case. Can anyone help me fix this problem or suggest any work around solution? Unfortunately I'm unable to reproduce the problem. Do you have the latest version of IMDbPY installed from the Mercurial repository? You can download it from here: https://bitbucket.org/alberanid/imdbpy/ (see the get source link) By the way, do you have SQLObject or SQLAlchemy installed? (or both?) Another thing you can try, is to use the -c /path/to/an/empty/directory argument: imdbpy2sql.py will write a series of .csv files in that directory, that will be later imported into the database. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Fwd: imdbpy2sql 4.7 - invalid byte sequence for encoding UTF8
On Mon, Sep 19, 2011 at 21:36, darklow dark...@gmail.com wrote: This fix worked for about some months and unfortunately there is similar encoding error in latest data files (16.sep.2011) Tried right now, and everything went fine for me, using the same version. :-( Are you using SQLObject or SQLAlchemy? Version of MySQL? -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign - first proposal
On Thu, Sep 22, 2011 at 11:10, Alberto Malagoli albem...@gmail.com wrote: After a long time, I had my computer back and I had a chance to work on the IMDbPY website, so here you are the last version: http://dl.dropbox.com/u/921872/IxD/IMDbPY/index.html As usual, very very cool. :-) I like a lot the various effects. I would like to ask if someone could help me on that, because it's a big work and if I do it all alone I will probably make mistakes... Thanks! I can surely, in the weekend (which and of which month, is still to be decided... :) What's the best way to coordinate the effort? A repository somewhere? -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign - first proposal
On Fri, Sep 30, 2011 at 18:43, Alberto Malagoli albem...@gmail.com wrote: What's the best way to coordinate the effort? A repository somewhere? Well, probably a repository would be a good idea, anyway I have a kind of hate against them, I always have problems and I've never completly understood them :P Eheh, no problem. I've arranged a Mercurial repository on sourceforge: http://imdbpy.hg.sourceforge.net/hgweb/imdbpy/imdbpy-site/ You can clone it (using the 'hg' command from the mercurial package): hg clone ssh://your_user_n...@imdbpy.hg.sourceforge.net/hgroot/imdbpy/imdbpy-site and added a site-sync rsync script to it, to syncronize the local content to http://imdbpy.sf.net/new/ (for now) If you are working on a Windows environment, let me know: I'm sure there are replacements for these tools. Basic introduction to the sf repository is here: https://sourceforge.net/apps/trac/sourceforge/wiki/Mercurial Veery basically: - to add a new file or directory: hg add FILE_NAME - to update from the repository: hg pull - once you're happy with a set of changes you made: # download new things hg pull # apply them to the local files - there may be conflicts. hg update # if any, resolve conflicts; then save your changes locally: hg commit -m 'description of what was changes' # send your changes to the server: hg push - to see if there's something new in the repository: hg in - to see if you've something not committed (locally!): hg status - to see if you've something to send to the repository: hg out - to see what you've changed and not committed yet (locally!): hg diff A very good introduction: http://hginit.com/ But fear not: you only need very basic operations, trust me. :-) And if you need any help, ask! and here you can find a zipped version of it: http://dl.dropbox.com/u/921872/IxD/IMDbPY/IMDbPY.7z Copied to the repository and synced on the web site (on the /new path, for now) I will not work on the website on this weekend Eh, I'm busy too. :-/ Bye, have a good weekend! You too! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- All of the data generated in your IT infrastructure is seriously valuable. Why? It contains a definitive record of application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-d2dcopy2 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] website redesign - first proposal
On Tue, Aug 23, 2011 at 15:04, ronaldhatcher ronaldhatc...@mac.com wrote: since most of my other work is in git so I think it would be great. And now bitbucket also supports Git. :-) http://blog.bitbucket.org/2011/10/03/bitbucket-now-rocks-git/ -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql.py
On Mon, Oct 10, 2011 at 13:47, Tiago Serra tiagofse...@gmail.com wrote: Thanks i find it in the /usr/share/doc….. Ok, then you have an old version. :-P Sorry, you have to use the one in the Mercurial repository: https://bitbucket.org/alberanid/imdbpy/ (notice the get the source link, on the right). To install it (as root): python setup.py install You may need to remove the installed package and to compile it you need python-dev and possibly sqlobject, sqlalchemy and some other packages (easy_install tries to satisfy these dependencies, but sometimes it fails on sqlobject... if that's the case, let me know) -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] cant retrieve movie list from person
On Fri, Oct 28, 2011 at 19:57, Gustaf Nilsson gus...@laserpanda.com wrote:
Hi
Im trying to retrieve a list of movies a person has worked on, but it only
seems to work on famous people (?)
Hi!
First of all, if you're not already doing it, I suggest
to use the version from the Mercurial repository:
http://bitbucket.org/alberanid/imdbpy/
The stable version is very old: I hope to release a
new one in a few days.
b = x.get_person(1459663)
b.get_titlesRefs()
Ok, get_titlesRefs is (mostly) an internal method,
and only returns links to titles as found in
the texts (biographies, plot summaries and so on).
To get the other information, the objects instances
of the Movie class work like python dictionaries.
You can get a list of available information with:
b.keys()
After that, you can do things like:
b.get('actor')
b['visual effects']
Hope this helps! (by the way: wow, you worked on some
very cool movies!)
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
The demand for IT networking professionals continues to grow, and the
demand for specialized networking skills is growing even more rapidly.
Take a complimentary Learning@Cisco Self-Assessment and learn
about Cisco certifications, training, and career opportunities.
http://p.sf.net/sfu/cisco-dev2dev
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.8 and new site released
Hi all, I've just released the long-awaited IMDbPY 4.8, with too many bug fixes to mention. Pardon for the slowdown in the development; I'm sure there are still many bugs and I'd like to see some fixes to some core pieces of code (after more than 7 years and almost 50 releases, it's probably a good idea to rewrite a function or two ;-) So, if anyone wants to help, let us know! With this release, we also have a shiny new web site, courtesy of Alberto Malagoli who kindly joined the development team (thanks and welcome aboard, Alberto!) As usual, you can download IMDbPY from: http://imdbpy.sf.net/ Enjoy! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- RSAreg; Conference 2012 Save #36;700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] getting grossing data
On Sat, Nov 12, 2011 at 20:44, Zsolt Ero zsolt@gmail.com wrote:
I have just started using IMDbPY. I would like to get the grossing
field of a given movie, but I don't know how.
These information are included in the 'business' data set, that
you've to retrieve, first.
An example:
import imdb
ia = imdb.IMDb()
avatar = ia.get_movie(0499549)
ia.update(avatar, 'business') # get the business information
business = avatar.get('business') or {} # may be empty
print business.get('gross') # to see what else is available: print
business.keys()
Unfortunately I notice now that a lot of garbage is colleted, too:
you should just get a list of information... :-/
I'll try to fix it ASAP.
When the data is retrieved from a SQL db, there are no problems.
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] getting grossing data
On Sun, Nov 13, 2011 at 15:00, Zsolt Ero zsolt@gmail.com wrote:
Yesterday I finally figured out how to get the grossing data out.
Good. :-)
match = re.match(r\$([1-9][0-9,]+), movie['business']['gross'][0])
gross = match.group()[1:]
grossint = int(gross.replace(',', ''))
Ok, but keep in mind that the currency may be English pound
or anything else, also... (and I think it can be before the value
or even after)
Do you say that you have implemented this now in the SVN?
No, the previous code introduced in the list things that were not
business information at all: I just stripped those.
Also, can you help me how to get the English title of a movie, what is
listed on the www site not on the aka site?
Hmmm... do they ever differ? Do you have an example?
Or to figure out which is the www title from the akas list?
I'd probably need to do some tests (and have at least an example
to work on).
Maybe you can use the list of akas and the 'guessLanguage' of
the Movie instances (it tries to guess the language of the title/movie),
but I'm not too sure.
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
RSA(R) Conference 2012
Save $700 by Nov 18
Register now
http://p.sf.net/sfu/rsa-sfdev2dev1
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql: KeyError: md5sum
On Tue, Nov 15, 2011 at 16:33, Frederik Vaassen frederik.vaas...@gmail.com wrote: Hi, I'm attempting to transform the IMDb text files (acquired from one of the mirrors today) into a mySQL database using imdbpy2sql Hi! Are you working on a brand new database, or you're using one where there was an older version of IMDbPY? Can you provide the whole command line that you've used? Thanks for the report! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Retrieving actor info set from Person
On Thu, Nov 17, 2011 at 07:10, Nicholas Cafarchio cafni...@pct.edu wrote: I'm attempting to create a script that lets you find the connections between two actors (i.e. through movies and actors they have worked with, how are they connected, 6 Degrees of Kevin Bacon?). Retrieving the information is easy enough, but the logic that looks for the connections is slightly more complicated than I can handle in Python, so, being a Java programmer, I decided to use Jython. Here's where I run in to some issues, and I'm fairly confused to say the least. Right, 'cause the task was not already difficult enough, so better put jython into the equation! Makes sense... :P 5a. When I call i.update(bacon) in the interactive python shell, the 'actor' info set is now listed as an info set of bacon 5b. When I call pi.exec(i.update(bacon)); inside my Jython script, about 1/4 of the info sets that should be returned do not show up when you call .keys(), 'actor' being one of them. :-/ 6b. Using http instead of mobile in my Jython script gets me some of the info sets, but still not 'actor'. 7b. Even though I import sys and append the path to lxml to path, the script always falls back to beautifulsoup, and it takes forever. This doesn't happen in the interactive shell. lxml is (partially) a C python module. I don't remember how Jython works with these. So, now that the background is out of the way, does anyone know why I'm getting different results in the shell and the script? Better yet, does anyone know how to fix it? If someone can help me fix the lxml thing, that would be helpful, but it's not a primary concern. My only idea is crazy and already disproved by a fast test I made: while I did swear some time ago that the keys used by the Movie/Person/... classes to store information are strings (besided the few cases where they are instances of other objects), the truth is that many times they are unicode. Not that it could (should?) create any problem, but I noticed that for 'mobile' some are strings and some are unicode (u'actor' is unicode) The same is true for 'http', but in that case 'actor' is a string... :-/ Another absolutely crazy option: is it possible that in the environments you're using (jython and the normal python), there's a proxy used in one an not in the other? (i.get_proxy() should tell you). And, by the way, how does Jython and its PythonInterpreter share memory, if they do? i is a very common variable name... ;-) Ok, I'm officially out of ideas. :D -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Error returned when running imdbpy2sql.py with MySQL database
On Sat, Nov 19, 2011 at 20:17, Cody Bohn codybohnh...@gmail.com wrote: I am getting this issue when I run imdbpy2sql.py with the proper arguments with a mysql scheme and I can't seem to figure out the issue. I posted on Stack Overflow but I only managed to resolve part of the problem and this is what remains. Hi! Looks like you've some serious problem with dependencies. Can you tell us how and where you're installing IMDbPY, and what's your environment? It whould help to know: 1. your distribution, 2. if you're installing IMDbPY from the tar.gz, or if you're using a package or you're installing it using easy_install or pip. 3. are you installing it in a virtualenv? Also, check to not have other versions of IMDbPY installed on your system. If you're using easy_install, please be sure to have only one executable installed in your path. Thanks for the report! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Adding foreign keys fail with postgresql database
On Wed, Dec 14, 2011 at 20:58, Davide Alberani davide.alber...@gmail.com wrote: Thank you very much for the detailed report. I think to have fixed it in the Mercurial repository: the problem was created by the fact that right now, the crazy credits doesn't contain any reference to movie titles. :-/ I hope they will fix this soon. Thank you very much for your report - I'll add your name to the credits. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql - some movies are missing nr_order for the cast
On Tue, Dec 13, 2011 at 19:58, Davide Alberani davide.alber...@gmail.com wrote: However, this doesn't explain your 19 and 0... :-/ My db is not up-to-date: I'll recreate it tomorrow and see the result of your queries. Pardon for the delay. I have the same data, and they are from the 'writer.list' file. Most probably, they are wrong submissions or have some arcane meaning for the IMDb database. :-) Happy holidays! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Write once. Port to many. Get the SDK and tools to simplify cross-platform app development. Create new or port existing apps to sell to consumers worldwide. Explore the Intel AppUpSM program developer opportunity. appdeveloper.intel.com/join http://p.sf.net/sfu/intel-appdev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDB database creation
On Wed, Feb 15, 2012 at 15:39, Márton Szabó habaku...@gmail.com wrote: First of all thanks for IMDbPY it works great. Thanks. :-) I wanted to play with the IMDB UNIX search programs, that can be found here: http://www.imdb.com/interfaces Wow! They are a little... demodé. :-) I don't really expect them to work on a recent set of data: since some time all the titles in the plain text data files are listed in the The Title format, while previously they were Title, The. We supported the output of moviedb up to IMDbPY 4.1, after that the changes were so many that it was no more worth the effort (and the 'sql' method works much better). I was able to compile the programs, but I couldn't create the databases. Lots of .list files failed to convert to .data/.names/.titles files with error codes like these: /usr/local/moviedb-3.24/etc/mkdb -movie Adding Movies List... make[3]: *** [movies.data] Bus error Here's the most important excerpt from my old README.local: NOTE: the current (3.24) moviedb version is old an it was not thought with tv series episodes support in mind. It can still work very well, but you've to modify some constants in the code: edit the moviedb.h file in the src directory, and change MAXTITLES to _at least_ 160, MAXNAKAENTRIES to 70, MAXFILMOGRAPHIES to 20470, LINKSTART to 100 and MAXBIOENTRIES to 50. Also, setting MXLINELEN to 1023 is a good idea. See http://us.imdb.com/database_statistics for more up-to-date statistics. You can read a complete copy here: http://erlug.linux.it/~da/erlugtmp/README.local The current version: https://bitbucket.org/alberanid/imdbpy/src/74e6f583f9cf/docs/README.local If you need the other tools we developed to use these data, you can download IMDbPY 4.1 from http://sourceforge.net/projects/imdbpy/files/IMDbPY/4.1/ but I guess that at this point you've changed your mind. :-P HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] install fails
On Sun, Mar 18, 2012 at 18:50, Jan Behrendt jan.behre...@gmail.com wrote: Hi all! I've a problem with the installation. Could you help me please? root@ubuntu:/home/ubuntu/imdbpy/IMDbPY-4.8.2# python ./setup.py /usr/lib/python2.7/distutils/dist.py:267: UserWarning: Unknown distribution option: 'zip_safe' warnings.warn(msg) I admit that this error is somewhat strange, and I'm unable to reproduce it (python 2.7, ubuntu 11.10 - but distutils doesn't seem to be changed). Are you sure to have the python-setuptools package installed, and that there are no other packages/modules with the same name installed somewhere else on the system? Anyway, the command to install it should be: python ./setup.py install Let us know if this fixes the problem. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] timing out
On Thu, Mar 22, 2012 at 09:29, Tomáš Hnyk tomash...@gmail.com wrote: Thanks a lot, that will be great:-). Ok, in the mercurial repository you can set the 'reraiseExceptions' argument (of the IMDb() instance or in the imdbpy.cfg file, if you use it) to True, to re-raise every caught exception. By default, it's false. Thanks for the idea, I've added your name to the credits. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] timing out
On Sat, Mar 24, 2012 at 11:51, Tomáš Hnyk tomash...@gmail.com wrote: and it has been hanging at this for about two minutes now and counting. Should not it raise some sort of exception, or is it going to wait for something indefinitely? :-/ It's very possible that it will wait until the end of time, I fear. I was originally asking if there were a way to set a timeout to retrieving film info. Is there some internal timeout in imdbpy after which it justs gives up? My fault: I misunderstood the problem. No, IMDbPY uses urllib, and it doesn't provide a way to set a timeout. urllib2 has it, but only in very recent python versions, so I'd prefer not to use it (and, by the way, a conversion from urllib to urllib2 would take me some time). I found a way to do this on in internet: import signal Yes, it's more or less reasonable, but I want to think twice about it, before I introduce an alarm in the IMDbPY code... after all i's a library and it could heavily interfere with signal handling made by the programs that uses it. :-( Real point is, it's very strange that it happens so methodically on your system... I think I've never seen it hangs during a request. Anyway, if you can try to put _before_ IMDb is imported/instanced this: import socket socket.setdefaulttimeout(10) With that, at least the 'open' should fail after at most 10 seconds. If this doesn't work for you, it means that the connection is established, but hangs (or is really slow) reading the data. If it's so, I fear an alarm-based solution is the only way... :-/ Thanks for the debugging! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Episode information of not available
On Sat, Mar 24, 2012 at 14:44, David Kaufman david.kauf...@gmx.de wrote: I was looking if I could retreive episode information through imdbpy, but was unable to do so: Recently the episode pages changed a lot. A fix is already available on the Mercurial repository: https://bitbucket.org/alberanid/imdbpy/ Let me know if you encounter other problems! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] SQL Server Express 2008 R2 - Connection Error
On Mon, Mar 26, 2012 at 09:25, Damien damien.stew...@profero.com wrote: Traceback (most recent call last): File imdbpy2sql.py, line 519, in module connectObject = conn.getConnection() File C:\Python27\lib\site-packages\sqlobject\dbconnection.py, line 336, in g etConnection conn = self.makeConnection() File C:\Python27\lib\site-packages\sqlobject\mssql\mssqlconnection.py, line 94, in makeConnection con = self.dbconnection( *self.make_conn_str(self) ) File pymssql.pyx, line 552, in pymssql.connect (pymssql.c:7162) pymssql.InterfaceError: Connection to the database failed for an unknown reason. Any ideas? First of all, pardon for the long delay. :-/ Hmmm... not much, not knowing enough of SQL Server. Maybe you can just try interfacing with it using SQLAlchemy, instead of the default SQLObject. Try adding the -o sqlalchemy argument to the imdbpy2sql.py command line... HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- This SF email is sponsosred by: Try Windows Azure free for 90 days Click Here http://p.sf.net/sfu/sfd2d-msazure ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] SQL Server Express 2008 R2 - Connection Error
On Tue, Apr 3, 2012 at 05:42, Damien damien.stew...@profero.com wrote: File pymssql.pyx, line 549, in pymssql.connect (pymssql.c:7110) sqlalchemy.exc.OperationalError: (OperationalError) (20009, 'Net-Lib error durin g Unknown error') None None [...] I have created a blank DB in SQL Server which I can connect to okay using SQL Server Management Studio. I would expect the error messages to be a little more helpful, perhaps stating the login is incorrect or something - but the last few lines seem to indicate something is broken :( Yep. :-( Sorry, but I know next to nothing about databases on a Windows environment, so I can't be of any help on this. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- For Developers, A Lot Can Happen In A Second. Boundary is the first to Know...and Tell You. Monitor Your Applications in Ultra-Fine Resolution. Try it FREE! http://p.sf.net/sfu/Boundary-d2dvs2 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Want to create xml files for every type of supported imdbpy object from mysql database
On Sat, Apr 28, 2012 at 00:48, Benjamin benjaminbondt...@gmail.com wrote:
I've followed the instructions and populated a mysql db w/ the info from the
imdb flat files.
I want to output from this db an xml file for every single imdbpy object
type(Movie, Person, etc.).
Is there a way to do this?
Sort of, but I can't guarantee you that it will not be painfully slow. :-)
First of all, you have to know the maximum ID of the object
you're considering (movies, persons, ...)
For movies, you can use something like:
SELECT MAX(id) FROM title;
After that you can write a Python script to extract the information.
Something like this (beware that it's mostly pseudo-code) should work:
from imdb import IMDb
ia = IMDb('sql', uri='mysql://USERNAME:PASSWORD@localhost/imdb')
for idx in xrange(idx, MAX_ID+1):
# get the movie object.
movie = ia.get_movie(idx)
# fetch all other information, if you need them.
# ia.update(movie, 'all')
# or maybe you're only interested in goofs:
# ia.update(movie, 'goofs')
# get the XML representation, and do what you want with it.
xml = movie.asXML()
# YOUR XML PROCESSING HERE
You better put all the code inside the 'for' cycle in a try/except clause,
since a lot of things can go wrong... just skip to the next item.
A recent suggestion that I want to implement is a JSON output,
but I still had no time to look at it. :-(
HTH
--
Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47]
http://www.mimante.net/
--
Live Security Virtual Conference
Exclusive live event will cover all the ways today's security and
threat landscape has changed and how IT managers can respond. Discussions
will include endpoint security, mobile security and the latest in malware
threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/
___
Imdbpy-help mailing list
Imdbpy-help@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] help running imdbpy2sql
On Thu, May 24, 2012 at 11:14 PM, Brian Abelson br...@harmony-institute.org wrote: 4. Downloaded the stable version of IMDbPY - 4.8.2 If possible, use the development version: https://bitbucket.org/alberanid/imdbpy/ Traceback (most recent call last): File bin/imdbpy2sql.py, line 34, in module from imdb.parser.sql.dbschema import * ImportError: No module named imdb.parser.sql.dbschema You can't use the downloaded file without installing them. You can try to run, from the main directory of the IMDbPY package, this: python setup.py install it should take care of all the dependencies, provided that you have a working environment to compile the needed libraries and/or that they are pre-packaged for MacOS X (I don't know, since I don't use it). Basically, the point is: - try to install it (it will try to fetch some dependencies and compile some modules) - if some dependency fails, try to satisfy it manually. - if there's no way and/or the compile step stops you, see the options shown by the setup.py file to exclude some pieces of IMDbPY. It would be cool to have an auto-installer for Mac OS X or at least the steps to follow to manage the installation of python modules; is anyone out there expert on the subject? :-) HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] workaround for imdbpy2sql.py slowness
Hi all, lately I've got some reports about a huge slowdown of the imdbpy2sql.py script, especially on Ubuntu 12.04, with both MySQL and Postgresql. Maybe even other databases. I'm not too sure about what the problem is, but the workaround seems easy: do not use the default SQLOjbect, but install SQLAlchemy and add to the command line this argument: -o sqlalchemy If someone knows about the reason and has any idea about a real fix (maybe it's just an option that must be added to recent versions of SQLObject), please let me know. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.9 released
Hi all, I've just released IMDbPY 4.9, containing the fix for search queries and all the latest fixes and improvements. It can be downloaded from http://imdbpy.sf.net/ As usual, let me know of any problems. Enjoy, and thanks to anyone for the help! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy2sql.py problem on WINDOWS xp--hanging on FLUSHING MoviesCache
On Sat, Jun 30, 2012 at 9:31 AM, qiuxia lu luqiu...@gmail.com wrote: I am running imdbpy2sql.py using IMDbPY 4.9. It hangs on FLUSHING MoviesCache for hours, is that normal? The message is as follows: Hi, it seems to be a known problem with recent versions of SQLObject. Until we find a solution, as a workaround you can install SQLAlchemy and use it adding the -o sqlalchemy argument to the command line. I think I'll add a warning about SQLObject in the code... Let us know if it fix your problem. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] About imdbPY plugin.
On Tue, Jul 10, 2012 at 4:09 PM, Venom Panic venom.pa...@yahoo.com wrote: Is there a way to integrate IMDbPy in Wordpress (as plugin, maybe or some)? Not that I'm aware of. It's hard to do that, can you help me, please? I don't know how hard it can be... I guess it should be quite easy, given how many wordpress plugins exist. :-) If you don't find any help here, I suggest to try asking on some wordpress development mailing lists. Good luck and let us know if you release it publicly! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
