[Imdbpy-help] IMDbPY 2021.04.18
Hi, thanks to many contributors we are releasing IMDbPY 2021.04.18. It can be downloaded from https://github.com/alberanid/imdbpy/releases/tag/2021.04.18 As always, any contribution to the code is welcome. * What's new in release 2021.04.18 (Black Sails) [general] - #289: Serbian translations - #290: update Portuguese translations - #291: allow direct access to filmography keys - #299: add pagination to get_movie_list - #300: any iterable can be used as season_nums parameter of update_series_seasons - #305: fix ability to set logging level - #310: fix build of locales .mo files - #312: extract localized title [sql] - #287: fix missing logger -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 2020.09.25
Hi, I hope everyone is doing fine in this complicated timeline. I've just released IMDbPY 2020.09.25, since the last official release was over a year old. There are many fixes and some improvements, and we also switched to a calendar versioning schema. All the best. -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 6.8
Hi all, I had some spare time, so I released IMDbPY 6.8. In this version: - #224: introduce the search_movie_advanced(title, adult=None, results=None, sort=None, sort_dir=None) method - #145: names are stored in normal format (Name Surname) - #225: remove obsolete cookie - #182: box office information - #168: parse series and episode number searching for movies - #217: grab poster from search - #218: extract MPAA rating - #220: extract actor headshot from full credits As always, IMDbPY is looking for developers; if you want to contribute, contact us! -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 6.7 released
Hi, it's been a long since the last release, so I've published 6.7: https://imdbpy.sourceforge.io/ This version is dedicated to all Game of Thrones fans waiting for the end of the show. ;-) There are still a lot of fixes and improvements to be done; if anyone wants to contribute, please look at https://github.com/alberanid/imdbpy/issues For the next release, I hope we can switch to a calendar-based versioning schema as suggested by Uyar https://github.com/alberanid/imdbpy/issues/112 and we'll probably have a new homepage. Best regards, -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPy Question
Hi Jacob, unfortunately that feature is missing: we do not parse reviews. To make things more complex, a page like this https://www.imdb.com/user/ur2467618/reviews shows only a small amount of data, and more is loaded dynamically on request. So... I fear we'll never support this, sorry. On Tue, Nov 13, 2018 at 7:57 PM Jacob Higgins wrote: > > Hi, > > My name is Jacob, and I was wondering: is there a way to access an individual > user's reviews for different movies? For example, can I use IMDbPy to access > the score of every movie rated by imdb_user1978 (a made-up user)? > > I've been playing around with a project where I rate movies on my own, then > go through top users of IMDb to try and find people who have the same taste > in movie as I do. That way, whenever a new movie comes out I can see their > reviews and determine if I would enjoy it! > > I am a big-time amateur at this kind of stuff, so its easy for me to get lost > in the documentation trying to find what I 'm looking for. > > Thanks! > Jacob Higgins > > ___ > Imdbpy-help mailing list > Imdbpy-help@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/imdbpy-help -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPy Question
Hi, My name is Jacob, and I was wondering: is there a way to access an individual user's reviews for different movies? For example, can I use IMDbPy to access the score of every movie rated by imdb_user1978 (a made-up user)? I've been playing around with a project where I rate movies on my own, then go through top users of IMDb to try and find people who have the same taste in movie as I do. That way, whenever a new movie comes out I can see their reviews and determine if I would enjoy it! I am a big-time amateur at this kind of stuff, so its easy for me to get lost in the documentation trying to find what I 'm looking for. Thanks! Jacob Higgins ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMdbPY 6.6
Hi everyone, IMDbPY 6.6 is out! A lot of parsers fixed, a solution for the certificate verification problems and improvements to the new dataset. Plus, now our tests are automatically run at each commit using https://travis-ci.org/alberanid/imdbpy Enjoy the summer, -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY on readthedocs
Hi all, I've just added IMDbPY to readthedocs: https://imdbpy.readthedocs.io/ The build was done in a full-automated fashion, so maybe something is missing or not correctly organized. For the moment, it's better than nothing. :-) The docs should be automatically updated with new commits. Hope this helps, -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 6.5
Hi all, I've released IMDbPY, which comes with two main features, both developed by H. Turgut Uyar: - the web parsers are now based on hi project piculet - the documentation was converted to the sphinx format Besides this, some minor fixed. Enjoy it, -- Davide Alberani[PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPY 6.4
Hi all, packaged and released version 6.4.1 to remove some debug code that I forgot. Enjoy, On Wed, Mar 14, 2018 at 10:37 PM, Davide Alberaniwrote: > Hello, > We have just released IMDbPY 6.4 to fix some serious problems > with the search parsers: https://imdbpy.sourceforge.io/downloads.html > > As always, IMDbPY is looking for developers; if you want to > contribute, let us know. :-) > > Thanks to anyone involved! > > -- > Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] > http://www.mimante.net/ -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 6.3
Hi all, I've just release IMDbPY 6.3. While the development on the new web pages of IMDb is probably far from complete, I feel confident that the current set of features are already good enough. Plus, in this release, there's support to the new dataset distributed by IMDb: https://www.imdb.com/interfaces/ The new version can be downloaded from https://imdbpy.sourceforge.io/ Enjoy it! -- Davide Alberani[PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [imdbpy-help] After i.update(movie) wrong (and always the same) title is returned
I suppose you already know about this, but just in case, I noticed that imdbPY isn't returning info for 'producer', 'cinematography' and 'editor'. I didn't change my code, which was working previously. If I have to change something, please tell me. On 1 January 2018 at 18:52, Davide Alberaniwrote: > Hi all, > yes: since a recent redesign of the web pages, IMDbPY is badly broken. > > We started working on master to fix it, but there's still much to do; > see https://github.com/alberanid/imdbpy/issues/103 > > As always, any help is welcome. > > If you want to start fixing something, run the tox and chose one of > the parsers you want to work on (and tell us on that issue, so that we > can avoid duplicate work). > > Thanks! > > > On Sat, Dec 30, 2017 at 12:08 PM, Filip Bačić > wrote: > > Hi, > > > > Lately, after i.update(movie), I always get the same movie title "The > > leading information resource for the entertainment industry". > > I used some old imdbpy package, but I also now tried it with the latest > one > > and I am getting the same result. > > > > For example, for this code: > > > > import imdb > > > > i = imdb.IMDb() > > > > movie_list = i.search_movie('pacific rim') > > > > first_match = movie_list[0] > > > > print(first_match) > > > > i.update(first_match) > > > > print(first_match) > > > > I get this output: > > > > Pacific Rim > > The leading information resource for the entertainment industry > > > > Does anyone else have this problem? > > > > Best Regards, > > > > Filip > > > > -- > > You received this message because you are subscribed to the Google Groups > > "imdbpy-users" group. > > To unsubscribe from this group and stop receiving emails from it, send an > > email to imdbpy-users+unsubscr...@googlegroups.com. > > For more options, visit https://groups.google.com/d/optout. > > > > -- > Davide Alberani [PGP KeyID: > 0x3845A3D4AC9B61AD] > http://www.mimante.net/ > > > -- > Check out the vibrant tech community on one of the world's most > engaging tech sites, Slashdot.org! http://sdm.link/slashdot > ___ > Imdbpy-help mailing list > Imdbpy-help@lists.sourceforge.net > https://lists.sourceforge.net/lists/listinfo/imdbpy-help > -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [imdbpy-help] After i.update(movie) wrong (and always the same) title is returned
Hi all, yes: since a recent redesign of the web pages, IMDbPY is badly broken. We started working on master to fix it, but there's still much to do; see https://github.com/alberanid/imdbpy/issues/103 As always, any help is welcome. If you want to start fixing something, run the tox and chose one of the parsers you want to work on (and tell us on that issue, so that we can avoid duplicate work). Thanks! On Sat, Dec 30, 2017 at 12:08 PM, Filip Bačićwrote: > Hi, > > Lately, after i.update(movie), I always get the same movie title "The > leading information resource for the entertainment industry". > I used some old imdbpy package, but I also now tried it with the latest one > and I am getting the same result. > > For example, for this code: > > import imdb > > i = imdb.IMDb() > > movie_list = i.search_movie('pacific rim') > > first_match = movie_list[0] > > print(first_match) > > i.update(first_match) > > print(first_match) > > I get this output: > > Pacific Rim > The leading information resource for the entertainment industry > > Does anyone else have this problem? > > Best Regards, > > Filip > > -- > You received this message because you are subscribed to the Google Groups > "imdbpy-users" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to imdbpy-users+unsubscr...@googlegroups.com. > For more options, visit https://groups.google.com/d/optout. -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPY revamp
On Wed, Nov 1, 2017 at 3:02 PM, Davide Alberaniwrote: > > as many of you know, IMDbPY is in need of a revamp. :-) A quick update: I've just merged back into master the many changes of the "codename-simply" branch (which should now be considered closed; I'll delete it soon). The old version, suitable for Python 2.7, is available in the "imdbpy-legacy" branch, and probably will receive very little updates from now on. Main changes: - Python 3 support (and only Python 3: no Python 2.7 compatibility, sorry) - removed the 'mobile' set of parsers - removed dependencies: SQLObject, C compiler, BeautifulSoup - introduced a testsuite, please help with it: https://sourceforge.net/p/imdbpy/mailman/message/36107729/ I want to thanks all the contributors, and especially H. Turgut Uyar for such a huge amount of work! I hope to be able to update the website and pypi tomorrow. There are for sure many many bugs, please help and report them. -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY revamp
Hi all, as many of you know, IMDbPY is in need of a revamp. :-) So, while I (again and again) have very little time to devote to it, I try to slowly improve it. Right now I've created a "codename-simply" branch, with the intent of reducing the amount of legacy code and some of the oddities of my previous choices: https://github.com/alberanid/imdbpy/tree/codename-simplify See also issue https://github.com/alberanid/imdbpy/issues/61 My plan is more or less as follow: * remove the "mobile" parser (done) * remove SQLObject support (done) * remove cutils, the utilities written in C (done, not sure it will not be useful again in the future) * introduce support for the new data set (to be done: https://github.com/alberanid/imdbpy/issues/60 ) * move to Python 3 (to be done: https://github.com/alberanid/imdbpy/issues/27 ) Another possible point is: * remove the BeautifulSoup dependency (python-lxml will be required) but on this I wait the opinion of Turgut, the main author of that code. The rationale is to remove unneeded dependencies (like the old SQLObject). For the moment I've set lxml as a mandatory dependency, but I can revert it to an optional one. It has to be said that _bsoup is shipped with our package, so maybe we can leave it there. After this little clean-up, I'd like to work, in this order on: 1. the switch to Python 3 2. the new dataset, using SQLAlchemy (unless there are strong opinions and helping hands to switch to a no-SQL db) If you have other ideas and/or if you want to help, let us know. :-) -- Davide Alberani[PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] Imdbpy help
Hi Davide, Thank you for creating imdbpy. When I tried to retrieve box office gross by ia.update(m,'business'), the key business is always empty. Could you tell me what I should do to get box office info and budget info? Thank you! Sincerely, Qianyu Cheng -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPY
On Sat, Jul 8, 2017 at 11:41 PM, Anthony Pecorarowrote: > > Just wondering if there is a list of data points that are retrievable using > IMDbPY. Specifically, wondering if there is a way to pull by user rating, or > number of users who rated a film. Hi Anthony, not exactly: data are grouped in 'info sets' based on the web page (or table) you can find them on, and a simple page/table can obviously contains a lot of information. Let's say that we have the usual ia handler: from imdb import IMDb ia = IMDb() You can now know which info sets are available: ia.get_movie_infoset() (there are matching methods for company, person, character) Now, let's fetch information about a movie: m = ia.get_movie('0078748') You can add a new info set to the available information with: ia.update(m, 'trivia') To know which info set are now available: m.current_info You can also associate info sets to keys: m.infoset2keys (unfortunately, you can't do it before fetching a data set. I.e.: there's no fixed association between info sets and keys) That's more or less everything. Hope this helps, -- Davide Alberani [PGP KeyID: 0x3845A3D4AC9B61AD] http://www.mimante.net/ -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY
Hi, Just wondering if there is a list of data points that are retrievable using IMDbPY. Specifically, wondering if there is a way to pull by user rating, or number of users who rated a film. Thanks, Anthony -- Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] imdbpy installation problem
hi, i’m sorry writing this email to bother you, but i really need help about how to install imdbpy. I’m using MAC Air with MacOS Sierra, python 3.5 yesterday, I’m trying to use “pip install imdbpy” to install imdbpy, but its failed because some statements not fit for python3.5. then i fixed these statements, but when i use “python ./setup.py” to install imdbpy, it shows " ERROR = Aaargh! An error! An error! Curse my metal body, I wasn't fast enough. It's all my fault! Anyway, if you were trying to build a package or install IMDbPY to your system, looks like we're unable to fetch or install some dependencies, or to compile the C module. The best solution is to resolve these dependencies (maybe you're not connected to Internet?) and/or install a C compiler. You may, however, go on without some optional pieces of IMDbPY; try re-running this script with the corresponding optional argument: --without-lxml exclude lxml (speeds up 'http') --without-cutils don't compile the C module (speeds up 'sql') --without-sqlobject exclude SQLObject (you need at least one of) --without-sqlalchemy exclude SQLAlchemy (SQLObject or SQLAlchemy,) (if you want to access a ) (local SQL database ) --without-sql no access to SQL databases (implied if both --without-sqlobject and --without-sqlalchemy are used) Example: python ./setup.py --without-lxml --without-sql install The caught exception, is re-raise below: error in IMDbPY setup command: access to SQL databases wants to be able to remove imdb.parser.sql, but the distribution doesn't contain any packages or modules under imdb.parser.sql” i even used python ./setup.py --without-lxml --without-sql install to try again, still same problem. i installed lxml, cutils, sqlobject even sqlalchemy, but always shows this problem. what should i do to install imdbpy? thank you every much and loo forward your reply David HAN -- Check out the vibrant tech community on one of the world's most engaging tech sites, SlashDot.org! http://sdm.link/slashdot___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] imdbpy install errors with SQLAlchemy version
i was trying to install imdbpy and ran into some errors any help would be so helpful . iam attaching the error log thanks in advance karthik Installed /usr/local/lib/python2.7/dist-packages/IMDbPY-5.0-py2.7-linux-x86_64.egg Processing dependencies for IMDbPY==5.0 ERROR = Aaargh! An error! An error! Curse my metal body, I wasn't fast enough. It's all my fault! Anyway, if you were trying to build a package or install IMDbPY to your system, looks like we're unable to fetch or install some dependencies, or to compile the C module. The best solution is to resolve these dependencies (maybe you're not connected to Internet?) and/or install a C compiler. You may, however, go on without some optional pieces of IMDbPY; try re-running this script with the corresponding optional argument: --without-lxmlexclude lxml (speeds up 'http') --without-cutils don't compile the C module (speeds up 'sql') --without-sqlobject exclude SQLObject (you need at least one of) --without-sqlalchemy exclude SQLAlchemy (SQLObject or SQLAlchemy,) (if you want to access a ) (local SQL database ) --without-sql no access to SQL databases (implied if both --without-sqlobject and --without-sqlalchemy are used) Example: python ./setup.py --without-lxml --without-sql install The caught exception, is re-raise below: error: Installed distribution SQLAlchemy 1.0.6 conflicts with requirement SQLAlchemy=0.7.8,!=0.9.5,=0.9.99 -- ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [imdbpy-help] Re: Installing IMDbPY on MAC OS X
On Sat, Mar 29, 2014 at 5:26 AM, Tony C cappy2...@gmail.com wrote: When I installed via pip, I didn't have this error, but a lot of imdb-py files were installed in the directory where I ran the pip command. I didn't like this, so I used pip to uninstall everything. I'll try to package the latest version as soon as possible (within a month?) even if it's far from a good working state. In the meanwhile, can't you install SQLAlchemy separately, on your system? -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDbPY search status
Sameer, Version 5 is available directly from the source repositories, I believe that Davide just hasn't uploaded 5.0 to pypi. On 19 April 2013 02:14, Sameer Indarapu sameer.indar...@gmail.com wrote: David, Any updates on when 5.0 will be available? Thanks, Sameer On Monday, December 31, 2012 4:00:11 AM UTC-8, Davide Alberani wrote: Hi all, I think that now most of the issues related to web search (both 'http' and 'mobile') are fixed. The code is not nice, and a lot of obsolete lines are left there, but it should handle most of the cases. Please report any problem you may find. I'd like to release 5.0 within Jan 2013; is there any other urgent issue left open? PS: recently, I've also fixed the gateway between these mailing lists and the google groups, that stopped worked some months ago. Happy new year! -- Davide Alberani davide@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ --**--**-- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_**122412http://p.sf.net/sfu/learnmore_122412 __**_ Imdbpy-devel mailing list Imdbpy...@lists.**sourceforge.net https://lists.sourceforge.net/**lists/listinfo/imdbpy-develhttps://lists.sourceforge.net/lists/listinfo/imdbpy-devel -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy to mysql help
On Sun, Feb 24, 2013 at 12:32 AM, D L dlm...@hotmail.com wrote: Ok, well here's an update. I just let the foreign keys run for a little over a full day and it actually completed for mysql: # TIME FINAL : 1883min, 1sec (wall) 23min, 57sec (user) 0min, 5sec (system) I see. I've just run it with a subset of the db (1% taken from each file) and my numbers are: # TIME TOTAL TIME TO INSERT/WRITE DATA : 12min, 18sec (wall) 5min, 23sec (user) 0min, 43sec (system) building database indexes (this may take a while) # TIME createIndexes() : 1min, 25sec (wall) 0min, 0sec (user) 0min, 0sec (system) adding foreign keys (this may take a while) # TIME createForeignKeys() : 10min, 2sec (wall) 0min, 0sec (user) 0min, 0sec (system) RESTORING imdbIDs values for movies... DONE! (restored 0 entries out of 0) # TIME restore movies : 0min, 0sec (wall) 0min, 0sec (user) 0min, 0sec (system) RESTORING imdbIDs values for people... DONE! (restored 0 entries out of 0) # TIME restore people : 0min, 0sec (wall) 0min, 0sec (user) 0min, 0sec (system) RESTORING imdbIDs values for characters... DONE! (restored 0 entries out of 0) # TIME restore characters : 0min, 0sec (wall) 0min, 0sec (user) 0min, 0sec (system) RESTORING imdbIDs values for companies... DONE! (restored 0 entries out of 0) # TIME restore companies : 0min, 0sec (wall) 0min, 0sec (user) 0min, 0sec (system) # TIME FINAL : 23min, 45sec (wall) 5min, 23sec (user) 0min, 43sec (system) What kind of CPU/RAM/disk have you used? One of my main questions right now is the difference in results between the web search and the sql search. For example, if I ran a search on all the movies that Denzel Washington has acted in via the web search, it basically outputs all the main ones, Yep, they are just grouped in a different way. It would be not easy for us (even if it's not impossible, I guess) to identify alle the various categories used on the web and the rules used to categorize the movies, but... For the moment, I think you could take the whole filmography and search for tv series and/or movies in which an actor is playing Himself (or anything that starts with Himself/Herself/Themselves) And I haven't tested it that much, but it appears that sqlite and mysql have roughly the same speeds in running these queries, but I'm not completely sure yet. I expect them to be comparable in speed, but not to be slower than a web search. :-/ -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy to mysql help
Ok, well here's an update. I just let the foreign keys run for a little over a full day and it actually completed for mysql: # TIME FINAL : 1883min, 1sec (wall) 23min, 57sec (user) 0min, 5sec (system) One of my main questions right now is the difference in results between the web search and the sql search. For example, if I ran a search on all the movies that Denzel Washington has acted in via the web search, it basically outputs all the main ones, whereas if I do it via the sql search it will include a lot of random stuff like award ceremonies and random tv shows that he may have had a cameo on. How would I make the sql search more like the web search so that it excludes stuff like award ceremonies and only outputs the main movies? And I haven't tested it that much, but it appears that sqlite and mysql have roughly the same speeds in running these queries, but I'm not completely sure yet. From: dlm...@hotmail.com To: davide.alber...@gmail.com; imdbpy-help@lists.sourceforge.net Subject: RE: [Imdbpy-help] imdbpy to mysql help Date: Fri, 22 Feb 2013 00:10:28 -0800 So after updating those dependencies, the MySQL still gets stuck on the foreign keys section, however sqlite actually manages to finish. But one of my concerns is that even the requests with sqlite can be slow the first time, and on occasion the web access was a lot faster than using the sqlite. For example, the search_person script is faster via the web, but if I run it twice (searching the same person) using the sql database, the 2nd time is noticeably much faster, most likely due to the data already being cached. My question is how fast does something like search_person take on MySQL (if I can eventually get it to work), since using sqlite seems like it's slower than just going the web route so far. From: dlm...@hotmail.com To: davide.alber...@gmail.com Subject: RE: [Imdbpy-help] imdbpy to mysql help Date: Tue, 19 Feb 2013 17:54:16 -0800 Date: Tue, 19 Feb 2013 21:28:18 +0100 Subject: Re: [Imdbpy-help] imdbpy to mysql help From: davide.alber...@gmail.com To: dlm...@hotmail.com CC: imdbpy-help@lists.sourceforge.net On Sun, Feb 17, 2013 at 11:45 PM, D L dlm...@hotmail.com wrote: Yeah tried that and ran it overnight, still no luck - it gets stuck on the foreign keys part. I'm just trying this on my laptop, so I may just proceed with using the web access for the data. Once I get everything set up for a web hosting, I may try other databases such as sqlite to see if that works. D'oh! :( Versions of: - IMDbPY - SQLAlchemy - SQLObject - MySQL - python-mysqldb - python-migrate ? IMDbPY - 5.0dev20130210 SQLAlchemy - 0.8.0b2 SQLObject - 1.3.2 MySQL - Server version: 5.5.29-0ubuntu0.12.04.1 (Ubuntu) python-mysqldb - 1.2.3 python-migrate - 0.7.2 Both my python-mysqldb and python-migrate were older versions, which I just updated as I typed this. I tried the process with sqlite a night ago and it was stuck on the foreign keys section as well, I will try it again now that mysqldb and migrate have been updated and hopefully it will work. I also wrote a rough script for the data retrieval using the webaccess method, and you're right it does take a while. Anyway, if you interrupt it while it's creating the foreign key, maybe you can try to see which were already created, and add the missing one following the scheme you can find in imdb/parser/sql/dbschema.py Anyway, obviously I'll try to reproduce the problem, since it's not nice at all. :-/ Hopefully, the updated mysqldb and migrate would fix it, but we'll see. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] imdbpy + imdbpykit newbie question - how to start it ?
Hi all Imdbpy installed, sql databse installed, i'm trying to start it with imdbpykit, corresponding with https://bitbucket.org/alberanid/imdbpykit/src/tip/docs/README.txt SGIUtils (http://pypi.python.org/pypi/WSGIUtils/) You can install this package via easy_install. Copy the sample file for imdbpykit.ini to any directory, comment out the section:main part for the Paste Script server and run the command:paster serve imdbpykit.ini as result: paster serve imdbpykit.ini /usr/local/lib/python2.6/dist-packages/paste/urlparser.py:104: DeprecationWarning: The global_conf argument to URLParser is deprecated; either pass in None or {}, or use make_url_parser DeprecationWarning) Starting server in PID 11907. Starting HTTP server on http://127.0.0.1:8085 Error - type 'exceptions.ImportError': cannot import name CONFIG 127.0.0.1 - - [27/Nov/2012:12:09:18 +0200] GET / HTTP/1.0 500 - - Lynx/2.8.8dev.5 libwww-FM/2.14 SSL-MM/1.4.1 GNUTLS/2.8.6 in lynx http://127.0.0.1:8085/ .. .. Module ?:21 in module import os from paste import wsgilib from imdbpykit.sitepage import CONFIG def urlparser_hook(environ): from imdbpykit.sitepage import CONFIG Module ?:25 in module from WebKit.Page import Page from paste.deploy import CONFIG from paste.url import URL from paste.deploy import CONFIG ImportError: cannot import name CONFIG CGI Variables HTTP_ACCEPT 'text/html, text/plain, text/css, text/sgml, */*;q=0.01' HTTP_ACCEPT_ENCODING 'gzip, compress, bzip2' HTTP_ACCEPT_LANGUAGE 'en' HTTP_HOST'127.0.0.1:8085' HTTP_USER_AGENT 'Lynx/2.8.8dev.5 libwww-FM/2.14 SSL-MM/1.4.1 GNUTLS/2.8.6' PATH_INFO'/' REMOTE_ADDR '127.0.0.1' REQUEST_METHOD 'GET' SERVER_NAME '127.0.0.1' SERVER_PORT '8085' SERVER_PROTOCOL 'HTTP/1.0' Configuration __file__ '/root/imdbpykit/imdbpykit.ini' here '/root/imdbpykit' maxtime '3600' output 'xml' pretty_print 'on' style'imdbpyweb' WSGI Variables application paste.session.SessionMiddleware object at 0xf1a850 paste.expected_exceptions[class 'paste.httpexceptions.HTTPException'] paste.httpexceptions paste.httpexceptions.HTTPExceptionHandler object at 0xe99810 paste.printdebug_listeners [cStringIO.StringO object at 0xfdfa40, open file 'stderr', mode 'w' at 0x7f325871c1e0] paste.recursive.forward paste.recursive.Forwarder from / paste.recursive.include paste.recursive.Includer from / paste.recursive.include_app_iter paste.recursive.IncluderAppIter from / paste.recursive.script_name '' paste.remove_printdebug function remove_printdebug at 0xf92578 paste.session.factorypaste.session.SessionFactory object at 0xfe1610 paste.throw_errors True paste.urlparser.base_python_name 'imdbpykit.web' wsgi process 'Multithreaded' URL: http://127.0.0.1:8085/ So, could you point me !? What i'm doing wrong ? Thanks Sergey -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.9 released
Hi all, I've just released IMDbPY 4.9, containing the fix for search queries and all the latest fixes and improvements. It can be downloaded from http://imdbpy.sf.net/ As usual, let me know of any problems. Enjoy, and thanks to anyone for the help! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.8 and new site released
Hi all, I've just released the long-awaited IMDbPY 4.8, with too many bug fixes to mention. Pardon for the slowdown in the development; I'm sure there are still many bugs and I'd like to see some fixes to some core pieces of code (after more than 7 years and almost 50 releases, it's probably a good idea to rewrite a function or two ;-) So, if anyone wants to help, let us know! With this release, we also have a shiny new web site, courtesy of Alberto Malagoli who kindly joined the development team (thanks and welcome aboard, Alberto!) As usual, you can download IMDbPY from: http://imdbpy.sf.net/ Enjoy! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- RSAreg; Conference 2012 Save #36;700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] Getting information about genres only
On Sun, Jul 31, 2011 at 09:58, Saravanan saravanan.thirumuruganat...@mavs.uta.edu wrote: I am trying to write some Ubuntu Unity lens using IMDBPy. Great! Let me know when it's done! For this purpose, I need only genre information. I took a look at the code and did not find any get_movie_genre kinda function. So currently, I am using ia.update(x,main) . Is there anyway to just the genre information? There's no way to get only the 'genres' key; you're doing right, asking for only the 'main' info set (after that, you can access the 'genres' key of the Movie object). If you're worried about performances, you can consider using the 'mobile' data access system (i.e.: ia = imdb.IMDb('mobile') ) and leaving the rest of your code untouched. It should be slightly faster, but... I notice right now that it fails to parse the 'genres' key. I have to fix it. :-) -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] Getting information about genres only
On Sun, Jul 31, 2011 at 13:38, Davide Alberani davide.alber...@gmail.com wrote: It should be slightly faster, but... I notice right now that it fails to parse the 'genres' key. I have to fix it. :-) Ok, fixed in the Mercurial repository and added your name to the credits. Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Got Input? Slashdot Needs You. Take our quick survey online. Come on, we don't ask for help often. Plus, you'll get a chance to win $100 to spend on ThinkGeek. http://p.sf.net/sfu/slashdot-survey ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] Imdbpy
Hello, I would love to use imdbpy in an add-on that I'm writing for HTPC program XBMC. The only issue that I'm having is that I'd prefer to only have 1 include file. Is this possible? Thanks. Zach Anderson -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] Imdbpy
On Tue, Jul 26, 2011 at 02:25, Zach Anderson zpanderso...@gmail.com wrote: I would love to use imdbpy in an add-on that I'm writing for HTPC program XBMC. The only issue that I'm having is that I'd prefer to only have 1 include file. Is this possible? I guess you can try to create an egg. From the source tree, you have to run the command: python ./setup.py bdist_egg After that you can install the egg on the target system with something like: easy_install name_of_the.egg I'm not sure that the current setup can suit your needs. Maybe you can try to set the zip_safe parameter in the setup.py file to True, instead of False. By the way, there's a known problem with the dependencies of SQLObjects, which depends on FormEncode. If the installation fails, try to run it again. If it fails again, manually install FormEncode before (or exclude it, if you don't need it). HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Magic Quadrant for Content-Aware Data Loss Prevention Research study explores the data loss prevention market. Includes in-depth analysis on the changes within the DLP market, and the criteria used to evaluate the strengths and weaknesses of these DLP solutions. http://www.accelacomm.com/jaw/sfnl/114/51385063/ ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPY User Support
Hi Davide, I figured out my friend's problems. The votes and rating information are stored in the movie_info_idx table, not the movie_info table... As for the cast info, it looks like maybe it was not imported into the database. I did not do the importing so I am not sure if anything went wrong (eg missed file). We'll investigate this more on our own and come back to the list if it seems like a library bug and not a silly mistake on our part. Thanks, Ben On Sun, Feb 20, 2011 at 11:08 AM, Davide Alberani davide.alber...@gmail.com wrote: On Sun, Feb 20, 2011 at 16:46, Te Kim tk...@ucsc.edu wrote: My group is using a SQL database to make these calls. Is this a bug or could there possibly be something wrong with our database? What you're trying to do should work. Are you sure to have downloaded and processed all the available plain text data files? Please download each available file (and be sure that they're not corrupted) and - if possible - keep track of the output of the imdbpy2sql.py script. In a Unix environment, use something like: imdbpy2sql.py -d /path/to/imdb_files/ -u db::/connection_params 21 | tee debug.txt Notice that you can somewhat speed-up the process creating temporary CSV files (see the README.sqldb for complete information - essentially you need to add a '-c /path/to/a/tmp/directory/' argument to the command line). In addition, I am unable to find ratings or votes in the database. The query: select * from movie_info mi join info_type it on mi.info_type_id = it.id where it.info = 'runtimes' group by mi.info; yields results, as well as in the case where it.info = 'genres'. However, if I change it.info to 'votes', 'cast', or 'rating', I get 0 rows. Well, 'cast' information are not there: they are in the cast_info table. Votes and rating should be present. What could be the reason for the missing information? Hard to tell: or you don't have some information, or something has gone wrong importing it. Unfortunately I'll be unable to do deeper tests on the database access system for another day or two, so in the meanwhile try to collect as much information as possible to spot the problem. HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPY User Support
On Sun, Feb 20, 2011 at 16:46, Te Kim tk...@ucsc.edu wrote: My group is using a SQL database to make these calls. Is this a bug or could there possibly be something wrong with our database? What you're trying to do should work. Are you sure to have downloaded and processed all the available plain text data files? Please download each available file (and be sure that they're not corrupted) and - if possible - keep track of the output of the imdbpy2sql.py script. In a Unix environment, use something like: imdbpy2sql.py -d /path/to/imdb_files/ -u db::/connection_params 21 | tee debug.txt Notice that you can somewhat speed-up the process creating temporary CSV files (see the README.sqldb for complete information - essentially you need to add a '-c /path/to/a/tmp/directory/' argument to the command line). In addition, I am unable to find ratings or votes in the database. The query: select * from movie_info mi join info_type it on mi.info_type_id = it.id where it.info = 'runtimes' group by mi.info; yields results, as well as in the case where it.info = 'genres'. However, if I change it.info to 'votes', 'cast', or 'rating', I get 0 rows. Well, 'cast' information are not there: they are in the cast_info table. Votes and rating should be present. What could be the reason for the missing information? Hard to tell: or you don't have some information, or something has gone wrong importing it. Unfortunately I'll be unable to do deeper tests on the database access system for another day or two, so in the meanwhile try to collect as much information as possible to spot the problem. HTH, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- The ultimate all-in-one performance toolkit: Intel(R) Parallel Studio XE: Pinpoint memory and threading errors before they happen. Find and fix more than 250 security defects in the development cycle. Locate bottlenecks in serial and parallel code that limit performance. http://p.sf.net/sfu/intel-dev2devfeb ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDbPY 4.7 released
On Wed, Jan 26, 2011 at 8:56 AM, H. Turgut Uyar u...@itu.edu.tr wrote: On 01/23/2011 05:12 PM, Davide Alberani wrote: Please contribute to the development with fixes and bug reports. Will development continue on the main fork or on the imdbpy_parsers2010 fork? Good question. :-) If there're no reason to do otherwise, I'd prefer to move the future development in the main repository (removing imdbpy_parsers2010). -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.7 released
Released after a long delay, and despite that in a hurry, IMDbPY 4.7 can be found here: http://imdbpy.sf.net/ This is a transitional release, after the recent redesign of the IMDb pages. A new account is used and the new pages are parsed; for sure there are still many bugs; please read the README.redesign file for other details. Please contribute to the development with fixes and bug reports. Enjoy! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Special Offer-- Download ArcSight Logger for FREE (a $49 USD value)! Finally, a world-class log management solution at an even better price-free! Download using promo code Free_Logger_4_Dev2Dev. Offer expires February 28th, so secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsight-sfd2d ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy 4.6 can't fetch person head shot (and who add IMDb to the names ?)
On Tue, Sep 28, 2010 at 10:50 PM, Israel Fruchter israel.fruch...@gmail.com wrote: I fixed my two issues (Person name headshot), Great! Thank you very much. I'll have time to check the patch and commit it to Mercurial only tomorrow. By the way, anyone should feel free to fork the IMDbPY repository on Bitbucket (specifically http://bitbucket.org/alberanid/imdbpy_new_search_parsers/ ), commit his changes and ask for a pull. BTW, I really like the parser, it's a good scraper, Most of the credit should go to H. Turgut Uyar, who wrote that wonderful DOM/XPath-based parser. tried building it myself, imdb page killed lxml, elementtree and BeautifulSoup, Strange: we're based on lxml (falling back to BeautifulSoup if lxml is not installed). you think they are writing an ill-formed html deliberately to keep us scrapers away ?? Hmmm... I fear that, like every big portal, they have to cope with so many browsers and environment that a nice and standard HTML can't work. :-) Thanks again! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy 4.6 can't fetch person head shot (and who add IMDb to the names ?)
On 09/29/2010 12:45 PM, Davide Alberani wrote: I'll have time to check the patch and commit it to Mercurial only tomorrow. By the way, anyone should feel free to fork the IMDbPY repository on Bitbucket (specifically http://bitbucket.org/alberanid/imdbpy_new_search_parsers/ ), commit his changes and ask for a pull. Is it that fork or the one called imdbpy_parsers2010? I might have forked the wrong one. tried building it myself, imdb page killed lxml, elementtree and BeautifulSoup, Strange: we're based on lxml (falling back to BeautifulSoup if lxml is not installed). At some places we have to fix the HTML before feeding it to lxml or beautifulsoup (the preprocessors in the code). Maybe it was one of those pages. -- H. Turgut Uyar u...@itu.edu.tr [GPG KeyID: 0xEAF45FB8] http://web.itu.edu.tr/uyar/ -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy 4.6 can't fetch person head shot (and who add IMDb to the names ?)
On Wed, Sep 29, 2010 at 12:45 PM, H. Turgut Uyar u...@itu.edu.tr wrote: Is it that fork or the one called imdbpy_parsers2010? No, pardon: my mistake. The correct project to fork is imdbpy_parsers2010 : http://bitbucket.org/alberanid/imdbpy_parsers2010 Thanks! -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDb redesign: call for help
On Sun, Sep 26, 2010 at 4:31 PM, H. Turgut Uyar u...@itu.edu.tr wrote: I'll try to help. I have quite lot of work these days but I'll get to the parsers as soon as I can. As usual, thank you! :-) I hope to have time to check to at least the main problems about people's pages within this week. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://www.mimante.net/ -- Start uncovering the many advantages of virtual appliances and start using them to simplify application deployment and accelerate your shift to cloud computing. http://p.sf.net/sfu/novell-sfdev2dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.6 released
IMDbPY 4.6 is available: http://imdbpy.sf.net/ In this release, a lot of minor fixes and some overall improvements. Sorry for the long delay - we really need a release manager. :-) Have fun! -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- ThinkGeek and WIRED's GeekDad team up for the Ultimate GeekDad Father's Day Giveaway. ONE MASSIVE PRIZE to the lucky parental unit. See the prize list and enter to win: http://p.sf.net/sfu/thinkgeek-promo ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY: build objects from XML dumps
I love Mercurial. :-) Have fun with this fork (soon it will be merged): http://bitbucket.org/alberanid/imdbpy_import_xml/overview/ New feature: since some time IMDbPY can dump its object into XML strings/files; now you can use this XML to recreated IMDbPY objects (instances of classes Movie, Person, Character and Company). I've not tested it extensively, and for sure it will fail here and there (and the code is not exactly nice to look at). You can test it this way (no output is a good sign ;-): from imdb import IMDb from imdb.helpers import parseXML ia = IMDb('http') matrix = ia.get_movie('0133093') xml = matrix.asXML() matrix_rebuilt = parseXML(xml) for key in sorted(matrix.keys()): origValue = matrix[key] rebuiltValue = matrix_rebuilt.get(key) if origValue != rebuiltValue: print 'ERROR', key print origValue print rebuiltValue print '' -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] IMDbPY 4.5
On Feb 28, Davide Alberani davide.alber...@gmail.com wrote: Released in a little rush, here it is 4.5: http://imdbpy.sourceforge.net/ You may have noticed that 4.5.1 is already out. :-) -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.5
Released in a little rush, here it is 4.5: http://imdbpy.sourceforge.net/ This fixes (I really do hope...) the recent issues with the imdb.com web server and contains other fixes and improvements (most notably, the ability to decouple the creation of CSV files from their insertion in a SQL database, using the imdbpy2sql.py script). As usual, report any bug and share your ideas for improvements! Enjoy, -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://www.mimante.net/ -- Download Intel#174; Parallel Studio Eval Try the new software tools for yourself. Speed compiling, find bugs proactively, and fine-tune applications for parallel performance. See why Intel Parallel Studio got high marks during beta. http://p.sf.net/sfu/intel-sw-dev ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] IMDbPY 4.3
IMDbPY 4.3 is out, to fix the last problems with some changes to the IMDb website. There're also some cool new features, like the ability to guess the language of a movie title. As usual, it can be downloaded from here: http://imdbpy.sf.net/ Since IMDb is introducing a new style for page about persons, soon we'll change the parsers to support it. -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] changes to imdb.com
On Nov 17, H. Turgut Uyar u...@itu.edu.tr wrote: Yes, it returns this: Good - even if it's making me crazy. ;-) You get what I see on my local system. Please, can you try it again in a hour or so (or tomorrow) and see if it's still ok (I've made some last-minute changes to the code and the account)? And if possible, can you try to install it (I tried it with Python 2.6, and it works, on my system)? I've tested it in clean virtual environments (2.6) both with lxml and without lxml and got the same result. It's the same on my system (Python 2.5) where it works. On a remote system, where IMDbPY is updated from the SVN and installed locally (on a clean directory), it still doesn't work. :-( The funny thing is that both my system and the remote one fetch pages that are served for the IMDbPYweb user account (I can see it in the HTML), and so I can't figure out why the HTML served to the remote system contains the sortable lists. It should, since I've set up the IMDbPYweb IMDb account to _not_ use the sortable lists, showing information about people. Do I need to set something in imdbpy.cfg for this or is it already handled by the code? No - it should be fine without any changes. I suspect there's something veeery wrong somewhere - but I can't find what - keep in mind that the remote system I'm talking about is clean, with no transparent proxies or strange things like these, and hosted on a more-than-trusted provider. Thanks! -- Davide Alberani davide.alber...@gmail.com [GPG KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Let Crystal Reports handle the reporting - Free Crystal Reports 2008 30-Day trial. Simplify your report design, integration and deployment - and focus on what you do best, core application coding. Discover what's new with Crystal Reports now. http://p.sf.net/sfu/bobj-july ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDbPY 4.0 and IMDbPYKit 1.0
On Mar 13, Jean-Yves Avenard jyaven...@gmail.com wrote: Doesn't build on any of my machines (Ubuntu 8.10 and CentOS 5.3) :( The kind of things you don't want to hear 12 hours after a release. ;-) imdb/parser/common/cutils.c:63:20: error: Python.h: No such file or directory Looks like you're missing the python-dev package, used to compile an (optional, but suggested) C module. Let me know if it solves the problem. As an extreme measure, you can add the --without-cutils command line argument to exclude the cutils.c module, but... Bye, -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDbPY 4.0 and IMDbPYKit 1.0
Hi Doesn't build on any of my machines (Ubuntu 8.10 and CentOS 5.3) :( admin...@hi-2-backend:~/IMDbPY-4.0$ sudo python setup.py build running build running build_py running build_ext building 'imdb.parser.common.cutils' extension gcc -pthread -fno-strict-aliasing -DNDEBUG -g -fwrapv -O2 -Wall -Wstrict-prototypes -fPIC -I/usr/include/python2.5 -c imdb/parser/common/cutils.c -o build/temp.linux-i686-2.5/imdb/parser/common/cutils.o imdb/parser/common/cutils.c:63:20: error: Python.h: No such file or directory imdb/parser/common/cutils.c: In function ‘strings_check’: imdb/parser/common/cutils.c:104: warning: implicit declaration of function ‘strlen’ imdb/parser/common/cutils.c:104: warning: incompatible implicit declaration of built-in function ‘strlen’ imdb/parser/common/cutils.c:112: warning: implicit declaration of function ‘strcmp’ imdb/parser/common/cutils.c: In function ‘ratcliff’: imdb/parser/common/cutils.c:182: warning: incompatible implicit declaration of built-in function ‘strlen’ imdb/parser/common/cutils.c: In function ‘strtolower’: imdb/parser/common/cutils.c:194: warning: incompatible implicit declaration of built-in function ‘strlen’ imdb/parser/common/cutils.c:194: warning: implicit declaration of function ‘tolower’ imdb/parser/common/cutils.c: At top level: imdb/parser/common/cutils.c:199: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token imdb/parser/common/cutils.c:233: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token imdb/parser/common/cutils.c:365: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token imdb/parser/common/cutils.c:556: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token imdb/parser/common/cutils.c:627: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token imdb/parser/common/cutils.c:721: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘*’ token imdb/parser/common/cutils.c:768: error: expected ‘=’, ‘,’, ‘;’, ‘asm’ or ‘__attribute__’ before ‘cutils_methods’ imdb/parser/common/cutils.c: In function ‘initcutils’: imdb/parser/common/cutils.c:788: warning: implicit declaration of function ‘Py_InitModule’ imdb/parser/common/cutils.c:788: error: ‘cutils_methods’ undeclared (first use in this function) imdb/parser/common/cutils.c:788: error: (Each undeclared identifier is reported only once imdb/parser/common/cutils.c:788: error: for each function it appears in.) === 2009/3/13 Davide Alberani davide.alber...@gmail.com Hi all, I'm proud to announce that IMDbPY 4.0 is here! :-) At the same time, you can download IMDbPYKit 1.0, the new web interface of choice for IMDbPY. Mostly developed by H. Turgut Uyar, IMDbPYKit has a lot of nice feature (e.g.: it can output data in both XML and HTML). There are a lot of new feature in this IMDbPY release, like support for keywords, support for top250/bottom100 lists and the ability to export information in XML. Many other minor features and a lot of bug-fixes are included! As usual, everything is available from here: http://imdbpy.sf.net/ Download enjoy! :-) -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ http://erlug.linux.it/%7Eda/ -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ Imdbpy-devel mailing list imdbpy-de...@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-devel -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] [Imdbpy-devel] IMDbPY 4.0 and IMDbPYKit 1.0
Hi 2009/3/13 Jean-Yves Avenard jyaven...@gmail.com imdb/parser/common/cutils.c:63:20: error: Python.h: No such file or directory imdb/parser/common/cutils.c: In function ‘strings_check’: Oops, forget my last message.. easily fixed... -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
[Imdbpy-help] imdbpy and imdb licences/access terms
Hi, I was just tinkering with imdbpy and looking through the source. From what I can gather, imdbpy functions as some sort of web scraper. Now, looking at the imdb terms and conditions, it seems the only interface provided is ftp access to compressed text files. In fact, the terms and conditions explicitly forbid screen/web scrapers. What I would like to know is: - are my assumptions above correct? - Does imdbpy have a special license from imdb to provide this package? - Do I need to get special permission for imdb to actually use this package (or at least the query functions)? Thanks, Andre -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help
Re: [Imdbpy-help] imdbpy and imdb licences/access terms
On Mar 02, Andre Eliatamby andre.eliata...@gmail.com wrote: What I would like to know is: - are my assumptions above correct? Mostly; see also the other recent (and old) threads in imdbpy-devel. - Does imdbpy have a special license from imdb to provide this package? No. I honestly can't even remember if I ever asked (maybe, maybe not: I started the project in 2004). For sure at some point (years ago) I questioned the developers of other related libraries; many (most) never asked, some had tried. The results can be summarized in: - no reply at all. - no, you can't: buy this (license for X thousands of data, missing the point of the request). - yes you can (very few). As far as I can remember there was a discussion about this kind of software in debian-devel, and their opinion was that they are ok for inclusion in Debian (it goes without saying that they are more interested in the license of the code, so this may not be conclusive). The fact is (as usual: I'm not a lawyer) that similar terms _as far as I know_ aren't clearly enforceable (I'm not sure about the USA laws, however). IMDbPY fetches the data the same way your browser (or your cache, or your crap-filtering proxy) does. After that, _for your own usage_ I'm pretty sure you can do whatever you like (your how usage completely excludes redistribution and making any kind of profit, no matter how indirect, that's for sure). Moreover: keep in mind that IMDbPY is not (and never will be) tailored as a mass-scraper bot: it can handle only single (and serial) requests, and for the way it's structured using it on the whole database is impossible. In fact, it's as nice as it can be with the IMDb server. - Do I need to get special permission for imdb to actually use this package (or at least the query functions)? You can use IMDbPY to access the plain text data files (putting them in a SQL database: see README.sqldb). If you plan to use 'http'/'mobile', obviously a permission won't hurt, but as long as you use it for your own personal non-commercial usage, I can't see any problem. It goes without saying that if you plan to create a public service with it (or make money out of it in any other way), you can stop right now. :-) Again: as said other times, I'm genuinely convinced that using it non-commercially is legal, but... I'm not a lawyer. -- Davide Alberani davide.alber...@gmail.com [PGP KeyID: 0x465BFD47] http://erlug.linux.it/~da/ -- Open Source Business Conference (OSBC), March 24-25, 2009, San Francisco, CA -OSBC tackles the biggest issue in open source: Open Sourcing the Enterprise -Strategies to boost innovation and cut costs with open source participation -Receive a $600 discount off the registration fee with the source code: SFAD http://p.sf.net/sfu/XcvMzF8H ___ Imdbpy-help mailing list Imdbpy-help@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/imdbpy-help