The branch, frodo has been updated
via b7b9192d10a7b9c17bd3b7d66d4d55ad33dceeb7 (commit)
from 10b178767dbea4a7ce682ab2ade14f959795dab5 (commit)
- Log -----------------------------------------------------------------
http://xbmc.git.sourceforge.net/git/gitweb.cgi?p=xbmc/scripts;a=commit;h=b7b9192d10a7b9c17bd3b7d66d4d55ad33dceeb7
commit b7b9192d10a7b9c17bd3b7d66d4d55ad33dceeb7
Author: Martijn Kaijser <[email protected]>
Date: Sat Sep 28 10:49:17 2013 +0200
[script.games.rom.collection.browser] 2.0.10
diff --git a/script.games.rom.collection.browser/addon.xml
b/script.games.rom.collection.browser/addon.xml
index fb22af3..5778bc6 100644
--- a/script.games.rom.collection.browser/addon.xml
+++ b/script.games.rom.collection.browser/addon.xml
@@ -1,7 +1,7 @@
<?xml version="1.0" encoding="UTF-8"?>
<addon
id="script.games.rom.collection.browser"
- version="2.0.7"
+ version="2.0.10"
name="Rom Collection Browser"
provider-name="carabalb, jimyx17, malte, redkiller">
<requires>
@@ -13,6 +13,12 @@
<provides>game</provides>
</extension>
<extension point="xbmc.addon.metadata">
+ <language></language>
+ <platform>all</platform>
+ <license>GNU GENERAL PUBLIC LICENSE. Version 2, June 1991</license>
+ <forum>http://forum.xbmc.org/showthread.php?tid=70115</forum>
+ <website>https://code.google.com/p/romcollectionbrowser/</website>
+ <source>https://romcollectionbrowser.googlecode.com/svn/trunk/</source>
<summary lang="da">Importere, gennemse og starte spil i XBMC</summary>
<summary lang="de">Importiert, verwaltet und startet Spiele in
XBMC</summary>
<summary lang="el">Î ÏοÏθÎÏÏε, αναζηÏήÏÏε και
ÏαίξÏε ÏαιÏνίδια ÏÏο XBMC</summary>
@@ -33,8 +39,5 @@
<description lang="hu">Jelenlegi funkciók: * játékok szűrése
konzoltÃpus, műfaj, év és kiadó alapján * rom-ok indÃtása külsÅ
emulátorral * a program beállÃtása egy xml fájl segÃtségével
* egyszerű de testreszabható leÃrófájl-feldolgozó * játékonként
különbözÅ képek és videók megjelenÃtése * játéktartalom és
kiegészÃtÅ információk megjelenÃtése </description>
<description lang="pt">Com este Addon pode navegar e executar roms de
emuladores e jogos individuais no seu XBMC. Veja a página de wiki para mais
informação:
http://code.google.com/p/romcollectionbrowser/wiki/Welcome. Primeiro
uso: * Na primeira execução, RCB iniciará um assistente de
configuração * Após a configuraçã estar completa, o RCB irá
importar os seus jogos * Se precisar, execuIf needed you can launch the
wizard anytime via context menu * Just press "C" on your keyboard or
"Menu" on your remote Features: * scan your rom folders and
import games to local database * scrape artwork and meta data from
various online sources * import local available descriptions, artwork and
videos * filter games by console, genre, year and publisher *
favorites function * automatic video playback in video window *
fullscreen video browser * game details page * dedicated MAME views
to
display cabinets and marquees * launch roms and standalone games in solo
or non-solo mode * detect games in zip and 7z archives and ask user which
game to load * direct launching of savestate files *
... </description>
<description lang="pt_BR">Com esse addon você pode navegar e lançar roms
de emuladores e jogos através do xbmc. Veja a pagina de wiki para mais
informações: http://code.google.com/p/romcollectionbrowser/wiki/Welcome.(\n)(\n) Primeiro
Uso: No primeiro lançamento o RCB irá iniciar um assistente de
configuração simples( *Depois de concluir o assistente RCB irá pedir
para importar seus jogos *Se necessário, você pode iniciar o assistente a
qualquer momento através do menu de contexto * Basta pressionar "C" no seu
teclado ou "Menu" em seu controle remoto CaracterÃsticas: *
varredura de suas pastas com roms e importação de jogos para um banco de
dados local * procura arte e meta-dados de várias fontes on-line *
importação local para descrições e obras de arte e vÃdeos se
disponÃveis * filtrar jogos por console, gênero, ano e editor *
função favoritos * reprodução automática de videos na janela do na
vegador(\n) *vÃdeo em tela cheia no navegador * pagina de detalhes do
jogo * vistas dedicadas a MAME para exibir cabines e letreiros(\n) *
lançamento de roms e jogos independentes no modo sozinho ou não sozinho *
detectar jogos em arquivos ZIP e 7z e perguntar ao usuário qual jogo
carregar * lançamento direto de arquivos de
savestate . </description>
- <language></language>
- <license>GPL 2.0</license>
- <platform>all</platform>
</extension>
</addon>
diff --git a/script.games.rom.collection.browser/changelog.txt
b/script.games.rom.collection.browser/changelog.txt
index f115332..a96f5f1 100644
--- a/script.games.rom.collection.browser/changelog.txt
+++ b/script.games.rom.collection.browser/changelog.txt
@@ -1,3 +1,18 @@
+2.0.10 (09/2013)
+~~~~~~~~~~~~~
+General:
+ * bugfix: an error occured when creating a new config file
+
+2.0.9 (09/2013)
+~~~~~~~~~~~~~
+Import Games:
+ * fix thegamesdb scraper: requests have been blocked by cloudflare
+
+2.0.8 (09/2013)
+~~~~~~~~~~~~~
+Import Games:
+ * update moby scraper: reflect changes on page
+
2.0.7 (06/2013)
~~~~~~~~~~~~~
Import Games:
diff --git
a/script.games.rom.collection.browser/resources/database/config_template.xml
b/script.games.rom.collection.browser/resources/database/config_template.xml
index fae7393..ad87e0d 100644
--- a/script.games.rom.collection.browser/resources/database/config_template.xml
+++ b/script.games.rom.collection.browser/resources/database/config_template.xml
@@ -1,4 +1,4 @@
-<config version="1.0.6">
+<config version="2.0.8">
<RomCollections>
</RomCollections>
@@ -51,11 +51,11 @@
<type>video</type>
<parent>game</parent>
</FileType>
- <FileType id="13" name="cabinet">
+ <FileType id="13" name="cabinet">
<type>image</type>
<parent>game</parent>
</FileType>
- <FileType id="14" name="marquee">
+ <FileType id="14" name="marquee">
<type>image</type>
<parent>game</parent>
</FileType>
@@ -64,51 +64,51 @@
<fileTypeFor name="gameinfobig">
<fileTypeForGameList>boxfront</fileTypeForGameList>
<fileTypeForGameList>screenshot</fileTypeForGameList>
- <fileTypeForGameList>title</fileTypeForGameList>
- <fileTypeForGameList>action</fileTypeForGameList>
+ <fileTypeForGameList>title</fileTypeForGameList>
+ <fileTypeForGameList>action</fileTypeForGameList>
<fileTypeForGameListSelected>boxfront</fileTypeForGameListSelected>
<fileTypeForGameListSelected>screenshot</fileTypeForGameListSelected>
- <fileTypeForGameListSelected>title</fileTypeForGameListSelected>
- <fileTypeForGameListSelected>action</fileTypeForGameListSelected>
+ <fileTypeForGameListSelected>title</fileTypeForGameListSelected>
+ <fileTypeForGameListSelected>action</fileTypeForGameListSelected>
<fileTypeForMainViewBackground>fanart</fileTypeForMainViewBackground>
<fileTypeForMainViewBackground>boxfront</fileTypeForMainViewBackground>
<fileTypeForMainViewBackground>screenshot</fileTypeForMainViewBackground>
- <fileTypeForMainViewBackground>title</fileTypeForMainViewBackground>
- <fileTypeForMainViewBackground>action</fileTypeForMainViewBackground>
+ <fileTypeForMainViewBackground>title</fileTypeForMainViewBackground>
+ <fileTypeForMainViewBackground>action</fileTypeForMainViewBackground>
<fileTypeForMainViewGameInfoBig>screenshot</fileTypeForMainViewGameInfoBig>
<fileTypeForMainViewGameInfoBig>boxfront</fileTypeForMainViewGameInfoBig>
-
<fileTypeForMainViewGameInfoBig>action</fileTypeForMainViewGameInfoBig>
+ <fileTypeForMainViewGameInfoBig>action</fileTypeForMainViewGameInfoBig>
<fileTypeForMainViewGameInfoBig>title</fileTypeForMainViewGameInfoBig>
- <fileTypeForMainView1>publisher</fileTypeForMainView1>
- <fileTypeForMainView2>romcollection</fileTypeForMainView2>
- <fileTypeForMainView3>developer</fileTypeForMainView3>
+ <fileTypeForMainView1>publisher</fileTypeForMainView1>
+ <fileTypeForMainView2>romcollection</fileTypeForMainView2>
+ <fileTypeForMainView3>developer</fileTypeForMainView3>
</fileTypeFor>
<fileTypeFor name="gameinfosmall">
<fileTypeForGameList>boxfront</fileTypeForGameList>
<fileTypeForGameList>screenshot</fileTypeForGameList>
- <fileTypeForGameList>title</fileTypeForGameList>
- <fileTypeForGameList>action</fileTypeForGameList>
+ <fileTypeForGameList>title</fileTypeForGameList>
+ <fileTypeForGameList>action</fileTypeForGameList>
<fileTypeForGameListSelected>boxfront</fileTypeForGameListSelected>
<fileTypeForGameListSelected>screenshot</fileTypeForGameListSelected>
- <fileTypeForGameListSelected>title</fileTypeForGameListSelected>
- <fileTypeForGameListSelected>action</fileTypeForGameListSelected>
+ <fileTypeForGameListSelected>title</fileTypeForGameListSelected>
+ <fileTypeForGameListSelected>action</fileTypeForGameListSelected>
<fileTypeForMainViewBackground>fanart</fileTypeForMainViewBackground>
<fileTypeForMainViewBackground>boxfront</fileTypeForMainViewBackground>
<fileTypeForMainViewBackground>screenshot</fileTypeForMainViewBackground>
- <fileTypeForMainViewBackground>title</fileTypeForMainViewBackground>
- <fileTypeForMainViewBackground>action</fileTypeForMainViewBackground>
+ <fileTypeForMainViewBackground>title</fileTypeForMainViewBackground>
+ <fileTypeForMainViewBackground>action</fileTypeForMainViewBackground>
<fileTypeForMainViewGameInfoUpperLeft>screenshot</fileTypeForMainViewGameInfoUpperLeft>
-
<fileTypeForMainViewGameInfoUpperLeft>action</fileTypeForMainViewGameInfoUpperLeft>
-
<fileTypeForMainViewGameInfoUpperLeft>title</fileTypeForMainViewGameInfoUpperLeft>
+
<fileTypeForMainViewGameInfoUpperLeft>action</fileTypeForMainViewGameInfoUpperLeft>
+
<fileTypeForMainViewGameInfoUpperLeft>title</fileTypeForMainViewGameInfoUpperLeft>
<fileTypeForMainViewGameInfoUpperRight>boxfront</fileTypeForMainViewGameInfoUpperRight>
<fileTypeForMainViewGameInfoLowerLeft>cartridge</fileTypeForMainViewGameInfoLowerLeft>
<fileTypeForMainViewGameInfoLowerRight>boxback</fileTypeForMainViewGameInfoLowerRight>
-
<fileTypeForMainViewGameInfoLowerRight>title</fileTypeForMainViewGameInfoLowerRight>
- <fileTypeForMainView1>publisher</fileTypeForMainView1>
- <fileTypeForMainView2>romcollection</fileTypeForMainView2>
- <fileTypeForMainView3>developer</fileTypeForMainView3>
+
<fileTypeForMainViewGameInfoLowerRight>title</fileTypeForMainViewGameInfoLowerRight>
+ <fileTypeForMainView1>publisher</fileTypeForMainView1>
+ <fileTypeForMainView2>romcollection</fileTypeForMainView2>
+ <fileTypeForMainView3>developer</fileTypeForMainView3>
</fileTypeFor>
- <fileTypeFor name="gameinfomamemarquee">
+ <fileTypeFor name="gameinfomamemarquee">
<fileTypeForGameList>marquee</fileTypeForGameList>
<fileTypeForGameList>boxfront</fileTypeForGameList>
<fileTypeForGameList>title</fileTypeForGameList>
@@ -154,21 +154,24 @@
<Scraper parseInstruction="03.01 - giantbomb - search.xml"
returnUrl="true"
source="http://api.giantbomb.com/search/?api_key=%GIANTBOMBAPIKey%&query=%GAME%&resources=game&field_list=api_detail_url,name&format=xml";
/>
<Scraper parseInstruction="03.02 - giantbomb - detail.xml" source="1" />
</Site>
- <Site name="mobygames.com" descFilePerGame="True" searchGameByCRC="False">
- <Scraper parseInstruction="04.01 - mobygames - gamesearch.xml"
returnUrl="true"
source="http://www.mobygames.com/search/quick?game=%GAME%&p=%PLATFORM%"; />
+ <Site descFilePerGame="True" name="mobygames.com" searchGameByCRC="False">
+ <Scraper parseInstruction="04.01 - mobygames - gamesearch.xml"
returnUrl="true"
source="http://www.mobygames.com/search/quick?game=%GAME%&amp;p=%PLATFORM%";
/>
<Scraper parseInstruction="04.02 - mobygames - details.xml" source="1" />
- <Scraper parseInstruction="04.03 - mobygames - coverlink.xml"
returnUrl="true" source="1" />
- <Scraper parseInstruction="04.04 - mobygames - coverart.xml" source="2"
/>
- <Scraper parseInstruction="04.05 - mobygames - screenshotlink.xml"
returnUrl="true" source="1" />
- <Scraper parseInstruction="04.06 - mobygames - screenoriglink.xml"
returnUrl="true" source="3" />
- <Scraper parseInstruction="04.07 - mobygames - screenshots.xml"
source="4" />
+ <Scraper parseInstruction="04.03 - mobygames - coverlink front.xml"
returnUrl="true" source="1" sourceAppend="cover-art" />
+ <Scraper parseInstruction="04.04 - mobygames - coverdetail front.xml"
source="2" />
+ <Scraper parseInstruction="04.05 - mobygames - coverlink back.xml"
returnUrl="true" source="1" sourceAppend="cover-art" />
+ <Scraper parseInstruction="04.06 - mobygames - coverdetail back.xml"
source="3" />
+ <Scraper parseInstruction="04.07 - mobygames - coverlink media.xml"
returnUrl="true" source="1" sourceAppend="cover-art" />
+ <Scraper parseInstruction="04.08 - mobygames - coverdetail media.xml"
source="4" />
+ <Scraper parseInstruction="04.09 - mobygames - screenshotlink.xml"
returnUrl="true" source="1" sourceAppend="screenshots" />
+ <Scraper parseInstruction="04.10 - mobygames - screenshot detail.xml"
source="5" />
</Site>
- <Site descFilePerGame="True" name="archive.vg" searchGameByCRC="False">
+ <Site descFilePerGame="True" name="archive.vg" searchGameByCRC="False">
<Scraper encoding="iso-8859-1" parseInstruction="05.01 - archive -
search.xml" returnUrl="true"
source="http://api.archive.vg/2.0/Archive.search/%ARCHIVEAPIKEY%/%GAME%"; />
- <Scraper encoding="iso-8859-1" parseInstruction="05.02 - archive -
detail.xml" source="1" />
+ <Scraper encoding="iso-8859-1" parseInstruction="05.02 - archive -
detail.xml" source="1" />
+ </Site>
+ <Site name="maws.mameworld.info" descFilePerGame="True"
searchGameByCRC="False">
+ <Scraper parseInstruction="06 - maws.xml"
source="http://maws.mameworld.info/maws/romset/%GAME%"; encoding='iso-8859-1'/>
</Site>
- <Site name="maws.mameworld.info" descFilePerGame="True"
searchGameByCRC="False">
- <Scraper parseInstruction="06 - maws.xml"
source="http://maws.mameworld.info/maws/romset/%GAME%"; encoding='iso-8859-1'/>
- </Site>
</Scrapers>
</config>
diff --git a/script.games.rom.collection.browser/resources/lib/config.py
b/script.games.rom.collection.browser/resources/lib/config.py
index 62a1852..ab8949d 100644
--- a/script.games.rom.collection.browser/resources/lib/config.py
+++ b/script.games.rom.collection.browser/resources/lib/config.py
@@ -188,6 +188,7 @@ class MediaPath:
class Scraper:
parseInstruction = ''
source = ''
+ sourceAppend = ''
encoding = 'utf-8'
returnUrl = False
replaceKeyString = ''
@@ -589,6 +590,10 @@ class Config:
if(returnUrl != None and returnUrl != ''):
scraper.returnUrl = returnUrl.upper() == 'TRUE'
+ sourceAppend = scraperRow.attrib.get('sourceAppend')
+ if(sourceAppend != None and sourceAppend != ''):
+ scraper.sourceAppend = sourceAppend
+
scraper.replaceKeyString = inReplaceKeyString
scraper.replaceValueString = inReplaceValueString
diff --git
a/script.games.rom.collection.browser/resources/lib/configxmlupdater.py
b/script.games.rom.collection.browser/resources/lib/configxmlupdater.py
index 5d4f034..095fea7 100644
--- a/script.games.rom.collection.browser/resources/lib/configxmlupdater.py
+++ b/script.games.rom.collection.browser/resources/lib/configxmlupdater.py
@@ -81,6 +81,12 @@ class ConfigxmlUpdater:
if(not success):
return False, message
+ if(configVersion == '1.0.6'):
+ success, message = self.update_106_to_208()
+ configVersion = '2.0.8'
+ if(not success):
+ return False, message
+
#write file
success, message = self.writeFile()
@@ -226,6 +232,83 @@ class ConfigxmlUpdater:
break
return True, ''
+
+
+ def update_106_to_208(self):
+ #update mobygames scraper
+ scraperSitesXml = self.tree.findall('Scrapers/Site')
+ for scraperSiteXml in scraperSitesXml:
+ siteName = scraperSiteXml.attrib.get('name')
+ if(siteName == 'mobygames.com'):
+ #delete all existing scraper elements
+ scraperElements =
scraperSiteXml.findall('Scraper')
+ for scraperElement in scraperElements:
+ self.removeElement(scraperSiteXml,
'Scraper')
+
+ #add new scraper elements
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.01 - mobygames -
gamesearch.xml',
+ 'source' :
'http://www.mobygames.com/search/quick?game=%GAME%&p=%PLATFORM%',
+ 'returnUrl' : 'true'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.02 - mobygames -
details.xml',
+ 'source' : '1'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.03 - mobygames -
coverlink front.xml',
+ 'source' : '1',
+ 'sourceAppend' : 'cover-art',
+ 'returnUrl' : 'true'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.04 - mobygames -
coverdetail front.xml',
+ 'source' : '2'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.05 - mobygames -
coverlink back.xml',
+ 'source' : '1',
+ 'sourceAppend' : 'cover-art',
+ 'returnUrl' : 'true'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.06 - mobygames -
coverdetail back.xml',
+ 'source' : '3'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.07 - mobygames -
coverlink media.xml',
+ 'source' : '1',
+ 'sourceAppend' : 'cover-art',
+ 'returnUrl' : 'true'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.08 - mobygames -
coverdetail media.xml',
+ 'source' : '4'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.09 - mobygames -
screenshotlink.xml',
+ 'source' : '1',
+ 'sourceAppend' : 'screenshots',
+ 'returnUrl' : 'true'
+ })
+ scraperXml = SubElement(scraperSiteXml,
'Scraper',
+ {
+ 'parseInstruction' : '04.10 - mobygames -
screenshot detail.xml',
+ 'source' : '5'
+ })
+
+ break
+
+ return True, ''
#TODO use same as in config
diff --git a/script.games.rom.collection.browser/resources/lib/dbupdate.py
b/script.games.rom.collection.browser/resources/lib/dbupdate.py
index 9c7bd77..fb1a5a5 100644
--- a/script.games.rom.collection.browser/resources/lib/dbupdate.py
+++ b/script.games.rom.collection.browser/resources/lib/dbupdate.py
@@ -4,6 +4,7 @@ import getpass, string, glob
import codecs
import zipfile
import time
+import urllib2
import xbmcvfs
import fnmatch
@@ -1186,15 +1187,22 @@ class DBUpdate:
pass
# fetch thumbnail and save to filepath
- try:
- #download file to local folder and copy
it to smb path with xbmcvfs
+ try:
+ target = fileName
if(fileName.startswith('smb://')):
- localFile =
util.joinPath(util.getTempDir(), os.path.basename(fileName))
- urllib.urlretrieve( thumbUrl,
localFile)
- xbmcvfs.copy(localFile,
fileName)
- xbmcvfs.delete(localFile)
- else:
- urllib.urlretrieve( thumbUrl,
str(fileName))
+ #download file to local folder
and copy it to smb path with xbmcvfs
+ target =
util.joinPath(util.getTempDir(), os.path.basename(fileName))
+
+ req = urllib2.Request(thumbUrl)
+
req.add_unredirected_header('User-Agent', 'Mozilla/5.0 (Windows NT 6.1; WOW64)
AppleWebKit/537.31 (KHTML, like Gecko) Chrome/26.0.1410.64 Safari/537.31')
+ f = open(target,'wb')
+ f.write(urllib2.urlopen(req).read())
+ f.close()
+
+ if(fileName.startswith('smb://')):
+ xbmcvfs.copy(target, fileName)
+ xbmcvfs.delete(target)
+
except Exception, (exc):
xbmcgui.Dialog().ok(util.localize(35012), util.localize(35011))
Logutil.log("Could not create file:
'%s'. Error message: '%s'" %(str(fileName), str(exc)), util.LOG_LEVEL_ERROR)
diff --git
a/script.games.rom.collection.browser/resources/lib/dialogcontextmenu.py
b/script.games.rom.collection.browser/resources/lib/dialogcontextmenu.py
index eb2f099..ccd9852 100644
--- a/script.games.rom.collection.browser/resources/lib/dialogcontextmenu.py
+++ b/script.games.rom.collection.browser/resources/lib/dialogcontextmenu.py
@@ -299,7 +299,7 @@ class ContextMenuDialog(xbmcgui.WindowXMLDialog):
for addonObj in jsonResult[u'result'][u'addons']:
id = addonObj[u'addonid']
- addon = xbmcaddon.Addon(id,
installedOnly=installedOnlyParam)
+ addon = xbmcaddon.Addon(id,
installed=installedOnlyParam)
# extensions and platforms are "|" separated,
extensions may or may not have a leading "."
if(addon.getAddonInfo('platforms') == platform):
addons.append(id)
diff --git
a/script.games.rom.collection.browser/resources/lib/pyscraper/descriptionparserflatfile.py
b/script.games.rom.collection.browser/resources/lib/pyscraper/descriptionparserflatfile.py
index 2cf93fb..67287dd 100644
---
a/script.games.rom.collection.browser/resources/lib/pyscraper/descriptionparserflatfile.py
+++
b/script.games.rom.collection.browser/resources/lib/pyscraper/descriptionparserflatfile.py
@@ -1,7 +1,7 @@
from pyparsing import *
from elementtree.ElementTree import *
-import urllib
+import urllib2
import time
import util
from util import Logutil
@@ -132,9 +132,9 @@ class DescriptionParserFlatFile:
fileAsString = ''
if(descFile.startswith('http://')):
- usock = urllib.urlopen(descFile)
- fileAsString = usock.read()
- usock.close()
+ req = urllib2.Request(descFile)
+ req.add_unredirected_header('User-Agent', 'Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko)
Chrome/26.0.1410.64 Safari/537.31')
+ fileAsString = urllib2.urlopen(req).read()
else:
fh = open(str(descFile), 'r')
fileAsString = fh.read()
diff --git
a/script.games.rom.collection.browser/resources/lib/pyscraper/descriptionparserxml.py
b/script.games.rom.collection.browser/resources/lib/pyscraper/descriptionparserxml.py
index 4293ef4..7bbe6ab 100644
---
a/script.games.rom.collection.browser/resources/lib/pyscraper/descriptionparserxml.py
+++
b/script.games.rom.collection.browser/resources/lib/pyscraper/descriptionparserxml.py
@@ -1,6 +1,6 @@
from elementtree.ElementTree import *
-import urllib
+import urllib2
import time
import util
@@ -20,7 +20,9 @@ class DescriptionParserXml:
results = None
if(descFile.startswith('http://')):
- descFile = urllib.urlopen(descFile).read()
+ req = urllib2.Request(descFile)
+ req.add_unredirected_header('User-Agent', 'Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko)
Chrome/26.0.1410.64 Safari/537.31')
+ descFile = urllib2.urlopen(req).read()
else:
fh = open(str(descFile), 'r')
descFile = fh.read()
@@ -58,7 +60,9 @@ class DescriptionParserXml:
Logutil.log('scanDescription: %s' % descFile,
util.LOG_LEVEL_INFO)
if(descFile.startswith('http://')):
- descFile = urllib.urlopen(descFile).read()
+ req = urllib2.Request(descFile)
+ req.add_unredirected_header('User-Agent', 'Mozilla/5.0
(Windows NT 6.1; WOW64) AppleWebKit/537.31 (KHTML, like Gecko)
Chrome/26.0.1410.64 Safari/537.31')
+ descFile = urllib2.urlopen(req).read()
else:
fh = open(str(descFile), 'r')
descFile = fh.read()
diff --git
a/script.games.rom.collection.browser/resources/lib/pyscraper/pyscraper.py
b/script.games.rom.collection.browser/resources/lib/pyscraper/pyscraper.py
index b8e2326..69f951d 100644
--- a/script.games.rom.collection.browser/resources/lib/pyscraper/pyscraper.py
+++ b/script.games.rom.collection.browser/resources/lib/pyscraper/pyscraper.py
@@ -33,6 +33,11 @@ class PyScraper:
url = urlsFromPreviousScrapers[int(scraper.source) -1]
Logutil.log("using url from previous scraper: "
+str(url), util.LOG_LEVEL_INFO)
+
+ if(scraper.sourceAppend != None and
scraper.sourceAppend != ""):
+ url = url + '/' +scraper.sourceAppend
+ Logutil.log("sourceAppend = '%s'. New url =
'%s'" %(scraper.sourceAppend, url), util.LOG_LEVEL_INFO)
+
scraperSource = url
if(scraper.source == 'nfo'):
diff --git a/script.games.rom.collection.browser/resources/lib/util.py
b/script.games.rom.collection.browser/resources/lib/util.py
index 3e96007..f000867 100644
--- a/script.games.rom.collection.browser/resources/lib/util.py
+++ b/script.games.rom.collection.browser/resources/lib/util.py
@@ -12,7 +12,7 @@ import time
SCRIPTNAME = 'Rom Collection Browser'
SCRIPTID = 'script.games.rom.collection.browser'
-CURRENT_CONFIG_VERSION = "1.0.6"
+CURRENT_CONFIG_VERSION = "2.0.8"
CURRENT_DB_VERSION = "0.7.4"
ISTESTRUN = False
diff --git a/script.games.rom.collection.browser/resources/scraper/04.01 -
mobygames - gamesearch.xml
b/script.games.rom.collection.browser/resources/scraper/04.01 - mobygames -
gamesearch.xml
index ff4796f..77e7a6f 100644
--- a/script.games.rom.collection.browser/resources/scraper/04.01 - mobygames -
gamesearch.xml
+++ b/script.games.rom.collection.browser/resources/scraper/04.01 - mobygames -
gamesearch.xml
@@ -1 +1 @@
-<parserConfig>
<GameGrammar type="multiline">
<SkippableContent skipTo="<div
class="searchTitle">Game: <a href=""
closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><div
class="searchTitle">Game: <a href="</SkippableContent>
<url closeStmnt="true" skipTo="">"
appendResultTo="http://www.mobygames.com";></url>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<SearchKey closeStmnt="true" skipTo="</a>"></SearchKey>
</GameGrammar>
</parserConfig>
\ No newline at end of file
+<parserConfig>
<GameGrammar type="multiline">
<SkippableContent skipTo="<h3 class='list'><a
href="" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><h3
class='list'><a href="</SkippableContent>
<url closeStmnt="true" skipTo="">"
appendResultTo="http://www.mobygames.com";></url>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<SearchKey closeStmnt="true" skipTo="</a>"></SearchKey>
</GameGrammar>
</parserConfig>
\ No newline at end of file
diff --git a/script.games.rom.collection.browser/resources/scraper/04.02 -
mobygames - details.xml
b/script.games.rom.collection.browser/resources/scraper/04.02 - mobygames -
details.xml
index d3df08b..aad6ed7 100644
--- a/script.games.rom.collection.browser/resources/scraper/04.02 - mobygames -
details.xml
+++ b/script.games.rom.collection.browser/resources/scraper/04.02 - mobygames -
details.xml
@@ -1 +1 @@
-<parserConfig>
<GameGrammar type="multiline">
<SkippableContent skipTo="<div
id="gameTitle"><a href=""
closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><div
id="gameTitle"><a href="</SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<Game closeStmnt="true" skipTo="</a>"></Game>
<SkippableContent skipTo="Published by</div><div
style="font-size: 90%; padding-left: 1em; padding-bottom:
0.25em;"><a href="" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true">Published
by</div><div style="font-size: 90%; padding-left: 1em;
padding-bottom: 0.25em;"><a href="</SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<Publisher closeStmnt="true" skipTo="</a>"></Publisher>
<SkippableContent skipTo="Developed by</div><div
style="font-size: 90%; padding-left: 1em; padding-bottom:
0.25em;"><a href="" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true">Developed
by</div><div style="font-size: 90%; padding-left: 1em;
padding-bottom: 0.25em;"><a href="</SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<Developer closeStmnt="true" skipTo="</a>"></Developer>
<SkippableContent skipTo="Released</div><div
style="font-size: 90%; padding-left: 1em; padding-bottom:
0.25em;"><a href="" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true">Released</div><div
style="font-size: 90%; padding-left: 1em; padding-bottom:
0.25em;"><a href="</SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<ReleaseYear closeStmnt="true"
skipTo="</a>"></ReleaseYear>
<SkippableContent skipTo="Genre</div><div
style="font-size: 90%; padding-left: 1em; padding-bottom:
0.25em;"><a href="" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true">Genre</div><div
style="font-size: 90%; padding-left: 1em; padding-bottom:
0.25em;"><a href="</SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<Genre closeStmnt="true" skipTo="</a>"
optional="true"></Genre>
<SkippableContent closeStmnt="true" skipTo="<h2
class="m5">Description</h2>"></SkippableContent>
<SkippableContent closeStmnt="true"><h2
class="m5">Description</h2></SkippableContent>
<Description closeStmnt="true" skipTo="<div
class="sideBarLinks">" optional="true"></Description>
</GameGrammar>
</parserConfig>
\ No newline at end of file
+<parserConfig>
<GameGrammar type="multiline">
<SkippableContent skipTo="<h1 class='hdr1'>"
closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><h1
class='hdr1'></SkippableContent>
<Game closeStmnt="true" skipTo="</h1>"></Game>
<SkippableContent skipTo="<div
class='listing'>Released</div>" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><div
class='listing'>Released</div></SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<ReleaseYear closeStmnt="true"
skipTo="</a>"></ReleaseYear>
<SkippableContent skipTo="<div class='listing'>Published
by</div>" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><div
class='listing'>Published by</div></SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<Publisher closeStmnt="true" skipTo="</a>"></Publisher>
<SkippableContent skipTo="<div class='listing'>Developed
by</div>" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><div
class='listing'>Developed by</div></SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<Developer closeStmnt="true" skipTo="</a>"></Developer>
<SkippableContent skipTo="<div
class='listing'>Genre</div>" closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><div
class='listing'>Genre</div></SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<Genre closeStmnt="true" skipTo="</a>"
optional="true"></Genre>
<SkippableContent skipTo="<div
class='listing'>Perspective</div>"
closeStmnt="true"></SkippableContent>
<SkippableContent closeStmnt="true"><div
class='listing'>Perspective</div></SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="">"></SkippableContent>
<SkippableContent
closeStmnt="true">"></SkippableContent>
<Perspective closeStmnt="true" skipTo="</a>"
optional="true"></Perspective>
<SkippableContent closeStmnt="true" optional="true"
skipTo="<div class='listing'>Groups</div>"></SkippableContent>
<SkippableContent closeStmnt="true" optional="true"><div
class='listing'>Groups</div></SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="</a></div>"></SkippableContent>
<SkippableContent
closeStmnt="true"></a></div></SkippableContent>
<SkippableContent closeStmnt="true"
skipTo="</div>"></SkippableContent>
<SkippableContent
closeStmnt="true"></div></SkippableContent>
<Description closeStmnt="true" skipTo="<br /><br
/>" optional="true"></Description>
</GameGrammar>
</parserConfig>
\ No newline at end of file
-----------------------------------------------------------------------
Summary of changes:
script.games.rom.collection.browser/addon.xml | 11 ++-
script.games.rom.collection.browser/changelog.txt | 15 ++++
.../resources/database/config_template.xml | 79 ++++++++++---------
.../resources/lib/config.py | 5 +
.../resources/lib/configxmlupdater.py | 83 ++++++++++++++++++++
.../resources/lib/dbupdate.py | 24 ++++--
.../resources/lib/dialogcontextmenu.py | 2 +-
.../lib/pyscraper/descriptionparserflatfile.py | 8 +-
.../lib/pyscraper/descriptionparserxml.py | 10 ++-
.../resources/lib/pyscraper/pyscraper.py | 5 +
.../resources/lib/util.py | 2 +-
.../scraper/04.01 - mobygames - gamesearch.xml | 2 +-
.../scraper/04.02 - mobygames - details.xml | 2 +-
.../04.03 - mobygames - coverlink front.xml | 9 ++
.../scraper/04.03 - mobygames - coverlink.xml | 1 -
.../scraper/04.04 - mobygames - coverart.xml | 13 ---
.../04.04 - mobygames - coverdetail front.xml | 7 ++
.../scraper/04.05 - mobygames - coverlink back.xml | 13 +++
.../scraper/04.05 - mobygames - screenshotlink.xml | 1 -
.../04.06 - mobygames - coverdetail back.xml | 7 ++
.../scraper/04.06 - mobygames - screenoriglink.xml | 1 -
.../04.06 - mobygames - screenshotoriginallink.xml | 1 -
.../04.07 - mobygames - coverlink media.xml | 13 +++
.../scraper/04.07 - mobygames - screenshots.xml | 7 --
.../04.08 - mobygames - coverdetail media.xml | 7 ++
.../scraper/04.09 - mobygames - screenshotlink.xml | 1 +
.../04.10 - mobygames - screenshot detail.xml | 1 +
27 files changed, 245 insertions(+), 85 deletions(-)
create mode 100644 script.games.rom.collection.browser/resources/scraper/04.03
- mobygames - coverlink front.xml
delete mode 100644 script.games.rom.collection.browser/resources/scraper/04.03
- mobygames - coverlink.xml
delete mode 100644 script.games.rom.collection.browser/resources/scraper/04.04
- mobygames - coverart.xml
create mode 100644 script.games.rom.collection.browser/resources/scraper/04.04
- mobygames - coverdetail front.xml
create mode 100644 script.games.rom.collection.browser/resources/scraper/04.05
- mobygames - coverlink back.xml
delete mode 100644 script.games.rom.collection.browser/resources/scraper/04.05
- mobygames - screenshotlink.xml
create mode 100644 script.games.rom.collection.browser/resources/scraper/04.06
- mobygames - coverdetail back.xml
delete mode 100644 script.games.rom.collection.browser/resources/scraper/04.06
- mobygames - screenoriglink.xml
delete mode 100644 script.games.rom.collection.browser/resources/scraper/04.06
- mobygames - screenshotoriginallink.xml
create mode 100644 script.games.rom.collection.browser/resources/scraper/04.07
- mobygames - coverlink media.xml
delete mode 100644 script.games.rom.collection.browser/resources/scraper/04.07
- mobygames - screenshots.xml
create mode 100644 script.games.rom.collection.browser/resources/scraper/04.08
- mobygames - coverdetail media.xml
create mode 100644 script.games.rom.collection.browser/resources/scraper/04.09
- mobygames - screenshotlink.xml
create mode 100644 script.games.rom.collection.browser/resources/scraper/04.10
- mobygames - screenshot detail.xml
hooks/post-receive
--
Scripts
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60133471&iu=/4140/ostg.clktrk
_______________________________________________
Xbmc-addons mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/xbmc-addons
