Re: apt-portal import opkg repos :) some comments
2009/9/5 Markus Törnqvist m...@nysv.org: On Wed, Sep 02, 2009 at 11:45:08AM +0200, David Reyes Samblas Martinez wrote: https://tuxbrain.org/websvn/ Hooray! Hi ssh+svn through a public key maybe is the most comfortable option, so send me a private mail with user name do you want i and a public key to generate the key(Markus sure you know but maybe other willing to colaborate and this aplies to anyone wanting to code on this) on the pc you want to use your local svn clone $ mkdir ~/.ssh/ $ cd ~/.ssh/ $ ssh-keygen -t dsa and send to me the ~/.ssh/id_dsa.pub I did it, but forgot to mention I'd like my username to be mjt :) Yes surelly this will get out of the vcs soon, but maybe someone is curiouse about structure or the info extracted from the repo so this was a qd to make it available for download, and ... what the hell !! I was proud of the success and want to show it :P Sure thing :) Mil gracias! -- mjt Done , so we will have to start a little more serious roadmap and start step by step, I will propose one later on but if anyone is faster don't be shy and tell us :) ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Wed, Sep 02, 2009 at 11:45:08AM +0200, David Reyes Samblas Martinez wrote: https://tuxbrain.org/websvn/ Hooray! Hi ssh+svn through a public key maybe is the most comfortable option, so send me a private mail with user name do you want i and a public key to generate the key(Markus sure you know but maybe other willing to colaborate and this aplies to anyone wanting to code on this) on the pc you want to use your local svn clone $ mkdir ~/.ssh/ $ cd ~/.ssh/ $ ssh-keygen -t dsa and send to me the ~/.ssh/id_dsa.pub I did it, but forgot to mention I'd like my username to be mjt :) Yes surelly this will get out of the vcs soon, but maybe someone is curiouse about structure or the info extracted from the repo so this was a qd to make it available for download, and ... what the hell !! I was proud of the success and want to show it :P Sure thing :) Mil gracias! -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
http://www.samsungapps.com/ - they're copying us! David: great to see the svn! r -- | risto h. kurppa | risto at kurppa dot fi | http://risto.kurppa.fi ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
2009/9/2 Risto H. Kurppa ri...@kurppa.fi: http://www.samsungapps.com/ - they're copying us! LOL, is a GPL project they are allowed to do it meanwhile they get access to the original source :) jokes a part , this can be a source of inspiration , lets see if they provide innovative ideas or will be only a list of payfordownload apps with some pics. David: great to see the svn! thanks :), looks pretty isn't it? :) so if the code looks so fine the final app has to be twice better :) r -- | risto h. kurppa | risto at kurppa dot fi | http://risto.kurppa.fi ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
2009/9/2 Markus Törnqvist m...@nysv.org: On Wed, Sep 02, 2009 at 04:09:30AM +0200, David Reyes Samblas Martinez wrote: [...] course make at least a devel om-showrom site :P but here is one step more of this long road. https://tuxbrain.org/websvn/ Hooray! also access to svn though ssh or https are setup and working but I have to figure out how to enable the Reading operations (checkout,... etc) without need to login just lack of time to read and code, this is one of my next steps, at least people can access to code with webfrontend, those who wants access please let me know (Markus? :P) Count me in! Shall I give you an ssh public key or how do we do this? Hi ssh+svn through a public key maybe is the most comfortable option, so send me a private mail with user name do you want i and a public key to generate the key(Markus sure you know but maybe other willing to colaborate and this aplies to anyone wanting to code on this) on the pc you want to use your local svn clone $ mkdir ~/.ssh/ $ cd ~/.ssh/ $ ssh-keygen -t dsa and send to me the ~/.ssh/id_dsa.pub the only difference from the original apt-portal checkout is opkg2sql.py[1] script and sqlite db file[2] generated from the succesfull import Hmm the db might be a bit volatile to be in the VCS, considering it is regenerated easily, but ok :) Yes surelly this will get out of the vcs soon, but maybe someone is curiouse about structure or the info extracted from the repo so this was a qd to make it available for download, and ... what the hell !! I was proud of the success and want to show it :P -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Tue, Sep 01, 2009 at 07:23:50AM +0200, David Reyes Samblas Martinez wrote: Just wondering what's happening :) the 8215 packages from all, armv4t and gta-02 from shr-unstable parsed successfully using the latest changes of Joao :) Wonderful news! What's the next step then? Set something up the dev server? You want my help with that? Then we need to figure out features and a kind of a roadmap! :) -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Mon, Aug 31, 2009 at 10:19:07PM +0100, Joao Pinto wrote: Hello, sorry but I need to quit the list, it has more traffic than I can handle :( Heh, I ignore threads that don't look relevant to the showroom, but sure :) Please feel free add me on the cc whenever you would like to have my input. I will also be available on IRC as always :) Obrigado :) -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
Just wondering what's happening :) the 8215 packages from all, armv4t and gta-02 from shr-unstable parsed successfully using the latest changes of Joao :) WOW David, this is awesome Now a) filter all libraries away b) maybe filter away files with no proper .desktop file.. Or something not to have it bloated and impossible to find anything there :) So what's the thing with descriptions? Do they have to be filled in manually - no changes of using some importer script there too? Screenshots need to be added manually of course. David: I'd like you to share your work with at least Markus. It's nice to hear that you're doing progress but as this is supposed to be a community effort, it's vital that also some other know about what's happening and can go and learn the code we're using etc. Can't wait to hear/see more. r -- | risto h. kurppa | risto at kurppa dot fi | http://risto.kurppa.fi ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Tue, Sep 01, 2009 at 09:54:46AM +0300, Risto H. Kurppa wrote: Now a) filter all libraries away b) maybe filter away files with no proper .desktop file.. Yeah, common sense ftw, maybe some libraries may or may not be relevant to our interests, but we'd figure those out. Do you have a guess on what the shares are of libraries and packages without .desktop? Or something not to have it bloated and impossible to find anything there :) +1 So what's the thing with descriptions? Do they have to be filled in manually - no changes of using some importer script there too? Screenshots need to be added manually of course. If a source provides only only binary, we should be able to take that binary's description and use it for the application. Note that source, binary and application are different concepts here. David: I'd like you to share your work with at least Markus. It's nice to hear that you're doing progress but as this is supposed to be a community effort, it's vital that also some other know about what's happening and can go and learn the code we're using etc. Preferrably not just me, chacnes are someone with more spare time would accomplish things faster, but at least I still like to help out :) Thanks guys! -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
2009/9/1 Markus Törnqvist m...@nysv.org: On Tue, Sep 01, 2009 at 09:54:46AM +0300, Risto H. Kurppa wrote: Now a) filter all libraries away b) maybe filter away files with no proper .desktop file.. Yeah, common sense ftw, maybe some libraries may or may not be relevant to our interests, but we'd figure those out. Do you have a guess on what the shares are of libraries and packages without .desktop? Or something not to have it bloated and impossible to find anything there :) +1 Ok this will be my next step on coding, I will try to make is good enough to be upstream on apt-portal, So what's the thing with descriptions? Do they have to be filled in manually - no changes of using some importer script there too? Screenshots need to be added manually of course. Most of them are only one line description and sure can be used as short description for lists but I thing they are poor for fill an the detailed app description, nevertheless we can use them as initialization of description when app is added to the showroom, one line is better than nothing, both bd and code seems clear an easy enough to implement this in the specific Neo showroom app :) If a source provides only only binary, we should be able to take that binary's description and use it for the application. Note that source, binary and application are different concepts here. To check if I understand it well(using shr as example) souce - ipkg repository binary-ipkg package application-description+image+package/s related+[opt]video+[opt]comments+[opt]opt votes piece of showroom for an specific software application David: I'd like you to share your work with at least Markus. It's nice to hear that you're doing progress but as this is supposed to be a community effort, it's vital that also some other know about what's happening and can go and learn the code we're using etc. Preferrably not just me, chacnes are someone with more spare time would accomplish things faster, but at least I still like to help out :) Thanks guys! Setting up a ssh+svn +webdav on the server(yes we have it :) ) to allow this I will grant ssh+svn access to anyone interested in participate, and allow anonymous checkout , after this I wil set up a webserver with two sites one will be the stable/release of the showroom and other using directly the developing version from that svn of course the devel version will be first available :) I'm really noob in all that matters but I hope to have the ssh+svn+webdab setup this night -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Tue, Sep 1, 2009 at 1:40 PM, David Reyes Samblas Martinezda...@tuxbrain.com wrote: Most of them are only one line description and sure can be used as short description for lists but I thing they are poor for fill an the detailed app description, nevertheless we can use them as initialization of description when app is added to the showroom, one line is better than nothing, both bd and code seems clear an easy enough to implement this in the specific Neo showroom app :) Sound's good! Setting up a ssh+svn +webdav on the server(yes we have it :) ) to allow this I will grant ssh+svn access to anyone interested in participate, and allow anonymous checkout , after this I wil set up a webserver with two sites one will be the stable/release of the showroom and other using directly the developing version from that svn of course the devel version will be first available :) Sound's good! I'm really noob in all that matters but I hope to have the ssh+svn+webdab setup this night hmm.. okay.. make sure you have decent backups :D r -- | risto h. kurppa | risto at kurppa dot fi | http://risto.kurppa.fi ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
I CC'd our friend Joao on this :) On Tue, Sep 01, 2009 at 12:40:39PM +0200, David Reyes Samblas Martinez wrote: So what's the thing with descriptions? Do they have to be filled in manually - no changes of using some importer script there too? Screenshots need to be added manually of course. Most of them are only one line description and sure can be used as short description for lists but I thing they are poor for fill an the detailed app description, nevertheless we can use them as initialization of description when app is added to the showroom, one line is better than nothing, both bd and code seems clear an easy enough to implement this in the specific Neo showroom app :) Yeah :) If a source provides only only binary, we should be able to take that binary's description and use it for the application. Note that source, binary and application are different concepts here. To check if I understand it well(using shr as example) souce - ipkg repository binary-ipkg package source is the source package, repository is repository. There's a separate table application which contains the description, which is not the same as a package's description, but if we have only one binary per application, we can use the binary package's description. That's where source comes in, tho it's not really in the db, which would be to find out how many binaries the source produces... application-description+image+package/s We need a table for images! I'm pretty sure we went over this somewhere bit looking at sqlite3 getdeb.db and .schema I can't find a relation between package and application! There's application.source_package VARCHAR(128) and package.source VARCHAR(64) but surely that's not used for joining ;) related+[opt]video+[opt]comments+[opt]opt votes piece of showroom for an specific software application Ooh video, I like that :) But this is tangenting on the roadmap, in which order are we going to do features. I sort of like the idea of doing images first, because it's a showroom after all. Thanks guys! Setting up a ssh+svn +webdav on the server(yes we have it :) ) to allow this I will grant ssh+svn access to anyone interested in participate, and allow anonymous checkout , after this I wil set up a webserver with two sites one will be the stable/release of the showroom and other using directly the developing version from that svn of course the devel version will be first available :) I'm really noob in all that matters but I hope to have the ssh+svn+webdab setup this night If that proves difficult, maybe git is an alternative? I've been using git for so long I can't even remember svn, I'd probably use git-svn for development too :D Thanks!! :) -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
2009/9/1 Risto H. Kurppa ri...@kurppa.fi: On Tue, Sep 1, 2009 at 1:40 PM, David Reyes Samblas Martinezda...@tuxbrain.com wrote: Most of them are only one line description and sure can be used as short description for lists but I thing they are poor for fill an the detailed app description, nevertheless we can use them as initialization of description when app is added to the showroom, one line is better than nothing, both bd and code seems clear an easy enough to implement this in the specific Neo showroom app :) Sound's good! Setting up a ssh+svn +webdav on the server(yes we have it :) ) to allow this I will grant ssh+svn access to anyone interested in participate, and allow anonymous checkout , after this I wil set up a webserver with two sites one will be the stable/release of the showroom and other using directly the developing version from that svn of course the devel version will be first available :) Sound's good! I'm really noob in all that matters but I hope to have the ssh+svn+webdab setup this night hmm.. okay.. make sure you have decent backups :D r -- | risto h. kurppa | risto at kurppa dot fi | http://risto.kurppa.fi ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community There is some(a lot of) work to be done yet, ex. cleaning all unnecessary imported files to the repo mostly all bzr stuff from original apt-portal repo , configure Apache to work with cherrypy, and set up with a stable and devel om-showrom site ;), and of course make at least a devel om-showrom site :P but here is one step more of this long road. https://tuxbrain.org/websvn/ also access to svn though ssh or https are setup and working but I have to figure out how to enable the Reading operations (checkout,... etc) without need to login just lack of time to read and code, this is one of my next steps, at least people can access to code with webfrontend, those who wants access please let me know (Markus? :P) the only difference from the original apt-portal checkout is opkg2sql.py[1] script and sqlite db file[2] generated from the succesfull import [1]https://tuxbrain.org/websvn/filedetails.php?repname=om-showroompath=%2Ftrunk%2Fdebfactory%2Fbin%2Fopkg2sql.py [2]https://tuxbrain.org/websvn/filedetails.php?repname=om-showroompath=%2Ftrunk%2Fshrunstable.db -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Wed, Sep 02, 2009 at 04:09:30AM +0200, David Reyes Samblas Martinez wrote: [...] course make at least a devel om-showrom site :P but here is one step more of this long road. https://tuxbrain.org/websvn/ Hooray! also access to svn though ssh or https are setup and working but I have to figure out how to enable the Reading operations (checkout,... etc) without need to login just lack of time to read and code, this is one of my next steps, at least people can access to code with webfrontend, those who wants access please let me know (Markus? :P) Count me in! Shall I give you an ssh public key or how do we do this? the only difference from the original apt-portal checkout is opkg2sql.py[1] script and sqlite db file[2] generated from the succesfull import Hmm the db might be a bit volatile to be in the VCS, considering it is regenerated easily, but ok :) -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Tue, Sep 01, 2009 at 11:00:01PM +0100, Joao Pinto wrote: I'm pretty sure we went over this somewhere bit looking at sqlite3 getdeb.db and .schema I can't find a relation between package and application! There's application.source_package VARCHAR(128) and package.source VARCHAR(64) but surely that's not used for joining ;) This relation is done by the source_package field, applications are related to a single source package, the source package must provide a binary package classified with m main to be shown on the updates list. So yes it is used for joining despite the lack of enforcement of referential integrity. Is this as heavy as it sounds to me?-) Either doing JOIN over VARCHAR or in code iterating results in a for loop and doing SELECT over them; both options are heavy. I know the showroom/apt-portal at large isn't necessarily useful for sources, but would it make sense to model source packages, just to get a normal foreign key? Thanks! -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Sun, Aug 30, 2009 at 09:00:44AM +0200, David Reyes Samblas Martinez wrote: Joao, has those fix been uploaded to the bzr repo? Have you checked it out? Just wondering what's happening :) -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
Hello, sorry but I need to quit the list, it has more traffic than I can handle :( Please feel free add me on the cc whenever you would like to have my input. I will also be available on IRC as always :) Best regards On Mon, Aug 31, 2009 at 7:25 PM, Markus Törnqvistm...@nysv.org wrote: On Sun, Aug 30, 2009 at 09:00:44AM +0200, David Reyes Samblas Martinez wrote: Joao, has those fix been uploaded to the bzr repo? Have you checked it out? Just wondering what's happening :) -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community -- João Luís Marques Pinto GetDeb Team Leader http://www.getdeb.net http://blog.getdeb.net ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
2009/8/31 Markus Törnqvist m...@nysv.org: On Sun, Aug 30, 2009 at 09:00:44AM +0200, David Reyes Samblas Martinez wrote: Joao, has those fix been uploaded to the bzr repo? Have you checked it out? Just wondering what's happening :) the 8215 packages from all, armv4t and gta-02 from shr-unstable parsed successfully using the latest changes of Joao :) -- mjt ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
2009/8/29 Joao Pinto joao.pi...@getdeb.net:
On Sat, Aug 29, 2009 at 11:48 AM, Markus Törnqvistm...@nysv.org wrote:
On Sat, Aug 29, 2009 at 12:27:49PM +0300, Markus T�rnqvist wrote:
After all that typing, I say we just remove the encode() call and use
utf-8 across the board ;)
The below patch works
--- opkg2sql.py.orig 2009-08-29 13:46:21.004642989 +0300
+++ opkg2sql.py 2009-08-29 13:46:05.995767671 +0300
@@ -135,7 +135,7 @@
source = control['Source']
version = control['Version']
architecture = control['Architecture']
- description = control['Description'].encode('ascii','ignore')
+ description = unicode(control['Description'])
homepage = control['HomePage']
package = Package.query.filter_by( \
package = package_name, \
--
mjt
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
I have applied the fix at dpkg_control.py, converting the entire
control file contents to utf8.
Thanks
--
João Luís Marques Pinto
GetDeb Team Leader
http://www.getdeb.net
http://blog.getdeb.net
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Joao, has those fix been uploaded to the bzr repo?
--
David Reyes Samblas Martinez
http://www.tuxbrain.com
Open ultraportable embedded solutions
Openmoko, Openpandora, Arduino
Hey, watch out!!! There's a linux in your pocket!!!
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
Joao, has those fix been uploaded to the bzr repo? -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community David, yes, the fix was commited: http://bazaar.launchpad.net/~apt-portal-devs/apt-portal/devel/revision/123 -- João Luís Marques Pinto GetDeb Team Leader http://www.getdeb.net http://blog.getdeb.net ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
Am Samstag 29 August 2009 04:31:28 schrieb David Reyes Samblas Martinez: the problem persist, so the conversion to ascii doesn't solve it, the problematic package description is pyelf-sudoku, it seems that is not on UTF-8 nor ascii, is that posible? Can any one on SHR mantainers take a look on this and fix it?, surelly the origin is in the description on the bb file ,isn't it? now is late but later on I will continue with import hardcoding the skip of this packages to see if there is another one problematic. It is UTF-8 (at least if I can believe my vim which ususally I can :). The only special char in there is the × in 9×9 and 3×3 -- Klaus 'mrmoku' Kurzmann ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Sat, Aug 29, 2009 at 12:27:49PM +0300, Markus T�rnqvist wrote:
After all that typing, I say we just remove the encode() call and use
utf-8 across the board ;)
The below patch works
--- opkg2sql.py.orig 2009-08-29 13:46:21.004642989 +0300
+++ opkg2sql.py 2009-08-29 13:46:05.995767671 +0300
@@ -135,7 +135,7 @@
source = control['Source']
version = control['Version']
architecture = control['Architecture']
- description = control['Description'].encode('ascii','ignore')
+ description = unicode(control['Description'])
homepage = control['HomePage']
package = Package.query.filter_by( \
package = package_name, \
--
mjt
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
On Sat, Aug 29, 2009 at 11:48 AM, Markus Törnqvistm...@nysv.org wrote:
On Sat, Aug 29, 2009 at 12:27:49PM +0300, Markus T�rnqvist wrote:
After all that typing, I say we just remove the encode() call and use
utf-8 across the board ;)
The below patch works
--- opkg2sql.py.orig 2009-08-29 13:46:21.004642989 +0300
+++ opkg2sql.py 2009-08-29 13:46:05.995767671 +0300
@@ -135,7 +135,7 @@
source = control['Source']
version = control['Version']
architecture = control['Architecture']
- description = control['Description'].encode('ascii','ignore')
+ description = unicode(control['Description'])
homepage = control['HomePage']
package = Package.query.filter_by( \
package = package_name, \
--
mjt
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
I have applied the fix at dpkg_control.py, converting the entire
control file contents to utf8.
Thanks
--
João Luís Marques Pinto
GetDeb Team Leader
http://www.getdeb.net
http://blog.getdeb.net
___
Openmoko community mailing list
community@lists.openmoko.org
http://lists.openmoko.org/mailman/listinfo/community
apt-portal import opkg repos :) some comments
Meanwhile the design, and the server is going on I was working on the backend and I hope this will help Attached is the adaptation of apt2sql to work on opkg repositories opkg2sql.py. now at least it provided almost the same information as apt2sql I have succesfuly import the all architecture, but I have some problems with the arm4t due a bug on the version of python-sqlalchemy with UTF-8 chars, I have forced it to ascii and trying again. Regarding apt-portal For those worried about package filtering and problems on automatic import the repo don't worry , as far as I can see (Joao please tell me if I'm wrong) the package import is not the information finally showing on description on the aplication showcase in fact app/app clasification/app info has to be introduced by hand with the web editor/adminstrator or directly in the table and you select wich package is related , on playdeb I see you can hide package classified them as int even if they are linked to any app. Well classifying all the thousand packages this way will be a little bit wasting of time so we will have to filter them previously to enter on the bd, there is a field called Section in the Package file of the repo that can be used for that propose I guess, filtering all packages marked as base, devel, etc, etc we must work on this list and look if we have to create another classes finally to manage the opkg packages instead of reusing the debian ones. BTW, I have not successfully yet classified a packages and make it apperar in the Uptades section from the demo playdeb application, they appears as linked and classified in the Packages tab I really noob on python, in fact this is my first python project (not counting the PoCoMoko attempt) I'm using my rusty php programing skills, but I think we can do great things with this apt-portal as base. -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! #!/usr/bin/python # # (C) Copyright 2009, TUXBRAIN S.L. - http://www.tuxbrain.com # # Based on apt2sql.py from GetDeb Team - https://launchpad.net/~getdeb # This program is free software: you can redistribute it and/or modify # it under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, # but WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the # GNU General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program. If not, see http://www.gnu.org/licenses/. # # # This file imports opkg packages from a repository to an sql database # control files. Usage: opkg2sql.py [--database mysql://user:passw...@localhost/apt2sql] \ [archive_root_url version arquitecture1[,arquetecture2,... ] Example: opkg2sql.py http://build.shr-project.org shr-unstable all # sqlalchemy uses a deprecated module import warnings warnings.simplefilter(ignore,DeprecationWarning) import sys import os import socket import urllib2 import zlib import gzip import tempfile import re from datetime import datetime from optparse import OptionParser from urllib2 import Request, urlopen, URLError, HTTPError from localaux import * from packages_model import * from dpkg_control import * from lockfile import * Log = Logger() def get_last_mofified_time(file_url): Returns the last mofidifed time for the specified url try: f = urllib2.urlopen(file_url) except HTTPError, e: Log.print_(Error %s : %s % (e.code, file_url)) return None last_modified = f.info()['Last-Modified'] f.close() d_last_modified = datetime.strptime(last_modified, '%a, %d %b %Y %H:%M:%S %Z') return d_last_modified #Due opkg lacks of Release file and informed PackageList #this is a simplified version of the original apt2sql.py fuction #avoinding checkings done using those files #TODO Maybe some arch checkings can be done scaning directories directly from # the html returned of the archive_url/version/ikpg directory def import_repository(archive_url, version, architectures): Import a repository into the dabase # Now let's import the Packages file for each architecture # Some redundancies and fixes values where done to mantain the same # class structure of Package and PackageList # TODO opkg is also used in OpenWRT based distros so we must find a way # to discrimite the Origin value now is hardcoded to OE for arch in architectures: packages_file = %s/%s/ipk/%s/Packages.gz \ % (archive_url, version, arch) packagelist = \
Re: apt-portal import opkg repos :) some comments
the problem persist, so the conversion to ascii doesn't solve it, the problematic package description is pyelf-sudoku, it seems that is not on UTF-8 nor ascii, is that posible? Can any one on SHR mantainers take a look on this and fix it?, surelly the origin is in the description on the bb file ,isn't it? now is late but later on I will continue with import hardcoding the skip of this packages to see if there is another one problematic. the pitty is if one package fail the app does not commit and the previous packages are not imported... and there are quite a lot before pyelf-sudoku :) I forget to mention the command used to test opkg2sql : opkg2sql.py -d sqlite:///shrunstable.db \ http://build.shr-project.org shr-unstable \ all,armv4t,om-gta02 -- David Reyes Samblas Martinez http://www.tuxbrain.com Open ultraportable embedded solutions Openmoko, Openpandora, Arduino Hey, watch out!!! There's a linux in your pocket!!! ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
Re: apt-portal import opkg repos :) some comments
David, awesome work there! Can't wait to see you release the devel version of the repo. Nice spotting broken pyelf-sudoku, I hope it'll get fixed soon. And I also hope the dpkg-branch could live in the original apt-portal project to avoid too much division. It sound's a bit nasty to me to write all package descriptions manually.. But if that has to be then we'll do it. Or maybe someone's skilled enough to write the python+sql lines to do the work for us.. If you want to discuss the apt-portal stuff live, you're welcome to join #getdeb (live java client: http://java.freenode.net//index.php?channel=getdeb) r -- | risto h. kurppa | risto at kurppa dot fi | http://risto.kurppa.fi ___ Openmoko community mailing list community@lists.openmoko.org http://lists.openmoko.org/mailman/listinfo/community
