Re: [Sugar-devel] [ANNOUNCE] Get Books Activity 3, now with SneakerNet/ChappalNet support
On Mon, Nov 9, 2009 at 4:16 AM, Jim Simmons nices...@gmail.com wrote: Sayamindu, I'm not sure why you think you'd need to use multithreading to deal with the multiple chunks. Can't you just bring them in one at a time, parse as you go, and then display the finished result set all at once? I'm a bit worried with incredibly long result sets (for example, a USB stick with a few thousand entries). I saw your Get Books activity got praised on the OLPC blog as being a key part of their latest efforts with Internet Archive books. (My name was listed also, but it was clearly your Activity they were praising). I'm not sure about this, but I think this was done to push OPDS as much as possible, which I believe is a sensible thing to do :-) It just seemed to me that it would be a good idea to give your efforts an official status (its own git repository, wiki page, ASLO downloads, etc.) as soon as possible. Even with the shortcomings you mention it deserves that much. If you want to keep the original around for awhile it's fine with me. In the short run it won;'t hurt anything. In the long term it makes more sense to have just one Activity to find and get books on the Internet and just one Activity to read them. I understood from you that eventually Read would be able to use the formats I wrote View Slides and Read Etexts for. Those Activities could also stay around but they might be of more use to teachers and other adults than they would be to the young students Sugar is targetted at. Agreed. I think will require a couple of releases more to fix the immediate problems, and then we could perhaps make the activity more official. Thanks, Sayamindu James Simmons On Sun, Nov 8, 2009 at 2:22 PM, Sayamindu Dasgupta sayami...@gmail.com wrote: Hi Jim, I think GIAB should stay. Get Books still has a few deficiencies : * Metadata: GIAB seems to pull in more metadata that what is currently offered by OPDS. * Large resultsets: The OPDS results returned by IA are chunked into segments of 50 items each. Get Books only handles the first segment. Getting it to handle the other segments (essentially, loading them automagically as the user scrolls down) requires quite a bit of coding effort (and the optimal way here would be probably to use threading, and threading in PyGTK can be a dangerous thing to tread on) - there is no timeframe for its implementation yet. * Formats: While the current format code is hardcoded, and I will offer whatever is there in the OPDS data from next version, it will not cover all the formats from IA. I would prefer not to hard-code anything (since I'm handling a number of sources, hardcoding for a particular source will lead to a lot of messy code) Let me know what you think. Thanks, Sayamindu On Sat, Nov 7, 2009 at 2:15 AM, Jim Simmons nices...@gmail.com wrote: Sayamindu, Regarding the friendly fork of Get Internet Archive Books, I'm thinking that perhaps you will soon have Get Books in good enough shape that the original Activity will no longer be needed, and at that point you should merge your clone with the mainline and take over the Activity. About the only thing that GIAB can do that you can't (yet) is give the user a choice of download formats. It shouldn't be too tough to implement that option if you restrict it to the Internet Archive. I could rename the original Activity in ASLO and rename its wiki page, then give you authority in git to update the mainline. You would need to renumber your Activity version to number 4, as GIAB is currently at version 3. It's pretty clear that GIAB is a dead end and what you're doing is the future. You've even addressed the future needs of the Rural Design Collective for distributing children's books on a thumb drive, which was something I had hoped to do with GIAB at some point. Let me know what you think. James Simmons Date: Fri, 6 Nov 2009 04:26:07 +0530 From: Sayamindu Dasgupta sayami...@gmail.com Subject: [Sugar-devel] [ANNOUNCE] Get Books Activity 3, now with SneakerNet/ChappalNet support To: OLPC Bookreader list bookrea...@lists.laptop.org, Sugar devel sugar-devel@lists.sugarlabs.org Message-ID: a1314bfe0911051456k2e7791a8h989e41265d127...@mail.gmail.com Content-Type: text/plain; charset=UTF-8 Hello, Get Books Activity is a friendly fork of Jim Simmon's Get Internet Archive Books activity and it allows Sugar users to search for and download Ebooks from various sources. One of the major points of concern for many was this activity required Internet access, and I have tried to address that issue with this release. The activity can now access catalogs in removable devices (pen drives, etc) and lets the user search through these catalogs. A removable device with a catalog will show up in the list of sources from which the books can be downloaded/copied. Utilizing this feature, one will be able to copy thousands
Re: [Sugar-devel] [ANNOUNCE] Get Books Activity 3, now with SneakerNet/ChappalNet support
Sayamindu, I followed your instructions to test Get Books on my XO. I had a USB drive with the unzipped data you provided such that the XML file and the books directory were in the root directory of the drive. I also used the thumb drive to copy the Activity from the drive to the Journal. There was nothing else in the drive other than your Activity and the unzipped data. When I tried to start Get Books it failed to start and the Log activity showed it reporting an XML error, something to do with not having a child element. I'm running the latest .82 on my XO. I will try to get you better information on this later. I'm at work now and I did the test on my lunch break. James Simmons I do not consider this release to be stable, so I have not uploaded it to ASLO yet (I had to do a lot more refactoring that I would have liked to get the removable device support working). To download the activity (it should work with Sucrose 0.82 upwards), follow the link: http://dev.laptop.org/~sayamindu/GetBooks-3.xo To test the removable device support, in a USB drive, unzip http://dev.laptop.org/~sayamindu/books_on_a_stick.zip [2] (make sure that the catalog.xml file stays in the toplevel directory of the drive), and plug the USB drive in wherever you are running Sugar. Get Books is _supposed_ to detect removable devices as and when they are plugged in/removed, but if your device does not show up, try restarting the activity before trying anything else. Thanks, Sayamindu ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [ANNOUNCE] Get Books Activity 3, now with SneakerNet/ChappalNet support
Hi Jim, I think GIAB should stay. Get Books still has a few deficiencies : * Metadata: GIAB seems to pull in more metadata that what is currently offered by OPDS. * Large resultsets: The OPDS results returned by IA are chunked into segments of 50 items each. Get Books only handles the first segment. Getting it to handle the other segments (essentially, loading them automagically as the user scrolls down) requires quite a bit of coding effort (and the optimal way here would be probably to use threading, and threading in PyGTK can be a dangerous thing to tread on) - there is no timeframe for its implementation yet. * Formats: While the current format code is hardcoded, and I will offer whatever is there in the OPDS data from next version, it will not cover all the formats from IA. I would prefer not to hard-code anything (since I'm handling a number of sources, hardcoding for a particular source will lead to a lot of messy code) Let me know what you think. Thanks, Sayamindu On Sat, Nov 7, 2009 at 2:15 AM, Jim Simmons nices...@gmail.com wrote: Sayamindu, Regarding the friendly fork of Get Internet Archive Books, I'm thinking that perhaps you will soon have Get Books in good enough shape that the original Activity will no longer be needed, and at that point you should merge your clone with the mainline and take over the Activity. About the only thing that GIAB can do that you can't (yet) is give the user a choice of download formats. It shouldn't be too tough to implement that option if you restrict it to the Internet Archive. I could rename the original Activity in ASLO and rename its wiki page, then give you authority in git to update the mainline. You would need to renumber your Activity version to number 4, as GIAB is currently at version 3. It's pretty clear that GIAB is a dead end and what you're doing is the future. You've even addressed the future needs of the Rural Design Collective for distributing children's books on a thumb drive, which was something I had hoped to do with GIAB at some point. Let me know what you think. James Simmons Date: Fri, 6 Nov 2009 04:26:07 +0530 From: Sayamindu Dasgupta sayami...@gmail.com Subject: [Sugar-devel] [ANNOUNCE] Get Books Activity 3, now with SneakerNet/ChappalNet support To: OLPC Bookreader list bookrea...@lists.laptop.org, Sugar devel sugar-devel@lists.sugarlabs.org Message-ID: a1314bfe0911051456k2e7791a8h989e41265d127...@mail.gmail.com Content-Type: text/plain; charset=UTF-8 Hello, Get Books Activity is a friendly fork of Jim Simmon's Get Internet Archive Books activity and it allows Sugar users to search for and download Ebooks from various sources. One of the major points of concern for many was this activity required Internet access, and I have tried to address that issue with this release. The activity can now access catalogs in removable devices (pen drives, etc) and lets the user search through these catalogs. A removable device with a catalog will show up in the list of sources from which the books can be downloaded/copied. Utilizing this feature, one will be able to copy thousands of ebooks in a pen-drive, put a catalog file (catalog.xml) in the root[1] directory of the device, and send it to bandwidth starved areas. (think of a library on a stick ;-) Apart from the above features, this release also has some subtle changes in the behavior which will hopefully make the overall use experience better. (for example, changing the source in the middle of a search will trigger a new search within the new selected source) I do not consider this release to be stable, so I have not uploaded it to ASLO yet (I had to do a lot more refactoring that I would have liked to get the removable device support working). To download the activity (it should work with Sucrose 0.82 upwards), follow the link: http://dev.laptop.org/~sayamindu/GetBooks-3.xo To test the removable device support, in a USB drive, unzip http://dev.laptop.org/~sayamindu/books_on_a_stick.zip [2] (make sure that the catalog.xml file stays in the toplevel directory of the drive), and plug the USB drive in wherever you are running Sugar. Get Books is _supposed_ to detect removable devices as and when they are plugged in/removed, but if your device does not show up, try restarting the activity before trying anything else. Thanks, Sayamindu -- Sayamindu Dasgupta [http://sayamindu.randomink.org/ramblings] ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
Re: [Sugar-devel] [ANNOUNCE] Get Books Activity 3, now with SneakerNet/ChappalNet support
Sayamindu, I'm not sure why you think you'd need to use multithreading to deal with the multiple chunks. Can't you just bring them in one at a time, parse as you go, and then display the finished result set all at once? I saw your Get Books activity got praised on the OLPC blog as being a key part of their latest efforts with Internet Archive books. (My name was listed also, but it was clearly your Activity they were praising). It just seemed to me that it would be a good idea to give your efforts an official status (its own git repository, wiki page, ASLO downloads, etc.) as soon as possible. Even with the shortcomings you mention it deserves that much. If you want to keep the original around for awhile it's fine with me. In the short run it won;'t hurt anything. In the long term it makes more sense to have just one Activity to find and get books on the Internet and just one Activity to read them. I understood from you that eventually Read would be able to use the formats I wrote View Slides and Read Etexts for. Those Activities could also stay around but they might be of more use to teachers and other adults than they would be to the young students Sugar is targetted at. James Simmons On Sun, Nov 8, 2009 at 2:22 PM, Sayamindu Dasgupta sayami...@gmail.com wrote: Hi Jim, I think GIAB should stay. Get Books still has a few deficiencies : * Metadata: GIAB seems to pull in more metadata that what is currently offered by OPDS. * Large resultsets: The OPDS results returned by IA are chunked into segments of 50 items each. Get Books only handles the first segment. Getting it to handle the other segments (essentially, loading them automagically as the user scrolls down) requires quite a bit of coding effort (and the optimal way here would be probably to use threading, and threading in PyGTK can be a dangerous thing to tread on) - there is no timeframe for its implementation yet. * Formats: While the current format code is hardcoded, and I will offer whatever is there in the OPDS data from next version, it will not cover all the formats from IA. I would prefer not to hard-code anything (since I'm handling a number of sources, hardcoding for a particular source will lead to a lot of messy code) Let me know what you think. Thanks, Sayamindu On Sat, Nov 7, 2009 at 2:15 AM, Jim Simmons nices...@gmail.com wrote: Sayamindu, Regarding the friendly fork of Get Internet Archive Books, I'm thinking that perhaps you will soon have Get Books in good enough shape that the original Activity will no longer be needed, and at that point you should merge your clone with the mainline and take over the Activity. About the only thing that GIAB can do that you can't (yet) is give the user a choice of download formats. It shouldn't be too tough to implement that option if you restrict it to the Internet Archive. I could rename the original Activity in ASLO and rename its wiki page, then give you authority in git to update the mainline. You would need to renumber your Activity version to number 4, as GIAB is currently at version 3. It's pretty clear that GIAB is a dead end and what you're doing is the future. You've even addressed the future needs of the Rural Design Collective for distributing children's books on a thumb drive, which was something I had hoped to do with GIAB at some point. Let me know what you think. James Simmons Date: Fri, 6 Nov 2009 04:26:07 +0530 From: Sayamindu Dasgupta sayami...@gmail.com Subject: [Sugar-devel] [ANNOUNCE] Get Books Activity 3, now with SneakerNet/ChappalNet support To: OLPC Bookreader list bookrea...@lists.laptop.org, Sugar devel sugar-devel@lists.sugarlabs.org Message-ID: a1314bfe0911051456k2e7791a8h989e41265d127...@mail.gmail.com Content-Type: text/plain; charset=UTF-8 Hello, Get Books Activity is a friendly fork of Jim Simmon's Get Internet Archive Books activity and it allows Sugar users to search for and download Ebooks from various sources. One of the major points of concern for many was this activity required Internet access, and I have tried to address that issue with this release. The activity can now access catalogs in removable devices (pen drives, etc) and lets the user search through these catalogs. A removable device with a catalog will show up in the list of sources from which the books can be downloaded/copied. Utilizing this feature, one will be able to copy thousands of ebooks in a pen-drive, put a catalog file (catalog.xml) in the root[1] directory of the device, and send it to bandwidth starved areas. (think of a library on a stick ;-) Apart from the above features, this release also has some subtle changes in the behavior which will hopefully make the overall use experience better. (for example, changing the source in the middle of a search will trigger a new search within the new selected source) I do not consider this release to be stable, so I
[Sugar-devel] [ANNOUNCE] Get Books Activity 3, now with SneakerNet/ChappalNet support
Hello, Get Books Activity is a friendly fork of Jim Simmon's Get Internet Archive Books activity and it allows Sugar users to search for and download Ebooks from various sources. One of the major points of concern for many was this activity required Internet access, and I have tried to address that issue with this release. The activity can now access catalogs in removable devices (pen drives, etc) and lets the user search through these catalogs. A removable device with a catalog will show up in the list of sources from which the books can be downloaded/copied. Utilizing this feature, one will be able to copy thousands of ebooks in a pen-drive, put a catalog file (catalog.xml) in the root[1] directory of the device, and send it to bandwidth starved areas. (think of a library on a stick ;-) Apart from the above features, this release also has some subtle changes in the behavior which will hopefully make the overall use experience better. (for example, changing the source in the middle of a search will trigger a new search within the new selected source) I do not consider this release to be stable, so I have not uploaded it to ASLO yet (I had to do a lot more refactoring that I would have liked to get the removable device support working). To download the activity (it should work with Sucrose 0.82 upwards), follow the link: http://dev.laptop.org/~sayamindu/GetBooks-3.xo To test the removable device support, in a USB drive, unzip http://dev.laptop.org/~sayamindu/books_on_a_stick.zip [2] (make sure that the catalog.xml file stays in the toplevel directory of the drive), and plug the USB drive in wherever you are running Sugar. Get Books is _supposed_ to detect removable devices as and when they are plugged in/removed, but if your device does not show up, try restarting the activity before trying anything else. Thanks, Sayamindu [1] Of course, manually cataloging thousands of books is a PITA, so I will soon release something like the Fedora LiveCD script - which will let you choose a directory of EPUB files and a USB disk, and generate the catalog, as well as convert the EPUBs to PDFs (with the corresponding linking in the catalog) so that older builds can read the files. [2] After uploading the zip I realized that the PDFs in the zip might be a bit too heavy for the XO and similar machines - apologies for that. The Epubs work fine on XO-1.5 and newer versions of Sugar [3] Chappal, in case you are wondering is the South Asian generic term for flip-flops/sandals. I hate wearing sneakers :-) -- Sayamindu Dasgupta [http://sayamindu.randomink.org/ramblings] ___ Sugar-devel mailing list Sugar-devel@lists.sugarlabs.org http://lists.sugarlabs.org/listinfo/sugar-devel
