Hey Srevin,
This looks really good! Page load times, search was fast. Haven't tried
downloading any Activities though.
It may just be me but the page has a little too much contrast with the
colors.Pagination might also be a good addition. Cool lava lamp toggle :P
Will be waiting for a full release š
Best,Shaan
On Tuesday, June 16, 2020, 9:24:53 PM GMT+5:30, Srevin Saju
<srevins...@sugarlabs.org> wrote:
Hello,
Hope all are having a safe week amid nCoV.
I and Manish (su...@radii.dev) had been working on the GSoC project which did
not get a slot (Python3 app store aslov4), and I guess we have made good
progress in it. We decided to collaborate on the project because a frontend and
backend developer combo helps to get the best features of both ends :D.
To avoid misconceptions and misinterpretations, please read the entire email
before replying. Thanks!Ā Ā :P
Need Statement:
In short, the most important points
- Create the simplest possible app store for Sugar activities, where each
activity included has been (a) ported to Python 3 and released, and (b) tested
on Sugar Live Build.
- We used to have an app store for Sugar activities, but because we can't
seem to attract any PHP developers the app store has failed to keep up with
development.
- We now use activities.sugarlabs.org for Python 2 activities only.
- We have tried to make a replacement for activities.sugarlabs.org three
times, and each time the features we need were not finished. These projects
have been too ambitious and have not been supported collectively by the Sugar
Labs community.
~ from
https://github.com/sugarlabs/GSoC/blob/master/Ideas-2020.md#sugar-app-store-for-python-3-activities-aslov4
This made us rethink the idea of a redesigned app store for sugarlabs. This,
would be a collection of all the activities in sugarlabs organization which can
be successfully built without errors.
The new app store is special in a lot of ways, let me bring out the most
important points:
- The appstore is built in pure static HTML, Vanilla Javascript and JQuery.
Anyone with basic Javascript knowledge can easily add features. Theming and
customization are provided by Bootstrap4. The theme, scripts are completely
independent of each other. Changes to the will not affect the generation of the
appstore.
- Search function is provided by a powerful Javascript library called
minisearch which is an open source search indexer. All processing is therefore
done on client side. The benefits of using an external library are many, as you
know,
- The App Store is based on JSON (Javascript Object Notation). an
"index.json" is produced by the generator helper script so that search
indexing, activity filtering can be easily done on the client side.
- Python scripts are used to generate static HTML files of the app bundles
and index.json. The activity's unique HTML document is produced without style /
css, and are externally loaded, so that it can be customized / removed whenever
necessary (for example on very low bandwidth connections, or for adding another
theme)
- Sugar Appstore depends on static HTMLs also because, it can be useful for
SEOs, and to provide an external link to the activity from a third party
website. Dynamically filtered search results might not be static. Providing a
correct link to the bundle helps another user to share a specific activity
bundle with another
- All the activities on https://github.com/sugarlabs are indexed
automatically <see (9)> and built using Continuous Integration (GitHub Actions)
over here. Out of 300+ bundles, 199 and still counting have successfully built
a *.xo bundle and are continuously uploaded to CDN.
- The appstore could be downloaded for Offline use, or can be hosted on a
local server if a user would not like to use the online one. This is
specifically useful, if a school would host, the sugar appstore on its local
server, so students could download activities without accessing the internet.
The entire app store (batteries and bundles included) weighs approximately 613
MB. See this for a detailed idea.
- In case a user would like to add another bundle to the sugar appstore,
previously he/she had to have a SugarLabs shell account (AFAIK, and as far as I
understand). This might limit the accessibility to some users. This is hence
solved by a new system of adding bundles to the activity. Pull Request to Add
ApplicationsĀ is system which I adopted from the AppImage Hub
(https://github.com/appimage/appimage.github.io) where a similar app store is
built based on the same concept. Every PR is tested by continuous integration,
and then all successfully merged PRs are added to the website, and the static
website rebuild is triggered on each commit. A similar idea is implemented in
Sugarlabs Appstore. This ensures that a .xo can be built on another host also
using the same commands. Improving the quality of the bundles. For information
regarding how to add a bundle, see this. For a sample PR which intends to add a
bundle, see PR#6 which intends to add Jupyter Activity. All urls which can be
cloned using git are accepted. You may look into how the build script works
here
https://github.com/srevinsaju/sugar-activity-build/blob/master/.github/workflows/bundle_test.yml.
For those users who are not interested in GitHub, its recommended if Sugarlabs
would host a GitLab which also supports CI; As this issue has been discussed
previously, I am not putting too much insight into the same; :D
- Builds are refreshed every 5th minute of a day; (cron: "5 * * * *"). There
is no specific reason why 5th minute is chosen, mostly because I can finish
pushing changes between 0th minute and 5th minute (normally because my school
classes finish at the 0th minute; LOL; )
- Builds are hosted to netlify CDN. Netlify CDN is chosen because its free
for open source, and it also has a capability to get form submissions without a
server. This is a cheap implementation without a server. There is no other
reason why netlify chosen. Netlify has a very permissive quota. Other static
site hosting as such as github.io will make the repo huge; or will require
payment. Moreover. build on netlify takes only 1minute and 5 seconds to
complete, with older fast network speed.
- Responsive HTML design. Makes it adaptable on all screen sizes, on small
android phones to 1080p screens and much more. This is better for SEO, as well
as accessibility.
- Lighthouse speed testing shows better cumulative speed score than
activities.sugarlabs.org. On Desktop, the score for ASLOv4 is 94/100; on mobile
its 76/100 for ASLOv4. For activities.sugarlabs.org, its 89/100 on Desktop,
76/100 on mobile. Results may vary slightly from time to time; Repeat tests to
get concurrent values. Results provided by Lighthouse powered by Google. Result
details for sugarstore.netlify.app :
https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Fsugarstore.netlify.app%2F&tab=desktop;
Result details for activities.sugarlabs.org:
https://developers.google.com/speed/pagespeed/insights/?url=https%3A%2F%2Factivities.sugarlabs.org&tab=desktop
- Its possible to embed (keeping a local copy of) JQuery and Bootstrap for
offline use, but I personally prefer to keep the cdn (i.e <script>), this is
because; modern browsers automatically cache commonly used script files. So
this would increase the performance with caching. This will not require the
user to download the libs again. Moreover, if the user has visited any other
site which uses the same library, it would be actually better.
- It looks sophisticated to keep a download count on static websites, but
that does not mean its impossible. Its nevertheless, not possible (afaik) to
get download files with only static files (i.e without a server to depend on).
I could
- send a POST request to another server, which would increment the download
count on every click of the download button; along with a GET request from the
server to get the download count. This would imply, for every activity; a GET
request will be emitted, or preferably, a single GET request to get all the
download counts and update each of the activity shown for display with the new
- Use Firebase Library to create a firebase server where I could use Cloud
Firestore and send and receive download stats. However, this has a few cons:
i.e increase the page load time from 1.1s to 2.3 seconds; as well as make it
complex to protect the server from user manipulation of the download count. I
have tried this, I will enable it, if necessary
- Add a download analytics with a one way download count (send only - no
receive). By using Google Analytics, I can send custom event trigger, and
administrators can analyze and filter data / sort data by download counts. I,
however, did not find a direct method to get data from Google Analytics.
- Future Additions:
- Add Featured Activities: Activities which are most worked on: i.e
MusicBlocks, TurtleArt, Physics Activities, Fructose Set of Activities could
get a wide card along with a feature image. This will act as a showcase to the
developer and the activity; mostly based on GitHub Stargazers. I would like to
follow https://snapcraft.io as a reference from which Ubuntu Software is built.
- Add a link to Flatpak Package: this aslov4 could be the final activity
store for sugarlabs. Recently many contributions have diverged. Its important
to unify them together. Some users might prefer flatpak over a .xo file.
Activities that are packaged as Flatpaks are good because some users, follow a
modern desktop system, or might not be interested to install Sugar OS to run
sugar desktop. For those corner cases, providing Flatpak as an alternative
would not let users turn away from using Sugar Activities. Primarily Sugar
Activities are meant for improving a child's skills or understanding; which
ever mode, an activity is distributed; i.e., the most accessible package.
- Add information to other packages of sugar activity bundle. This includes
Ubuntu Packages. Fedora RPMs, and the Arch Linux Community Packages; I will
need a way to identify each of them; But at the present, I would like to leave
it as is. If any developers at sugar is interested to change some structure of
Sugar Repository, this would make things faster.
- Add a screenshot, description tag in activity.info: Screenshots of the
working activity on the appstore will probably give a clearer idea on how the
activity works. The https://help.sugarlabs.org is not frequently updated with
changes on activity's repository. If we could add another tag called
screenshots (following the convention given by Martin on
https://github.com/tchx84/sugarapp. See
https://github.com/tchx84/sugarapp/blob/master/flatpak-guide.md#porting-an-application-with-sugarapp
(point 7) on how an ideal "activity.info" could be created. This would help me
out in creating a better appstore (faster). The new "activity.info" could
include 'screenshot' and 'description' tag which is not currently used. Martin
has also mentioned of the source of the new data in the link given below.
- Create a Python3 minimal activity which does not need to load the entire
website for downloading an activity. The activity will just pull index.json,
and parse it. As the extra HTML, CSS, and JS are unnecessary for a GTK3 based
sugar-activity, it would help to save a lot of data, especially on countries
where internet connection is extremely slow (50kbps ?)
I would like to convey my thanks to Manish for providing me frequent
suggestions and always pointing out important flaws ( High CPU usage bug in
Chromium/Chrome Browser: 65a7b8f3ea2593976ef21df7a2592b030501c6d8 ,
Accessibility ), and also the base repository, i.e.,
https://github.com/free-libre-software/sugarappstore as a template and idea
provider for the sugarlabs app store to take this shape.
________________________________________________
Logo:
The logo is designed under Creative Commons License Attribution Share Alike
4.0. You may get the source code of the logo here:
source: https://github.com/sugarlabs-appstore/appstore-assets
The logo has been designed with modern conventions of app design - symmetry and
vibrance. Maybe its similar to the colorful icon design proposal for GSoC :
https://github.com/sugarlabs/GSoC/blob/master/Ideas-2020.md#colored-desktop-and-activity-icons
________________________________________________
Generator (tool used to generate static webpages):
The generator is script which automates build process of bundles and writes
static HTML files. Primarily written on Python Programming Language, can be
customized with alternative script files which are supported on Linux.
Generator, aka (internally called), SaaSBuild (Build tool for Sugarlabs
Activity App Store), is designed to be multi platform supported, although
tested on Linux only. Its possible to run it on other OSes too.
source: https://github.com/sugarlabs-appstore/sugarappstore
NOTE [IMPORTANT]: Current development goes on in PR#4
(https://github.com/sugarlabs-appstore/sugarappstore/pull/4). You might need to
checkout oop-ss if you want to test out the latest features. The master branch
contains the work by Manish before the official start of GSoC. The PR is
waiting for a review.
________________________________________________
Static HTML, CSS, JS
These are files which can be run without the need of the generated index.json.
Using JQuery, the data is dynamically read, and updated on the index.html.
Q) Why was Static parts of the website not merged with the generator
(SaaSBuild)?
Its because of yet another few reasons
- Another developer could manually add his own themes / html styleset with,
maybe another JS Framework, for example React, Angular or Vue.js later. He/She
would not have to rewrite the Generator class again for changing the frontend
part.
- Developers who are willing to contribute to HTML / Web design / Js can
only work on the "static" repository. This will help to (probably) increase
contributors. A developer who is only good at JS would not be hence be
discouraged to contribute to the current repository, if Python is also involved.
source: https://github.com/sugarlabs-appstore/sugarappstore-static
________________________________________________
Continuous Integration
Continuous Integration Downloads and sets up Sugar desktop, i.e
sugar-toolkit-gtk3, sugar, sugar-datastore, sugar-artwork and creates bundles
and packages it for publish to CDN
source: https://github.com/srevinsaju/sugar-activity-build
________________________________________________
The DEPLOYED WEBSITE:
The best part, check it out here: https://sugarstore.netlify.app
For a list of indexed apps in a portable format:
- https://sugarstore.netlify.app/index.json
-
https://github.com/srevinsaju/sugar-activity-build/releases/download/latest/index.json
-
https://github.com/srevinsaju/sugar-activity-build/releases/download/latest/sitemap.xml
For bundles app store (portable - except cdn), see
https://github.com/srevinsaju/sugar-activity-build/releases
________________________________________________
Q) How to run the appstore?
Use any static hosting server; for testing I use:
- python3 -m http.server 3000
- gunicorn
- php
- npm install -g http-server; http-server
- apache
- ngrok
- serveo
Regarding whom to approach:
We are working on two different things. Manish knows a way lot more than me
regarding HTTP Headers, Sphinx, CORS, User-Agent, and other web server related
stuff. I am working on design (css, html, js) and the generator script
(python), and those related to sugar GTK shell, sugar activities. We both
collaborate on working with the generator script.
=====================================
Hope everyone like this. All questions are welcome. Please note that the App
Store has not yet released v0.1, so its basically in alpha.Ā
PS: If anyone is interested to contribute, PRs are welcome. Please reply me on
this thread if you require access to https://github.com/sugarlabs-appstore/
organization. It was created so that Manish and I could collaborate. And also
so that, development won't freeze in the absence of one.
PS: If you have some other sweet suggestions for Sugarlabs App Store (name),
please let me know. Just as jarabe, carquinyol, and sucrose are used within
sugar, it might be interesting for kids to see their favorite sweet on the
screen. We have stuck with sugarlabs-appstore, as we did not get any other name!
Thanks if you read all the way till here!!
Stay safe! Take care!!
V/r
Srevin Saju
https://github.com/srevinsaju
_______________________________________________
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel
_______________________________________________
Sugar-devel mailing list
Sugar-devel@lists.sugarlabs.org
http://lists.sugarlabs.org/listinfo/sugar-devel
