Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

Interesting... there's another simplified packaging script here:
https://github.com/ccpgames/pypackage

On Mon, Feb 6, 2017 at 11:43 PM, René Dudfield  wrote:

> Oh. I'd also like it to support pygame zero games :)
>
> On Mon, Feb 6, 2017 at 11:41 PM, René Dudfield  wrote:
>
>> I started making a tool for doing the zero config releases. It's not done
>> yet, but it is almost able to release itself.
>>
>> https://github.com/illume/pyrelease
>>
>> (still not sure it's a good idea... but maybe)
>>
>>
>> Usage:
>> pyrelease [optional folder defaults to current working directory]
>>
>>
>> The basic steps are these.
>>
>>- Gather facts. (much like ansible if you know it)
>>- Create setup.py files in a temp folder
>>- build sdist
>>- Upload files to pypi
>>- tag a new version in git.
>>
>> The three use cases I'm aiming to make it work with.
>> -
>> singlefile.py
>> 
>> mygame/game.py
>> data/
>> -
>> singlefile.py
>> test_singlefile.py
>> -
>>
>> It should support single file modules, packages, and also /data/ folders.
>> As well it supports making a script automatically if it finds a main(). As
>> well it finds dependencies by parsing the python code (I'll add
>> requirements.txt later). So if you import pygame, click, flask etc... it
>> adds them to install_requires in the setup.py.
>>
>>- I want to add logging of facts. eg. 'Found author: "Rene" in
>>~/.gitrc". Could not find autor in ~/.hgrc
>>- Suggest additions for missing things. eg. How to create a pypi
>>account, how to make ~.pypirc, git.
>>- Have to make uploading to pypi easier, especially when things go
>>wrong.
>>- thinking of removing the setuptools dependency... only thing I use
>>it for is find_packages so far. I've been writing another more modern
>>implementation of that anyway.
>>- Pynsist support (or py2exe, pyinstaller, whatever)
>>- tests, and tests against test . pypi . python . org
>>- "add setup files into this repo" for when the tool fails to be good
>>enough.
>>- notice telling people to go to packaging. python .org if they want
>>more
>>- decide on convention for screenshots (probably screenshots/ folder)
>>- bitbucket, and better hg support.
>>- pyweek upload support.
>>- try releasing a few more things with it.
>>- watch some other people try and use it.
>>
>>
>> ​A nice thing about it was I was easily able to rename the project three
>> times. Just by renaming files and folders.
>>
>>
>> Anyway... I'll try another day on it and see how it turns out.
>>
>
>

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

Oh. I'd also like it to support pygame zero games :)

On Mon, Feb 6, 2017 at 11:41 PM, René Dudfield  wrote:

> I started making a tool for doing the zero config releases. It's not done
> yet, but it is almost able to release itself.
>
> https://github.com/illume/pyrelease
>
> (still not sure it's a good idea... but maybe)
>
>
> Usage:
> pyrelease [optional folder defaults to current working directory]
>
>
> The basic steps are these.
>
>- Gather facts. (much like ansible if you know it)
>- Create setup.py files in a temp folder
>- build sdist
>- Upload files to pypi
>- tag a new version in git.
>
> The three use cases I'm aiming to make it work with.
> -
> singlefile.py
> 
> mygame/game.py
> data/
> -
> singlefile.py
> test_singlefile.py
> -
>
> It should support single file modules, packages, and also /data/ folders.
> As well it supports making a script automatically if it finds a main(). As
> well it finds dependencies by parsing the python code (I'll add
> requirements.txt later). So if you import pygame, click, flask etc... it
> adds them to install_requires in the setup.py.
>
>- I want to add logging of facts. eg. 'Found author: "Rene" in
>~/.gitrc". Could not find autor in ~/.hgrc
>- Suggest additions for missing things. eg. How to create a pypi
>account, how to make ~.pypirc, git.
>- Have to make uploading to pypi easier, especially when things go
>wrong.
>- thinking of removing the setuptools dependency... only thing I use
>it for is find_packages so far. I've been writing another more modern
>implementation of that anyway.
>- Pynsist support (or py2exe, pyinstaller, whatever)
>- tests, and tests against test . pypi . python . org
>- "add setup files into this repo" for when the tool fails to be good
>enough.
>- notice telling people to go to packaging. python .org if they want
>more
>- decide on convention for screenshots (probably screenshots/ folder)
>- bitbucket, and better hg support.
>- pyweek upload support.
>- try releasing a few more things with it.
>- watch some other people try and use it.
>
>
> ​A nice thing about it was I was easily able to rename the project three
> times. Just by renaming files and folders.
>
>
> Anyway... I'll try another day on it and see how it turns out.
>

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

I started making a tool for doing the zero config releases. It's not done
yet, but it is almost able to release itself.

https://github.com/illume/pyrelease

(still not sure it's a good idea... but maybe)


Usage:
pyrelease [optional folder defaults to current working directory]


The basic steps are these.

   - Gather facts. (much like ansible if you know it)
   - Create setup.py files in a temp folder
   - build sdist
   - Upload files to pypi
   - tag a new version in git.

The three use cases I'm aiming to make it work with.
-
singlefile.py

mygame/game.py
data/
-
singlefile.py
test_singlefile.py
-

It should support single file modules, packages, and also /data/ folders.
As well it supports making a script automatically if it finds a main(). As
well it finds dependencies by parsing the python code (I'll add
requirements.txt later). So if you import pygame, click, flask etc... it
adds them to install_requires in the setup.py.

   - I want to add logging of facts. eg. 'Found author: "Rene" in
   ~/.gitrc". Could not find autor in ~/.hgrc
   - Suggest additions for missing things. eg. How to create a pypi
   account, how to make ~.pypirc, git.
   - Have to make uploading to pypi easier, especially when things go wrong.
   - thinking of removing the setuptools dependency... only thing I use it
   for is find_packages so far. I've been writing another more modern
   implementation of that anyway.
   - Pynsist support (or py2exe, pyinstaller, whatever)
   - tests, and tests against test . pypi . python . org
   - "add setup files into this repo" for when the tool fails to be good
   enough.
   - notice telling people to go to packaging. python .org if they want more
   - decide on convention for screenshots (probably screenshots/ folder)
   - bitbucket, and better hg support.
   - pyweek upload support.
   - try releasing a few more things with it.
   - watch some other people try and use it.


​A nice thing about it was I was easily able to rename the project three
times. Just by renaming files and folders.


Anyway... I'll try another day on it and see how it turns out.

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

Yeah, I think it's ok for quick and dirty distribution.

You're right that it fails as soon as you want other packages, a different
version of python or to polish it nicely for wider distribution (start menu
items, various icon sizes, screenshots, videos, different length
description text, etc).

The idea of allowing people to gradually make the packaging good and
polished is important.

Rather than requiring 20-30 questions, and lots of extra packaging work, a
solution should allow people to roughly package to begin with but allow a
path towards highly polished packaging.

A good name for this might be, Gradual Packaging.



On Sat, Feb 4, 2017 at 5:00 PM, Thomas Kluyver  wrote:

> On 4 February 2017 at 15:29, René Dudfield  wrote:
>
>> @deshipu: mentioned another distribution method which is quite simple.
>> http://pygame.org/project-pyg.exe-2830-.html
>
>
> Neat! I think this approach gets more fiddly as the project gains
> dependencies other than pygame, though.
>
> Pynsist aims for a somewhat more polished installer that can set up things
> like start menu entries. But the layout of files it produces could pretty
> easily be used to make an 'unzip & execute' package as well.
>
> Thomas
>

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[Thomas Kluyver]

On 4 February 2017 at 15:29, René Dudfield  wrote:

> @deshipu: mentioned another distribution method which is quite simple.
> http://pygame.org/project-pyg.exe-2830-.html


Neat! I think this approach gets more fiddly as the project gains
dependencies other than pygame, though.

Pynsist aims for a somewhat more polished installer that can set up things
like start menu entries. But the layout of files it produces could pretty
easily be used to make an 'unzip & execute' package as well.

Thomas

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

Interesting. I think it's important to consider existing projects, because
that's often the case when you think of distributing it... when it exists :)

@deshipu: mentioned another distribution method which is quite simple.
http://pygame.org/project-pyg.exe-2830-.html

It's a pyg.exe for windows which can run .py or .zip files. Love2d has
similar ones for other platforms. So to make your game distributable for
windows you can just zip it up with the pyg.exe. Pretty simple really.



On Sat, Feb 4, 2017 at 3:59 PM, Thomas Kluyver  wrote:

> On 4 February 2017 at 13:42, René Dudfield  wrote:
>
>> I was pointed to pbr by lordmauve. http://docs.openstack.org/deve
>> loper/pbr/
>>
>> I like how pbr uses setup.cfg. Which I think other tools could use, by
>> putting their config inside a setup.cfg section.
>>
>
> Yep, there are a few tools to use declarative metadata for packaging
> (d2to1 is/was another). Flit was initially an experiment in whether it's
> possible to do Python packaging without using the distutils/setuptools
> machinery. As far as I know, all other tools build on top of
> distutils/setuptools, but I believe this is the source of many headaches.
>
> There are PEPs in the works (516, 517, 518) which define a new
> pyproject.toml file for packaging metadata. Once this is a bit more
> advanced, I plan to move flit's config inside that file. I hope that it
> will become a more modern, better specified alternative for setup.cfg.
>
>
>> Perhaps I agree with you about the magic. I guess it's more a thought
>> experiment about how simple we can make it at this point. Perhaps it could
>> be made usable by explaining to the user how to specify the missing data if
>> it's not there (eg. no git... then it tells people it tried to find it in
>> git, and for people to add author etc).
>>
>
> +1 to running the thought experiment.
>
> I guess how I might use these things is in a tool like 'flit init' - if
> git is present, it could use it to inform its default guess for author name
> & email, and then it writes these to a config file rather than using them
> directly.
>
> I have some thoughts on a tool which could add or modify features of a
> project like packaging or docs - as opposed to tools like cookiecutter,
> which expects to create an entire project. I wrote about this here:
> https://github.com/takluyver/flit/pull/97#issuecomment-270984130
>
>

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[Thomas Kluyver]

On 4 February 2017 at 13:42, René Dudfield  wrote:

> I was pointed to pbr by lordmauve. http://docs.openstack.org/
> developer/pbr/
>
> I like how pbr uses setup.cfg. Which I think other tools could use, by
> putting their config inside a setup.cfg section.
>

Yep, there are a few tools to use declarative metadata for packaging (d2to1
is/was another). Flit was initially an experiment in whether it's possible
to do Python packaging without using the distutils/setuptools machinery. As
far as I know, all other tools build on top of distutils/setuptools, but I
believe this is the source of many headaches.

There are PEPs in the works (516, 517, 518) which define a new
pyproject.toml file for packaging metadata. Once this is a bit more
advanced, I plan to move flit's config inside that file. I hope that it
will become a more modern, better specified alternative for setup.cfg.


> Perhaps I agree with you about the magic. I guess it's more a thought
> experiment about how simple we can make it at this point. Perhaps it could
> be made usable by explaining to the user how to specify the missing data if
> it's not there (eg. no git... then it tells people it tried to find it in
> git, and for people to add author etc).
>

+1 to running the thought experiment.

I guess how I might use these things is in a tool like 'flit init' - if git
is present, it could use it to inform its default guess for author name &
email, and then it writes these to a config file rather than using them
directly.

I have some thoughts on a tool which could add or modify features of a
project like packaging or docs - as opposed to tools like cookiecutter,
which expects to create an entire project. I wrote about this here:
https://github.com/takluyver/flit/pull/97#issuecomment-270984130

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

Interesting. I should look into flit more... somehow I skipped that line of
your email. That's really cool!  https://pypi.python.org/pypi/flit

I was pointed to pbr by lordmauve. http://docs.openstack.org/developer/pbr/

I like how pbr uses setup.cfg. Which I think other tools could use, by
putting their config inside a setup.cfg section.


Perhaps I agree with you about the magic. I guess it's more a thought
experiment about how simple we can make it at this point. Perhaps it could
be made usable by explaining to the user how to specify the missing data if
it's not there (eg. no git... then it tells people it tried to find it in
git, and for people to add author etc).





On Sat, Feb 4, 2017 at 2:12 PM, Thomas Kluyver  wrote:

> On 4 February 2017 at 12:43, René Dudfield  wrote:
>
>> I also kind of hate 10-30 config files in repos.
>>
>> What would a python package look like with no extra files apart from our
>> code? http://renesd.blogspot.de/2017/02/what-would-python-packagin
>> g-zero-look.html Then continued the idea here:
>> http://renesd.blogspot.de/2017/02/python-packaging-zero-part-one.html
>>
>> The main idea is that you can derive enough information from the python
>> code and the git(or hg) repo.
>>
>
> I also dislike having a load of different boilerplate files in a repo for
> packaging. This was part of what prompted me to write flit. However, I
> don't think that it's practical to use absolutely zero config. From your
> proposals, I don't like getting details out of git - it requires that the
> code is in a git repo, and it makes things quite brittle.
>
> The minimum working config for flit looks like this:
>
> [metadata]
> module = astsearch
> author = Thomas Kluyver
> author-email = tho...@kluyver.me.uk
> home-page = https://github.com/takluyver/astsearch
>
> It takes the description from the docstring (as in your proposal), and the
> version number from __version__ in the module. There's a command line tool
> 'flit init' that prompts you for these values, with defaults that you can
> accept by pressing enter. It doesn't need setup.py or MANIFEST.in or
> anything: you could literally have just your module and flit.ini.
>
> I recognise that this is still some boilerplate that may seem redundant to
> new users, but I think it's ultimately clearer to have these values
> explicitly associated with the project than to pull them out of a magic hat.
>
> Thomas
>
>
>

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[Thomas Kluyver]

On 4 February 2017 at 12:43, René Dudfield  wrote:

> I also kind of hate 10-30 config files in repos.
>
> What would a python package look like with no extra files apart from our
> code? http://renesd.blogspot.de/2017/02/what-would-python-packagin
> g-zero-look.html Then continued the idea here: http://renesd.blogspot.de/
> 2017/02/python-packaging-zero-part-one.html
>
> The main idea is that you can derive enough information from the python
> code and the git(or hg) repo.
>

I also dislike having a load of different boilerplate files in a repo for
packaging. This was part of what prompted me to write flit. However, I
don't think that it's practical to use absolutely zero config. From your
proposals, I don't like getting details out of git - it requires that the
code is in a git repo, and it makes things quite brittle.

The minimum working config for flit looks like this:

[metadata]
module = astsearch
author = Thomas Kluyver
author-email = tho...@kluyver.me.uk
home-page = https://github.com/takluyver/astsearch

It takes the description from the docstring (as in your proposal), and the
version number from __version__ in the module. There's a command line tool
'flit init' that prompts you for these values, with defaults that you can
accept by pressing enter. It doesn't need setup.py or MANIFEST.in or
anything: you could literally have just your module and flit.ini.

I recognise that this is still some boilerplate that may seem redundant to
new users, but I think it's ultimately clearer to have these values
explicitly associated with the project than to pull them out of a magic hat.

Thomas

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

I spoke a bit with DR0ID on irc... he made a good point about being able to
change things around, and had a concern about generating files with
cookiecutter. It's kind of harder to change things after you have first
generated things.

I also kind of hate 10-30 config files in repos.

What would a python package look like with no extra files apart from our
code? http://renesd.blogspot.de/2017/02/what-would-python-
packaging-zero-look.html Then continued the idea here:
http://renesd.blogspot.de/2017/02/python-packaging-zero-part-one.html

The main idea is that you can derive enough information from the python
code and the git(or hg) repo.

single file

> mygame.py
>

single file, data, and a test

> mygame.py
>
data/bla.png
>
test_mygame.py
>

a package

> mygame/main.py
> mygame/magic.py
> data/bla.png
>


Then you just run:

> packagezero release
>



(More details in the posts.)

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

tldr;

How about we recommend this API:
.image(file_path) -> pygame.Surface
.image('images/bla.png'), note no os.path.join needed.
.sound(file_path) -> pygame.Sound
.font(file_path) -> pygame.Font
.music(file_path) -> calls music.load() on the file.

We have these available, but don't recommend:
.get_file(file_path) -> open file
.find_data_dir() -> folder for 'data'.
.find_file(file_path) -> finds the full path to the file.

Something for later:
.preload(images, sounds, fonts, music) preloads files perhaps over network
or perhaps multi core, or just into a cache.

I think this API is simple, and it's possible to do a fairly good
implementation and a very quick easy implementation in time for pyweek.

Perhaps we can have a namespace package pygame.resources, released on pypi
as pygame_resources. Or maybe include it in a 'skellington' package? If so,
what should that package be called?




Ok, I am happy to avoid pkg_resources. Does it give us anything at all?

Here's the pyglet approach to resources loading. Note there's also a config
interface, for user game options. Something that everyone eventually
writes. I think these are all good, but out of scope for now.
http://pyglet.readthedocs.io/en/pyglet-1.2-maintenance/api/
pyglet/pyglet.resource.html

Note: for network connections I think gamejs (a JS lib that is fairly close
to the pygame API) shows a fairly nice way to do it. You can have a
preload('bla.png', 'yes.png') call to tell it to immediately go off and
load images. (This API could also let us easily implement a)
https://github.com/GameJs/gamejs/blob/master/examples/collisionmask/main.js

It's common for people to have their resources libraries make
resources.load('bla.png') return the same Surface, by storing a weak
reference to it in a cache. An advanced implementation could do this I
think.

I don't have benchmark numbers for you, but it definitely is faster to load
one file than 100. Way less seeks (on media like SD cards, and magnetic
disks this is more important). It's also faster to do one read, rather than
"random reads" (there's lots of IO benchmarks online proving this is true).
On high end modern SSDs with the latest kernels and best file systems you
can get multi core disk IO when the moon is aligned right. Many games use
'pack files' for speed, instead of the file system. Note, this is still
true even in web browsers - having a big image (sprite sheet) is faster,
and uses less memory than having many separate images.

For optimal loading from disk, having one big file with the separate files
start and end positions is a good idea. Then you can load each one with
mmap (so the image is only in memory once... the file cache), and then use
separate python processes to load each image in with mmap (pygame can load
images via mmap). Unfortunately some image routines are not multithread
safe, so the GIL is held sometimes... meaning processes are faster to load
pygame images on python. On some graphics card/OSes you can even mmap the
raw files directly from the file cache into GPU texture (on some apple
hardware you can do this). Another trick for when you have a cold file
cache (like the first time you load the file) is to first start a 'cat'
process sending the pack file to /dev/null, this uses an optimised kernel
path to put the data into the file cache.

Anyway... that was a bit of a digression.







On Thu, Feb 2, 2017 at 2:01 PM, Thomas Kluyver  wrote:

>
> On 2 February 2017 at 06:34, René Dudfield  wrote:
>
>> Whilst naming is important, the name of the package doesn't need to be
>> the name of the Game. I've worked on projects where the company name, and
>> app name changed at least three times whilst the package name stayed the
>> same. So I don't think people need to worry so much. Also, it's not too
>> hard to change the package name later on.
>>
>>
>> My vote would be to follow the python sampleproject way of doing things.
>>
>
> That gets my vote as well. If you're putting it on PyPI, sooner or later
> you have to give the package a good name (at least, not the same name as
> every other game's package). We can encourage people to use relative
> imports (from . import foo) which makes it less work to rename.
>
>
>
>> Data folder, and "get_data".
>>
>> 2). The other aspect I'm not sure of is having a data folder outside of
>> the source folder. Having it inside the source folder means you can more
>> easily include it inside a zip file for example. It also makes packaging
>> slightly easier. Since you can just use from within a MANIFEST.in a
>> recursive include of the whole "mygamepackage" folder.
>>
>> Having data/ separate is the choice of the sampleproject, and the
>> skellington.
>>
>> I haven't really seen a modern justification for keeping data out of the
>> package folder?
>>
>
> My vote would be for it to go inside the package, and for the skeleton to
> provide a bit of code like this:
>
> DATA_DIR = 

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[Thomas Kluyver]

On 2 February 2017 at 06:34, René Dudfield  wrote:

> Whilst naming is important, the name of the package doesn't need to be the
> name of the Game. I've worked on projects where the company name, and app
> name changed at least three times whilst the package name stayed the same.
> So I don't think people need to worry so much. Also, it's not too hard to
> change the package name later on.
>
>
> My vote would be to follow the python sampleproject way of doing things.
>

That gets my vote as well. If you're putting it on PyPI, sooner or later
you have to give the package a good name (at least, not the same name as
every other game's package). We can encourage people to use relative
imports (from . import foo) which makes it less work to rename.



> Data folder, and "get_data".
>
> 2). The other aspect I'm not sure of is having a data folder outside of
> the source folder. Having it inside the source folder means you can more
> easily include it inside a zip file for example. It also makes packaging
> slightly easier. Since you can just use from within a MANIFEST.in a
> recursive include of the whole "mygamepackage" folder.
>
> Having data/ separate is the choice of the sampleproject, and the
> skellington.
>
> I haven't really seen a modern justification for keeping data out of the
> package folder?
>

My vote would be for it to go inside the package, and for the skeleton to
provide a bit of code like this:

DATA_DIR = os.path.join(dirname(abspath(__file__)), 'data')
def find_data(*path):
return os.path.join(DATA_DIR, *path)

This is roughly what I recommend in Pynsist's FAQ.


> I have vague recollections of reasons being: 'because debian does it'. My
> recollection is that Debian traditionally did it to keep code updates
> smaller. Because if you only change 1KB of source code, there's no point
> having a 20MB update every time.
>

Linux packagers like to put data files in /usr/share to comply with the
filesystem hierarchy spec. However, with the snippet of code I gave above,
it's easy for them to patch DATA_DIR = '/usr/share' and move the files out.


> A bonus of keeping data separate is that it forces you to use relative
> addressing of file locations. You don't hardcode "data/myfile.png" in all
> your paths. Do we recommend the python way of finding the data folder? This
> is package_data, and data_files setup attributes. https://github.com/pypa/
> sampleproject/blob/master/setup.py
>

Actually, I think that's more of a risk if it's a separate top-level
directory, because then 'data' is going to be in the CWD when you run it as
a developer.


> They (package_data) are a giant pain. One, because they require mentioning
> every single file, rather than just the whole folder. Two, because they
> require you updating configuration in both MANIFEST.in and setup.py. Also,
> there are different files included depending on which python packaging
> option you use.
>

Agreed that they are a giant pain. They don't quite require mentioning
every single file, as you can glob directories, but you do need to mention
every subdirectory, at least for package_data.

This is one of the things that prompted me to write a packaging tool called
'flit', which doesn't require this. I don't know that it's quite ready to
be pushed on people in a game skeleton, though.


> Another issue is that, using the python way pkg_resources from setuptools
> needs to be used at runtime. pkg_resources gets you access to the resources
> in an abstract way. Which means you need setuptools at runtime (not just at
> install time). There is already work going into keeping that separate here:
> https://github.com/pypa/pkg_resources So I'm not sure this will be a
> problem in the next months.
>

Ugh, I avoid pkg_resources at all costs. If you use package_data (as
opposed to data_files), it's not necessary; see my snippet above.


> I haven't confirmed if pkg_resources works with the various .exe making
> tools. I've always just used file paths. Thomas, does it work with Pynsist?
>

I haven't tried, but I'd guess it may well not work.


> Having game resources inside a .zip (or .egg) makes loading a lot faster
> on many machines. pkg_resources supports files inside of .egg files. So, I
> think we should consider this possibility.
>

I've heard this claim made before, but I haven't seen numbers. Having
resources inside a zip file is awkward if you need to pass a file path or a
file handle to other libraries: you have to extract it and write it to a
temporary file before you can pass it in. If the performance difference is
important, I'd favour writing some helper routines using the zipfile module
rather than doing anything with pkg_resources.

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

@Daniel, Yeah, I think a GUI for apps using virtualenv, and pip would be
quite possible and good. (I wonder if such a thing already exists?)

I know when setuptools and easy_install were first being made, that was the
plan. There was a gui in their app which used them to install plugins for
the app. Also, I know some IDEs provide tools for installing things with a
GUI (conda, pycharm etc).

cu.

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

Excellent! Thanks Thomas :)

I'll wait to hear feedback from DR0ID about the repo. In the mean time,
below is some more discussion on a couple of topics I'd like feedback on?

(sorry that it is such a wall of text)



src, gamelib, and mygamepackage

1). One thing that is different in the skellington layout from the
sampleproject one is that the naming is a bit more specific for where the
code goes in skellington.

https://bitbucket.org/dr0id/pyknic/src/92449d2874e031da948fd7c537df0bb7106b2676/pyknic-skellington/?at=default
gamelib/
data/

Why this is good? Because you can start writing your code without first
having to decide the name. It's a small thing, but in the context of game
competitions it's more important. I'm not sure if it's really worth keeping
that idea though.

This would be the sampleproject layout after doing "skellington create
mygamepackage"
mygamepackage/
data/

(Where skellington is the name of our tool. It could be pygame create... or
whatever)


The benefits of this are that you can go into the repo and do:
import mygamepackage

Any it works. Because mygamepackage is just a normal package.

You can also do:
python mygamepackage/run.py


Whilst naming is important, the name of the package doesn't need to be the
name of the Game. I've worked on projects where the company name, and app
name changed at least three times whilst the package name stayed the same.
So I don't think people need to worry so much. Also, it's not too hard to
change the package name later on.


My vote would be to follow the python sampleproject way of doing things.



Data folder, and "get_data".

2). The other aspect I'm not sure of is having a data folder outside of the
source folder. Having it inside the source folder means you can more easily
include it inside a zip file for example. It also makes packaging slightly
easier. Since you can just use from within a MANIFEST.in a recursive
include of the whole "mygamepackage" folder.

Having data/ separate is the choice of the sampleproject, and the
skellington.

I haven't really seen a modern justification for keeping data out of the
package folder? I have vague recollections of reasons being: 'because
debian does it'. My recollection is that Debian traditionally did it to
keep code updates smaller. Because if you only change 1KB of source code,
there's no point having a 20MB update every time.

A bonus of keeping data separate is that it forces you to use relative
addressing of file locations. You don't hardcode "data/myfile.png" in all
your paths. Do we recommend the python way of finding the data folder? This
is package_data, and data_files setup attributes.
https://github.com/pypa/sampleproject/blob/master/setup.py

They are a giant pain. One, because they require mentioning every single
file, rather than just the whole folder. Two, because they require you
updating configuration in both MANIFEST.in and setup.py. Also, there are
different files included depending on which python packaging option you use.

See the packaging.python.org documentation on including data files:
https://packaging.python.org/distributing/?highlight=data#data-files

Another issue is that, using the python way pkg_resources from setuptools
needs to be used at runtime. pkg_resources gets you access to the resources
in an abstract way. Which means you need setuptools at runtime (not just at
install time). There is already work going into keeping that separate here:
https://github.com/pypa/pkg_resources So I'm not sure this will be a
problem in the next months.

I haven't confirmed if pkg_resources works with the various .exe making
tools. I've always just used file paths. Thomas, does it work with Pynsist?

Having game resources inside a .zip (or .egg) makes loading a lot faster on
many machines. pkg_resources supports files inside of .egg files. So, I
think we should consider this possibility.

A single file .exe on windows used to be possible with pygame including all
of the data. It worked by adding a .zip file to the end of the .exe and
then decompressing that before running it. It actually made startup time
slower, but the benefit was distribution was pretty easy. However, putting
everything in a .zip file was just as good.

Perhaps we could work on adding a find_data_files() type function for
setuptools, which would recursively add data files from data/. We could
include it in our 'skellington' package until such a thing is more widely
installed by setuptools.

Despite all the issues of having a separate data/ folder, it is the
convention so far. So my vote is to follow that convention and try fixing
the issues in setuptools/pkg_resources.



Too many files, too complex for newbies.

3) Modern python packages have 20-30 files in the root folder. I have heard
the complaint many times that it makes it difficult to figure out where to
put things. It makes it complex. This is the strong feedback I got in one
pyweek where lots of people decided to use the older skellington instead.

We 

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[Thomas Kluyver]

On 1 February 2017 at 06:31, René Dudfield  wrote:

> But now with free CI options... it seems more possible to make a tool
> which builds peoples apps for them. But again would require maintenance. By
> leaning on the python packaging infrastructure, we access to all the tools
> for packaging libraries.


I agree that leveraging the library packaging ecosystem to make app
packaging easier is a good idea. Part of why I pushed hard for pygame to
have wheels on PyPI (and a release ;-), is because that makes it very easy
to build a Windows installer with Pynsist. I'd be happy to help set up a
skeleton/example repo which uses Pynsist and a CI service like Travis to
build installers.

Eventually, I'd like it to be the case that game creators don't need to
build wheels or put their game on PyPI to distribute it. As you suggest, it
should be enough to upload the files to a website, or run something
locally, to build installers/packages for different platforms. But that
vision is clearly some way off, and I accept that PyPI is a decent interim
solution. Doing 'pip install bullet_dodger' (thanks Jorge for the example)
certainly beats unpacking a tarball and finding out about dependencies by
trial and error.

Thomas

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[Jorge]

Am 01.02.2017 um 07:31 schrieb René Dudfield:
> I agree that many of the things on your wonderful list would be
> useful. I've also starting adding them to a plan (see at the bottom).
>
> I still think promoting packaging is still very useful, and a very low
> effort thing to do.
>
> Game distribution for general users should definitely not happen on
> the cheeseshop. Especially not as the primary method. The audience I'm
> thinking of is more other game developers (and people who will
> eventually become developers).
>
> I think the package index is better than 'random free upload webpage
> on the internet', which many are uploading code to now. Also pyweek
> has proven that a code template can provide a helpful structure for
> people using other packaging tools. Often times, eventually, someone
> figures out the latest work arounds for the various packaging tools
> and a script appears which works for many platforms. Of course every
> year platforms update, and the packaging tools develop new
> features...*cough* bugs *cough* that means that last years script has
> stopped working.
>
> But now with free CI options... it seems more possible to make a tool
> which builds peoples apps for them. But again would require
> maintenance. By leaning on the python packaging infrastructure, we
> access to all the tools for packaging libraries.
>
> The pygame website, and things like pyweek have thousands of games on
> there already. There's also thousands of people who look through those
> games every month. I'm fully intending to improve apon the features on
> the website for people releasing games.
>
> We need to try and make it as simple as dragging a game folder onto
> website. Because that's basically what people are doing to upload
> their games. Some people don't even know git, github, pypi, travis...
> all those things. Hopefully we can take much of the tediousness out of it.
>
>
> I'm hoping to collaborate with DR0ID, who is working on the pyweek
> 'skellington' base code on a file structure we can use by combining it
> with the pypa 'sampleproject'.
>
> I've started writing a series of blog posts about all this stuff...
> about the benefits of packaging games for the python community, what
> we can do to make distributing games easier.
> http://renesd.blogspot.com/2017/01/promoting-pypi-for-python-game-releases.html
> http://renesd.blogspot.com/2017/01/using-common-file-layout-lets-us-create.html
>
>
> Looking forward to the day when we have build bots package games up
> for android, mac, windows, ubuntu, pip, raspberrypi etc, etc, then
> have things automatically do release announcements and such.
>
>
> cheers!
>
>
>
>
> On Mon, Dec 26, 2016 at 1:17 PM, Thomas Kluyver  > wrote:
>
> My tldr: PyPI and pip are the wrong tools for game distribution,
> there are better places to focus effort.
>
> If the instructions to get your game say 'pip install yourgame',
> you're limiting your audience to people who have Python installed
> and are comfortable with the command line. Even among those
> people, you may find yourself having to explain about using pip3
> on some systems, or about why running 'sudo pip ...' is a bad idea.
>
> PyPI and pip exist primarily to distribute Python libraries. We
> use them secondarily to distribute command-line tools, because
> it's a quick and easy alternative to building packages for
> different platforms, and the kind of people who use a tool like
> 'nosetests' know how to install it with pip. They're not a good
> fit for GUI applications where the user shouldn't need to know
> that Python is in use.
>
> So, where do I think we should focus effort?
>
>   * Tools to package up Python /applications/ into convenient
> installable bundles
>   o Shameless plug: Pynsist is a tool I made to build Windows
> installers.
>   o I'd particularly like to see work around the new Linux
> application packaging formats, Flatpak and Snappy. Can we
> make a tool that takes some form of description and builds
> both kinds of package?
>   o The BeeWare projec (http://pybee.org/ ) is doing some
> interesting work on packaging for mobile platforms.
>   o Stretch goal: can we start with a single application
> description and build packages for various platforms? I'm
> sceptical, but it would be cool, even if the packages
> lacked some polish.
>   * Guides on preparing & submitting games to various app
> marketplaces:
>   o Platform owners: Microsoft, Apple, Google, Canonical...
>   o Third party: Steam, Itch...
>   * (Maybe) A better catalogue of non-professional games, for
> creators who may not want to put their games up on Steam or
> whatever. I'm still unsure if there's an actual gap to be
> filled here, 

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[René Dudfield]

I agree that many of the things on your wonderful list would be useful.
I've also starting adding them to a plan (see at the bottom).

I still think promoting packaging is still very useful, and a very low
effort thing to do.

Game distribution for general users should definitely not happen on the
cheeseshop. Especially not as the primary method. The audience I'm thinking
of is more other game developers (and people who will eventually become
developers).

I think the package index is better than 'random free upload webpage on the
internet', which many are uploading code to now. Also pyweek has proven
that a code template can provide a helpful structure for people using other
packaging tools. Often times, eventually, someone figures out the latest
work arounds for the various packaging tools and a script appears which
works for many platforms. Of course every year platforms update, and the
packaging tools develop new features...*cough* bugs *cough* that means that
last years script has stopped working.

But now with free CI options... it seems more possible to make a tool which
builds peoples apps for them. But again would require maintenance. By
leaning on the python packaging infrastructure, we access to all the tools
for packaging libraries.

The pygame website, and things like pyweek have thousands of games on there
already. There's also thousands of people who look through those games
every month. I'm fully intending to improve apon the features on the
website for people releasing games.

We need to try and make it as simple as dragging a game folder onto
website. Because that's basically what people are doing to upload their
games. Some people don't even know git, github, pypi, travis... all those
things. Hopefully we can take much of the tediousness out of it.


I'm hoping to collaborate with DR0ID, who is working on the pyweek
'skellington' base code on a file structure we can use by combining it with
the pypa 'sampleproject'.

I've started writing a series of blog posts about all this stuff... about
the benefits of packaging games for the python community, what we can do to
make distributing games easier.
http://renesd.blogspot.com/2017/01/promoting-pypi-for-python-game-releases.html
http://renesd.blogspot.com/2017/01/using-common-file-layout-lets-us-create.html


Looking forward to the day when we have build bots package games up for
android, mac, windows, ubuntu, pip, raspberrypi etc, etc, then have things
automatically do release announcements and such.


cheers!




On Mon, Dec 26, 2016 at 1:17 PM, Thomas Kluyver  wrote:

> My tldr: PyPI and pip are the wrong tools for game distribution, there are
> better places to focus effort.
>
> If the instructions to get your game say 'pip install yourgame', you're
> limiting your audience to people who have Python installed and are
> comfortable with the command line. Even among those people, you may find
> yourself having to explain about using pip3 on some systems, or about why
> running 'sudo pip ...' is a bad idea.
>
> PyPI and pip exist primarily to distribute Python libraries. We use them
> secondarily to distribute command-line tools, because it's a quick and easy
> alternative to building packages for different platforms, and the kind of
> people who use a tool like 'nosetests' know how to install it with pip.
> They're not a good fit for GUI applications where the user shouldn't need
> to know that Python is in use.
>
> So, where do I think we should focus effort?
>
>
>- Tools to package up Python *applications* into convenient
>installable bundles
>   - Shameless plug: Pynsist is a tool I made to build Windows
>   installers.
>   - I'd particularly like to see work around the new Linux
>   application packaging formats, Flatpak and Snappy. Can we make a tool 
> that
>   takes some form of description and builds both kinds of package?
>   - The BeeWare projec (http://pybee.org/ ) is doing some interesting
>   work on packaging for mobile platforms.
>   - Stretch goal: can we start with a single application description
>   and build packages for various platforms? I'm sceptical, but it would be
>   cool, even if the packages lacked some polish.
>   - Guides on preparing & submitting games to various app
>marketplaces:
>   - Platform owners: Microsoft, Apple, Google, Canonical...
>   - Third party: Steam, Itch...
>- (Maybe) A better catalogue of non-professional games, for creators
>who may not want to put their games up on Steam or whatever. I'm still
>unsure if there's an actual gap to be filled here, though, and what shape
>it is if it exists.
>
>
> Thomas
>
> On 25 December 2016 at 00:52, René Dudfield  wrote:
>
>> Hello,
>>
>> tldr; promote using pypi and pip for game releases?
>>
>>
>> With all the great work from lots of people pygame is often easily
>> installable via pip - the standard python packaging system. We still 

Re: [pygame] Promoting cheeseshop/pypi for game releases?

[Thomas Kluyver]

My tldr: PyPI and pip are the wrong tools for game distribution, there are
better places to focus effort.

If the instructions to get your game say 'pip install yourgame', you're
limiting your audience to people who have Python installed and are
comfortable with the command line. Even among those people, you may find
yourself having to explain about using pip3 on some systems, or about why
running 'sudo pip ...' is a bad idea.

PyPI and pip exist primarily to distribute Python libraries. We use them
secondarily to distribute command-line tools, because it's a quick and easy
alternative to building packages for different platforms, and the kind of
people who use a tool like 'nosetests' know how to install it with pip.
They're not a good fit for GUI applications where the user shouldn't need
to know that Python is in use.

So, where do I think we should focus effort?


   - Tools to package up Python *applications* into convenient installable
   bundles
  - Shameless plug: Pynsist is a tool I made to build Windows
  installers.
  - I'd particularly like to see work around the new Linux application
  packaging formats, Flatpak and Snappy. Can we make a tool that takes some
  form of description and builds both kinds of package?
  - The BeeWare projec (http://pybee.org/ ) is doing some interesting
  work on packaging for mobile platforms.
  - Stretch goal: can we start with a single application description
  and build packages for various platforms? I'm sceptical, but it would be
  cool, even if the packages lacked some polish.
  - Guides on preparing & submitting games to various app marketplaces:
  - Platform owners: Microsoft, Apple, Google, Canonical...
  - Third party: Steam, Itch...
   - (Maybe) A better catalogue of non-professional games, for creators who
   may not want to put their games up on Steam or whatever. I'm still unsure
   if there's an actual gap to be filled here, though, and what shape it is if
   it exists.


Thomas

On 25 December 2016 at 00:52, René Dudfield  wrote:

> Hello,
>
> tldr; promote using pypi and pip for game releases?
>
>
> With all the great work from lots of people pygame is often easily
> installable via pip - the standard python packaging system. We still have
> some issues, but it works quite well on major platforms.
>
> Now our games can be installed with pip too!
>
> *pip install yourgame*
>
> Since many people enter the python world via games, it makes sense that
> they get used to publishing python packages as well. I've sat in python
> groups, and still 75% of the room has never published a python package
> despite many of them working with python every day.
>
> As a game developer why should I use pip? Firstly, there is a very large
> audience of people who can install python games. You don't need to worry
> about the platform issues of binaries so much. If they have pip on their
> platform, then they can install your game. Other benefits of publishing on
> pypi include syndication, since many people tweet and copy all the releases
> on pypi. Another benefit is all the infrastructure work that goes into
> pypi, CDN networks and such.
>
> I suggest efforts should be applied to:
>
>- updating tutorials, and spreading the idea of publishing python
>games to the cheeseshop (pyweek, pygame.org tutorials, external
>tutorials, books, youtube videos)
>- base code for a pygame game in a standard structure (skellington,
>cookiecutter etc)
>- contacting other python game communities to suggest pypi should be a
>priority
>- making the cheeseshop/pypi itself a better platform for game
>publishing needs
>
> What pypi doesn't do currently? It doesn't do many things that a good game
> release system would do. Video/youtube links, and even screenshots aren't
> available. Discussion has been disabled (they found it way too hard to
> moderate). Even ratings are not on there (which can help for
> discover-ability). Another issue is that closed source things aren't really
> looked apon nicely there(but it is allowed). Finally, packaging in python
> still isn't the easiest thing (it's definitely not as easily as uploading a
> zip file, but it is waay nicer now than ever before).
>
> Any work that goes into making the packaging system for python better for
> games helps out with other python game communities as well. We can perhaps
> even gain allies from the other communities to help improve things for
> games in general.
>
> Here are where the pypi projects live.
>
>- https://github.com/pypa
>- (current pypi) - https://github.com/pypa/pypi-legacy
>- (next gen pypi) - https://github.com/pypa/warehouse
>
>
>