Hello, Community
Big parts of Dominik's front-end work around introducing Bootstrap has
landed. Thanks!
The next step require a bit of an architectural decision. I would like
to share it here to get feedback/awareness from the rest of the community.
It is about how we combine the use of the npm package manager and
packages with Kallithea primarily being distributed on pypi as GPL
application. It might be an annoying problem, but we just have to find a
solution to it.
I have a proposal that describe the problem and some solutions ... and
implement one that seems to work. It is available on
https://kallithea-scm.org/repos/kallithea-incoming/changeset/1b9cb678344f80221f14bdb9b779d5a4823d829a
for testing and experimenting, but I will also paste it here.
Does it work for you, or what do you suggest?
/Mads
# HG changeset patch
# User Mads Kiilerich <[email protected]>
# Date 1510108450 -3600
# Wed Nov 08 03:34:10 2017 +0100
# Node ID 1b9cb678344f80221f14bdb9b779d5a4823d829a
# Parent 6bef1d7bafa6b1743ad10f644d72139e10b95b76
setup: install and run npm commands when installing Kallithea with pip
Kallithea is under the GPL license, and we can thus only distribute any
generated code if we also ship the corresponding source.
We are moving towards a web front-end that use npm to download and compile
various open source components. The components might not be GPL, but if we
distribute any parts of their code (compiled or converted to other
representation), then we also must distribute the corresponding source under
the GPL.
It doesn't seem feasible for us to distribute the source of everything
that npm
downloads and includes when we are building. It thus also doesn't seem
feasible
for us to build and ship the compiled (possibly minified) front-end code.
Instead, we have to make it as smooth as possible for our users to get
up and
running.
It doesn't seem feasible for us to ship or install npm. We must assume it is
available. That requirement must be documented clearly, and we must
recommend
how to install npm for the most common platforms.
We could perhaps just document what manual steps to run. Kallithea
doesn't work
out of the box anyway - it has to be configured and initialized. Extra steps
might not be a big problem. We could perhaps also have a custom gearbox
command
to run the commands. But running it on demand when launching the editor
would
probably scare most system administrators ... and would also create
different
problems with the different ways of running Kallithea.
But it would be nice if we could call out to npm while pip is installing
Kallithea and download the requirements and build the files. It can be
done by
customizing setuptools commands in setup.py . That is what is done here.
Python packaging is fragile. Even though we only support pip, it really
isn't
built for things like this. Custom output is muted and buffered and only
shown
if running with -v or the command fails. And pip and setup.py can be used to
build and install in so many ways that we probably can't make it work
reliably
with all ways of installing Kallithea. But we can try.
The npm invocation can be disabled by setting the environment variable
KALLITHEA_INSTALL_RUN_NPM=NO .
The custom 'install' method is used by a plain 'pip install' form pypi
or tar,
while 'develop' is used when running 'pip install -e' from source. While
developing, it might also be convenient to just run the 'less' command
directly, or to use 'nodemon' to run it automatically when the source files
change.
For now, this will just create/update style.css ... but currently probably
without any actual changes. The files created by npm (and the node_modules
directory) must *not* be a part of the release package made with 'setup.py
sdist'.
diff --git a/docs/overview.rst b/docs/overview.rst
--- a/docs/overview.rst
+++ b/docs/overview.rst
@@ -69,6 +69,10 @@ installed.
(``pip install kallithea`` from a source tree will do pretty much
the same
but build the Kallithea package itself locally instead of
downloading it.)
+Kallithea also include some front-end code that need npm_ to download
packages
+and build the front-end. The ``npm`` binary must thus be available, and
it will
+be invoked automatically and invisibly when installing with ``pip``.
+
Web server
----------
@@ -138,3 +142,4 @@ continuous hammering from the internet.
.. _WSGI: http://en.wikipedia.org/wiki/Web_Server_Gateway_Interface
.. _HAProxy: http://www.haproxy.org/
.. _Varnish: https://www.varnish-cache.org/
+.. _npm: https://www.npmjs.com/
diff --git a/setup.py b/setup.py
--- a/setup.py
+++ b/setup.py
@@ -3,6 +3,8 @@
import os
import sys
import platform
+import subprocess
+
if sys.version_info < (2, 6) or sys.version_info >= (3,):
raise Exception('Kallithea requires python 2.6 or 2.7')
@@ -122,6 +124,31 @@ class sdist_new(sdist_org):
self.owner = self.group = 'root'
sdist.sdist = sdist_new
+
+from setuptools.command.develop import develop
+from setuptools.command.install import install
+
+class DevelopNpm(develop):
+ def run(self):
+ develop.run(self)
+ install_npm()
+
+class InstallNpm(install):
+ def run(self):
+ install.run(self)
+ install_npm()
+
+def install_npm():
+ if os.environ.get('KALLITHEA_INSTALL_RUN_NPM') != 'NO':
+ import kallithea # has just been installed
+ rootdir =
os.path.dirname(os.path.dirname(os.path.abspath(kallithea.__file__)))
+ srcdir = os.path.join(rootdir, 'kallithea', 'public', 'less')
+ sys.stderr.write("\nRunning 'npm install' in %s to install
packages from package.json\n" % srcdir)
+ subprocess.check_call(['npm', 'install'], cwd=srcdir)
+ sys.stderr.write("\nRunning 'npm run less' in %s to build css
from package.json\n" % srcdir)
+ subprocess.check_call(['npm', 'run', 'less'], cwd=srcdir)
+
+
packages = setuptools.find_packages(exclude=['ez_setup'])
setuptools.setup(
@@ -140,6 +167,9 @@ setuptools.setup(
data_files=data_files,
packages=packages,
include_package_data=True,
+ cmdclass={
+ 'install': InstallNpm,
+ 'develop': DevelopNpm},
message_extractors={'kallithea': [
('**.py', 'python', None),
('templates/**.mako', 'mako', {'input_encoding': 'utf-8'}),
_______________________________________________
kallithea-general mailing list
[email protected]
https://lists.sfconservancy.org/mailman/listinfo/kallithea-general
