[gentoo-commits] proj/g-sorcery:dev commit in: scripts/
commit: dd4f5f2118ba536f6b23079c38870fcde5d00350 Author: Jauhien Piatlicki gentoo org> AuthorDate: Sat Jan 16 22:49:24 2016 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Sat Jan 16 22:49:24 2016 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=dd4f5f21 [scripts] run tests for Python 3.5 scripts/all_pythons.sh | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/scripts/all_pythons.sh b/scripts/all_pythons.sh index af4c1f1..fe1d74f 100755 --- a/scripts/all_pythons.sh +++ b/scripts/all_pythons.sh @@ -2,7 +2,7 @@ DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd )" -for VER in 2.7 3.3 3.4 +for VER in 2.7 3.3 3.4 3.5 do echo echo "testing python${VER}"
[gentoo-commits] proj/g-sorcery:dev commit in: /
commit: 703318df48fc70bfb232033753fe02b73dfd67c9 Author: Jauhien Piatlicki gentoo org> AuthorDate: Sun Sep 13 19:36:14 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Sun Sep 13 19:36:14 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=703318df [README] add link to trello board README.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/README.md b/README.md index 9bdd5b7..c2eb5ad 100644 --- a/README.md +++ b/README.md @@ -26,3 +26,5 @@ This project is aimed to create a framework for ebuild-generators for 3rd party software providers. If you want to develop a new backend see [developer's instructions](https://github.com/jauhien/g-sorcery/blob/master/docs/developer_instructions.rst). + +[TODO list](https://trello.com/b/8WdY2ZIs/framework-for-automated-ebuild-generators).
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/
commit: e71fd837faf51fc0e9a123477bfc7558c24b37c0 Author: Jauhien Piatlicki gentoo org> AuthorDate: Sun Aug 16 22:43:08 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Sun Aug 16 22:43:08 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=e71fd837 [g_sorcery/fileutils] set wget timeout to 2 seconds g_sorcery/fileutils.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/g_sorcery/fileutils.py b/g_sorcery/fileutils.py index d783c8a..235eaec 100644 --- a/g_sorcery/fileutils.py +++ b/g_sorcery/fileutils.py @@ -149,9 +149,9 @@ def wget(uri, directory, output=""): """ if output: ret = os.system('wget ' + uri + -' -O ' + os.path.join(directory, output)) +' -O ' + os.path.join(directory, output) + ' -T 2') else: -ret = os.system('wget -P ' + directory + ' ' + uri) +ret = os.system('wget -P ' + directory + ' ' + uri + ' -T 2') return ret def get_pkgpath(root = None):
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/, tests/, /, g_sorcery/git_syncer/, docs/
commit: 29875e07c1749ddf895d2058046c17092d72732e Author: Jauhien Piatlicki gentoo org> AuthorDate: Tue Aug 4 20:54:21 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Wed Aug 5 18:32:11 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=29875e07 [g_sorcery/package_db] new DB syncing docs/developer_instructions.html | 3 +- docs/developer_instructions.rst| 3 +- docs/g-sorcery.8 | 2 +- docs/g-sorcery.8.rst | 2 +- docs/g-sorcery.cfg.8 | 2 +- docs/g-sorcery.cfg.8.rst | 2 +- g_sorcery/backend.py | 47 +--- g_sorcery/git_syncer/__init__.py | 1 + g_sorcery/git_syncer/git_syncer.py | 67 +++ g_sorcery/package_db.py| 60 +++-- g_sorcery/syncer.py| 107 + setup.py | 4 +- tests/test_PackageDB.py| 9 +--- 13 files changed, 247 insertions(+), 62 deletions(-) diff --git a/docs/developer_instructions.html b/docs/developer_instructions.html index 4cea319..4af5e35 100644 --- a/docs/developer_instructions.html +++ b/docs/developer_instructions.html @@ -581,7 +581,8 @@ backends). PackageDB class API should be used instead. to add categories and packages and to do queries on them. Usually you do not want to customize this class. If you have a database that should be synced with another already generate database -you can redifine URI to be used for syncing using get_real_db_uri method. +you can use sync method. Two sync methods are available +currently: tgz and git. Note that before add any package you should add a category for it using add_category. Then packages can be added using add_package. PackageDB currently does not write changes automatically, so you should call write after changes are done. This is not relevant diff --git a/docs/developer_instructions.rst b/docs/developer_instructions.rst index f0667ea..f868d2f 100644 --- a/docs/developer_instructions.rst +++ b/docs/developer_instructions.rst @@ -242,7 +242,8 @@ to add categories and packages and to do queries on them. Usually you do not wan class. If you have a database that should be synced with another already generate database -you can redifine URI to be used for syncing using **get_real_db_uri** method. +you can use **sync** method. Two sync methods are available +currently: **tgz** and **git**. Note that before add any package you should add a category for it using **add_category**. Then packages can be added using **add_package**. PackageDB currently does not write changes diff --git a/docs/g-sorcery.8 b/docs/g-sorcery.8 index 6782c98..c2f5cbb 100644 --- a/docs/g-sorcery.8 +++ b/docs/g-sorcery.8 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH G-SORCERY 8 "2015-04-20" "0.2" "g-sorcery" +.TH G-SORCERY 8 "2015-04-20" "0.2.1" "g-sorcery" .SH NAME g-sorcery \- manage overlays for 3rd party software providers . diff --git a/docs/g-sorcery.8.rst b/docs/g-sorcery.8.rst index 1e9af4e..2ace6b0 100644 --- a/docs/g-sorcery.8.rst +++ b/docs/g-sorcery.8.rst @@ -11,7 +11,7 @@ manage overlays for 3rd party software providers by Brian Dolbec. Integration with layman based on work of Auke Booij. :Date: 2015-04-20 :Copyright: Copyright (c) 2013-2015 Jauhien Piatlicki, License: GPL-2 -:Version: 0.2 +:Version: 0.2.1 :Manual section: 8 :Manual group: g-sorcery diff --git a/docs/g-sorcery.cfg.8 b/docs/g-sorcery.cfg.8 index ea97ada..f9c02ce 100644 --- a/docs/g-sorcery.cfg.8 +++ b/docs/g-sorcery.cfg.8 @@ -1,6 +1,6 @@ .\" Man page generated from reStructuredText. . -.TH G-SORCERY.CFG 8 "2015-04-20" "0.2" "g-sorcery" +.TH G-SORCERY.CFG 8 "2015-04-20" "0.2.1" "g-sorcery" .SH NAME g-sorcery.cfg \- custom settings for g-sorcery . diff --git a/docs/g-sorcery.cfg.8.rst b/docs/g-sorcery.cfg.8.rst index f9b5aac..bd85d08 100644 --- a/docs/g-sorcery.cfg.8.rst +++ b/docs/g-sorcery.cfg.8.rst @@ -11,7 +11,7 @@ custom settings for g-sorcery by Brian Dolbec. :Date: 2015-04-20 :Copyright: Copyright (c) 2013-2015 Jauhien Piatlicki, License: GPL-2 -:Version: 0.2 +:Version: 0.2.1 :Manual section: 8 :Manual group: g-sorcery diff --git a/g_sorcery/backend.py b/g_sorcery/backend.py index e606348..809ff18 100644 --- a/g_sorcery/backend.py +++ b/g_sorcery/backend.py @@ -4,10 +4,10 @@ """ backend.py ~~ - + base class for backends - -:copyright: (c) 2013 by Jauhien Piatlicki + +:copyright: (c) 2013-2015 by Jauhien Piatlicki :license: GPL-2, see LICENSE for more details. """ @@ -30,7 +30,7 @@ class Backend(object): Command format is as follows: g-backend [-o overlay_dir] [-r repository] command - + where command is one of the following: sync list @@ -38,11 +38,11 @@ class Backend(object): generate package_name generate-
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/git_syncer/
commit: 31ae597d8dbe927729df0af181a1312fbbe84130 Author: Jauhien Piatlicki gentoo org> AuthorDate: Sun Aug 16 22:03:21 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Sun Aug 16 22:29:47 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=31ae597d [g_sorcery/git_syncer] detect changes of branch and remote URL g_sorcery/git_syncer/git_syncer.py | 25 +++-- 1 file changed, 23 insertions(+), 2 deletions(-) diff --git a/g_sorcery/git_syncer/git_syncer.py b/g_sorcery/git_syncer/git_syncer.py index 0f6c58e..abde034 100644 --- a/g_sorcery/git_syncer/git_syncer.py +++ b/g_sorcery/git_syncer/git_syncer.py @@ -12,6 +12,8 @@ """ import os +import shutil +import subprocess from g_sorcery.compatibility import TemporaryDirectory @@ -46,8 +48,11 @@ class GITSyncer(Syncer): branch = "master" if os.path.exists(path): -#TODO: allow changing of remotes/branches -self.pull(path) +if self.branch_not_changed(path, branch) and self.remote_url_not_changed(path, db_uri): +self.pull(path) +else: +shutil.rmtree(path) +self.clone(db_uri, branch, path) else: self.clone(db_uri, branch, path) @@ -65,3 +70,19 @@ class GITSyncer(Syncer): def pull(self, path): if os.system("cd " + path + " && git pull"): raise SyncError("sync failed (pulling): " + path) + + +def branch_not_changed(self, path, branch): +try: +result = subprocess.check_output(["git", "rev-parse", "--abbrev-ref", "HEAD"], cwd=path).rstrip().decode("utf-8") +except Exception: +return False +return result == branch + + +def remote_url_not_changed(self, path, url): +try: +result = subprocess.check_output(["git", "config", "--get", "remote.origin.url"], cwd=path).rstrip().decode("utf-8") +except Exception: +return False +return result == url
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/bson/, tests/, g_sorcery/, g_sorcery/file_bson/, /
commit: 0ace08976a688a6d74cb86c70a156a87b3453214 Author: Jauhien Piatlicki gentoo org> AuthorDate: Fri Apr 17 14:34:46 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Fri Apr 17 17:10:31 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=0ace0897 [g_sorcery/file_bson] fix BSON support and add tests for it g_sorcery/db_layout.py | 2 +- g_sorcery/exceptions.py| 3 - g_sorcery/{bson => file_bson}/__init__.py | 0 g_sorcery/{bson/bson.py => file_bson/file_bson.py} | 8 +- g_sorcery/serialization.py | 4 +- setup.py | 4 +- tests/test_FileBSON.py | 95 ++ 7 files changed, 103 insertions(+), 13 deletions(-) diff --git a/g_sorcery/db_layout.py b/g_sorcery/db_layout.py index ea4298a..1f4514c 100644 --- a/g_sorcery/db_layout.py +++ b/g_sorcery/db_layout.py @@ -40,7 +40,7 @@ SUPPORTED_FILE_FORMATS = {JSON_FILE_SUFFIX: CategoryJSON} # bson module is optional, we should check if it is installed try: -from g_sorcery.bson.bson import FileBSON +from g_sorcery.file_bson.file_bson import FileBSON class CategoryBSON(FileBSON): """ diff --git a/g_sorcery/exceptions.py b/g_sorcery/exceptions.py index 4691ce6..6d68f7f 100644 --- a/g_sorcery/exceptions.py +++ b/g_sorcery/exceptions.py @@ -52,6 +52,3 @@ class DigestError(GSorceryError): class DownloadingError(GSorceryError): pass - -class SerializationError(GSorceryError): -pass diff --git a/g_sorcery/bson/__init__.py b/g_sorcery/file_bson/__init__.py similarity index 100% rename from g_sorcery/bson/__init__.py rename to g_sorcery/file_bson/__init__.py diff --git a/g_sorcery/bson/bson.py b/g_sorcery/file_bson/file_bson.py similarity index 91% rename from g_sorcery/bson/bson.py rename to g_sorcery/file_bson/file_bson.py index fdb8bb9..7bc6b80 100644 --- a/g_sorcery/bson/bson.py +++ b/g_sorcery/file_bson/file_bson.py @@ -2,8 +2,8 @@ # -*- coding: utf-8 -*- """ -bson.py -~~~ +file_bson.py + bson file format support @@ -29,10 +29,10 @@ class FileBSON(FileJSONData): content = {} bcnt = None with open(self.path, 'rb') as f: -bcnt = f.read() +bcnt = bson.BSON(f.read()) if not bcnt: raise FileJSONError('failed to read: ', self.path) -rawcnt = bson.BSON.decode(bcnt) +rawcnt = bcnt.decode() content = from_raw_serializable(rawcnt) return content diff --git a/g_sorcery/serialization.py b/g_sorcery/serialization.py index 780de6f..3a7704b 100644 --- a/g_sorcery/serialization.py +++ b/g_sorcery/serialization.py @@ -14,8 +14,6 @@ import json import importlib -from .exceptions import SerializationError - def step_to_raw_serializable(obj): """ Make one step of convertion of object @@ -54,7 +52,7 @@ def to_raw_serializable(obj): else: sobj = step_to_raw_serializable(obj) if not sobj: -raise SerializationError('Non serializable object: ', sobj) +raise TypeError('Non serializable object: ', sobj) return to_raw_serializable(sobj) diff --git a/setup.py b/setup.py index 866a38f..95b715e 100644 --- a/setup.py +++ b/setup.py @@ -15,7 +15,7 @@ import os from distutils.core import setup -SELECTABLE = ['bson'] +SELECTABLE = {'bson': 'file_bson'} use_defaults = ' '.join(list(SELECTABLE)) USE = os.environ.get("USE", use_defaults).split() @@ -23,7 +23,7 @@ USE = os.environ.get("USE", use_defaults).split() optional_modules = [] for mod in SELECTABLE: if mod in USE: -optional_modules.append('g_sorcery.%s' % mod) +optional_modules.append('g_sorcery.%s' % SELECTABLE[mod]) setup(name = 'g-sorcery', version = '0.2', diff --git a/tests/test_FileBSON.py b/tests/test_FileBSON.py new file mode 100644 index 000..ff2a7b3 --- /dev/null +++ b/tests/test_FileBSON.py @@ -0,0 +1,95 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +""" +test_FileBSON.py + + +FileBSON test suite + +:copyright: (c) 2013-2015 by Jauhien Piatlicki +:license: GPL-2, see LICENSE for more details. +""" + +import os +import unittest + +from tests.base import BaseTest + +BSON_INSTALLED = False + +try: +from g_sorcery.file_bson.file_bson import FileBSON +BSON_INSTALLED = True +except ImportError as e: +pass + +class NonSerializableClass(object): +pass + + +class SerializableClass(object): + +__slots__ = ("field1", "field2") + +def __init__(self, field1, field2): +self.field1 = field1 +self.field2 = field2 + +def __eq__(self, other): +return self.field1 == other.field1 \ + and self.field2 == other.field2 + +def serialize(self): +return {"field1": self.field1, "field2": sel
[gentoo-commits] proj/g-sorcery:dev commit in: /
commit: 8206504dc26f2beb9dc2027585b6a86cc39bc306 Author: Jauhien Piatlicki gentoo org> AuthorDate: Thu Apr 16 20:02:58 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Thu Apr 16 20:02:58 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=8206504d [pylint] add pylint.rc file pylint.rc | 336 ++ 1 file changed, 336 insertions(+) diff --git a/pylint.rc b/pylint.rc new file mode 100644 index 000..582038f --- /dev/null +++ b/pylint.rc @@ -0,0 +1,336 @@ +[MASTER] + +# Specify a configuration file. +#rcfile= + +# Python code to execute, usually for sys.path manipulation such as +# pygtk.require(). +#init-hook= + +# Profiled execution. +profile=no + +# Add files or directories to the blacklist. They should be base names, not +# paths. +ignore=CVS + +# Pickle collected data for later comparisons. +persistent=yes + +# List of plugins (as comma separated values of python modules names) to load, +# usually to register additional checkers. +load-plugins= + +# DEPRECATED +include-ids=no + +# DEPRECATED +symbols=no + + +[MESSAGES CONTROL] + +# Enable the message, report, category or checker with the given id(s). You can +# either give multiple identifier separated by comma (,) or put this option +# multiple time. See also the "--disable" option for examples. +#enable= + +# Disable the message, report, category or checker with the given id(s). You +# can either give multiple identifiers separated by comma (,) or put this +# option multiple times (only on the command line, not in the configuration +# file where it should appear only once).You can also use "--disable=all" to +# disable everything first and then reenable specific checks. For example, if +# you want to run only the similarities checker, you can use "--disable=all +# --enable=similarities". If you want to run only the classes checker, but have +# no Warning level messages displayed, use"--disable=all --enable=classes +# --disable=W" +#disable= + + +[REPORTS] + +# Set the output format. Available formats are text, parseable, colorized, msvs +# (visual studio) and html. You can also give a reporter class, eg +# mypackage.mymodule.MyReporterClass. +output-format=text + +# Put messages in a separate file for each module / package specified on the +# command line instead of printing them on stdout. Reports (if any) will be +# written in a file name "pylint_global.[txt|html]". +files-output=no + +# Tells whether to display a full report or only the messages +reports=yes + +# Python expression which should return a note less than 10 (10 is the highest +# note). You have access to the variables errors warning, statement which +# respectively contain the number of errors / warnings messages and the total +# number of statements analyzed. This is used by the global evaluation report +# (RP0004). +evaluation=10.0 - ((float(5 * error + warning + refactor + convention) / statement) * 10) + +# Add a comment according to your evaluation note. This is used by the global +# evaluation report (RP0004). +comment=no + +# Template used to display messages. This is a python new-style format string +# used to format the message information. See doc for all details +#msg-template= + + +[MISCELLANEOUS] + +# List of note tags to take in consideration, separated by a comma. +notes=FIXME,XXX,TODO + + +[BASIC] + +# Required attributes for module, separated by a comma +required-attributes= + +# List of builtins function names that should not be used, separated by a comma +bad-functions=map,filter,apply,input,file + +# Good variable names which should always be accepted, separated by a comma +good-names=i,j,k,ex,Run,_ + +# Bad variable names which should always be refused, separated by a comma +bad-names=foo,bar,baz,toto,tutu,tata + +# Colon-delimited sets of names that determine each other's naming style when +# the name regexes allow several styles. +name-group= + +# Include a hint for the correct naming format with invalid-name +include-naming-hint=no + +# Regular expression matching correct function names +function-rgx=[a-z_][a-z0-9_]{2,30}$ + +# Naming hint for function names +function-name-hint=[a-z_][a-z0-9_]{2,30}$ + +# Regular expression matching correct variable names +variable-rgx=[a-z_][a-z0-9_]{2,30}$ + +# Naming hint for variable names +variable-name-hint=[a-z_][a-z0-9_]{2,30}$ + +# Regular expression matching correct constant names +const-rgx=(([A-Z_][A-Z0-9_]*)|(__.*__))$ + +# Naming hint for constant names +const-name-hint=(([A-Z_][A-Z0-9_]*)|(__.*__))$ + +# Regular expression matching correct attribute names +attr-rgx=[a-z_][a-z0-9_]{2,30}$ + +# Naming hint for attribute names +attr-name-hint=[a-z_][a-z0-9_]{2,30}$ + +# Regular expression matching correct argument names +argument-rgx=[a-z_][a-z0-9_]{2,30}$ + +# Naming hint for argument names +argument-name-hint=[a-z_][a-z0-9_]{2,30}$ + +# Regular expression matching correct class attribute names +cl
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/, tests/
commit: 4b58d04be635a430c6e24666ac43c92fbd79af73 Author: Jauhien Piatlicki gentoo org> AuthorDate: Sat Apr 18 17:21:21 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Sat Apr 18 20:03:58 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=4b58d04b [g_sorcery/serialization] fix serialization for collections g_sorcery/g_collections.py | 13 +++- g_sorcery/serialization.py | 15 +- tests/serializable.py | 38 +++ tests/test_FileBSON.py | 40 - tests/test_FileJSON.py | 50 -- tests/test_PackageDB.py| 11 +++--- 6 files changed, 94 insertions(+), 73 deletions(-) diff --git a/g_sorcery/g_collections.py b/g_sorcery/g_collections.py index b28db6a..b9f3d06 100644 --- a/g_sorcery/g_collections.py +++ b/g_sorcery/g_collections.py @@ -4,14 +4,14 @@ """ g_collections.py - + Customized classes of standard python data types for use withing g-sorcery for custom formatted string output substitution in our ebuild templates and classes for storing information about packages and dependencies. - + :copyright: (c) 2013 by Brian Dolbec -:copyright: (c) 2013 by Jauhien Piatlicki +:copyright: (c) 2013-2015 by Jauhien Piatlicki :license: GPL-2, see LICENSE for more details. """ @@ -52,7 +52,7 @@ class serializable_elist(object): """ __slots__ = ('data') - + def __init__(self, iterable=None, separator=' '): ''' iterable: initialize from iterable's items @@ -60,6 +60,9 @@ class serializable_elist(object): ''' self.data = elist(iterable or [], separator) +def __eq__(self, other): +return self.data == other.data + def __iter__(self): return iter(self.data) @@ -122,7 +125,7 @@ class Dependency(object): def __init__(self, category, package, version="", operator=""): atom_str = operator + category + "/" + package -if version: +if version: atom_str += "-" + str(version) object.__setattr__(self, "atom", portage.dep.Atom(atom_str)) object.__setattr__(self, "category", category) diff --git a/g_sorcery/serialization.py b/g_sorcery/serialization.py index 3a7704b..b5d71f3 100644 --- a/g_sorcery/serialization.py +++ b/g_sorcery/serialization.py @@ -14,6 +14,8 @@ import json import importlib +from .compatibility import basestring + def step_to_raw_serializable(obj): """ Make one step of convertion of object @@ -40,19 +42,22 @@ def to_raw_serializable(obj): Convert object to the raw serializable type. Logic is the same as in the standard json encoder. """ -if isinstance(obj, str) \ +if isinstance(obj, basestring) \ or obj is None \ or obj is True \ or obj is False \ or isinstance(obj, int) \ - or isinstance(obj, float) \ - or isinstance(obj, (list, tuple)) \ - or isinstance(obj, dict): + or isinstance(obj, float): return obj +elif isinstance(obj, dict): +return {k: to_raw_serializable(v) for k, v in obj.items()} +elif isinstance(obj, (list, tuple)): +return [to_raw_serializable(item) for item in obj] + else: sobj = step_to_raw_serializable(obj) if not sobj: -raise TypeError('Non serializable object: ', sobj) +raise TypeError('Non serializable object: ', obj) return to_raw_serializable(sobj) diff --git a/tests/serializable.py b/tests/serializable.py new file mode 100644 index 000..1d3981e --- /dev/null +++ b/tests/serializable.py @@ -0,0 +1,38 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +""" +serializable.py +~~~ + +test classes for serialization + +:copyright: (c) 2013-2015 by Jauhien Piatlicki +:license: GPL-2, see LICENSE for more details. +""" + +class NonSerializableClass(object): +pass + + +class SerializableClass(object): + +__slots__ = ("field1", "field2") + +def __init__(self, field1, field2): +self.field1 = field1 +self.field2 = field2 + +def __eq__(self, other): +return self.field1 == other.field1 \ + and self.field2 == other.field2 + +def serialize(self): +return {"field1": self.field1, "field2": self.field2} + + +class DeserializableClass(SerializableClass): + +@classmethod +def deserialize(cls, value): +return DeserializableClass(value["field1"], value["field2"]) diff --git a/tests/test_FileBSON.py b/tests/test_FileBSON.py index ff2a7b3..1bf1b7d 100644 --- a/tests/test_FileBSON.py +++ b/tests/test_FileBSON.py @@ -14,7 +14,10 @@ import os import unittest +from g_sorcery.g_collections import serializable_elist + from tests.base import BaseTest +from tests.serializable import NonSeria
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/
commit: eef3ee7168fbf38bb5ef56a47fee0c0873cd4540 Author: Jauhien Piatlicki gentoo org> AuthorDate: Fri Apr 17 07:27:02 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Fri Apr 17 07:27:02 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=eef3ee71 [g_sorcery] fix DB cleaning do not write empty DB on cleaning, so syncing with old (v. 0) DBs works g_sorcery/db_layout.py | 1 + g_sorcery/package_db.py | 1 - 2 files changed, 1 insertion(+), 1 deletion(-) diff --git a/g_sorcery/db_layout.py b/g_sorcery/db_layout.py index fe6f281..ea4298a 100644 --- a/g_sorcery/db_layout.py +++ b/g_sorcery/db_layout.py @@ -230,6 +230,7 @@ class DBLayout(object): """ if os.path.exists(self.directory): shutil.rmtree(self.directory) +os.makedirs(self.directory) def read(self): """ diff --git a/g_sorcery/package_db.py b/g_sorcery/package_db.py index 5eeeb63..ec2d45f 100644 --- a/g_sorcery/package_db.py +++ b/g_sorcery/package_db.py @@ -216,7 +216,6 @@ class PackageDB(object): """ self.db_layout.clean() self.reset_db() -self.write() def write(self):
[gentoo-commits] proj/g-sorcery:dev commit in: docs/, g_sorcery/
commit: e77fc9ed91def889db3627f485d2af24afb9d118 Author: Jauhien Piatlicki gentoo org> AuthorDate: Mon Apr 20 12:12:30 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Mon Apr 20 21:12:58 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=e77fc9ed [docs] update documentation docs/developer_instructions.html | 130 +++--- docs/developer_instructions.rst | 149 --- docs/g-sorcery.8 | 10 +-- docs/g-sorcery.8.rst | 10 +-- docs/g-sorcery.cfg.8 | 8 +-- docs/g-sorcery.cfg.8.rst | 6 +- g_sorcery/ebuild.py | 28 g_sorcery/metadata.py| 24 +++ g_sorcery/package_db.py | 6 +- 9 files changed, 260 insertions(+), 111 deletions(-) diff --git a/docs/developer_instructions.html b/docs/developer_instructions.html index f4e89ab..4cea319 100644 --- a/docs/developer_instructions.html +++ b/docs/developer_instructions.html @@ -3,13 +3,13 @@ http://www.w3.org/1999/xhtml"; xml:lang="en" lang="en"> -http://docutils.sourceforge.net/"; /> +http://docutils.sourceforge.net/"; /> Developer Instructions
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/, tests/
commit: 15e264af65e08505ec85d8cb0dd53170ac044985 Author: Jauhien Piatlicki gentoo org> AuthorDate: Fri Apr 17 09:31:25 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Fri Apr 17 17:10:08 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=15e264af [g_sorcery/package_db] add category common data setter and getter to DB API g_sorcery/package_db.py | 38 ++ tests/test_PackageDB.py | 17 + 2 files changed, 51 insertions(+), 4 deletions(-) diff --git a/g_sorcery/package_db.py b/g_sorcery/package_db.py index ec2d45f..5374ae5 100644 --- a/g_sorcery/package_db.py +++ b/g_sorcery/package_db.py @@ -105,6 +105,7 @@ class PackageDB(object): self.pkg_name, self.pkg_data = next(self.pkgs_iter) self.vers_iter = iter(self.pkg_data.items()) +ebuild_data = dict(ebuild_data) ebuild_data.update(self.cat_data['common_data']) return (Package(self.cat_name, self.pkg_name, ver), ebuild_data) @@ -129,6 +130,7 @@ class PackageDB(object): self.pkg_name, self.pkg_data = next(self.pkgs_iter) self.vers_iter = iter(self.pkg_data.items()) +ebuild_data = dict(ebuild_data) ebuild_data.update(self.cat_data['common_data']) return (Package(self.cat_name, self.pkg_name, ver), ebuild_data) @@ -275,6 +277,42 @@ class PackageDB(object): self.categories[category] = description +def set_common_data(self, category, common_data): +""" +Set common data for a category. + +Args: +category: Category name. +common_data: Category common data. +""" +if not category in self.categories: +raise InvalidKeyError('Non-existent category: ' + category) + +if not category in self.database: +self.database[category] = {'common_data': common_data, 'packages': {}} +else: +self.database[category]['common_data'] = common_data + + +def get_common_data(self, category): +""" +Get common data for a category. + +Args: +category: Category name. + +Returns: +Dictionary with category common data. +""" +if not category in self.categories: +raise InvalidKeyError('Non-existent category: ' + category) + +if not category in self.database: +return {} +else: +return self.database[category]['common_data'] + + def add_package(self, package, ebuild_data=None): """ Add a package. diff --git a/tests/test_PackageDB.py b/tests/test_PackageDB.py index f73f006..2a67385 100644 --- a/tests/test_PackageDB.py +++ b/tests/test_PackageDB.py @@ -38,11 +38,15 @@ class TestPackageDB(BaseTest): orig_db = PackageDB(orig_path) orig_db.add_category("app-test1") orig_db.add_category("app-test2") -ebuild_data = {"test1": "test1", "test2": "test2"} +ebuild_data = {"test1": "tst1", "test2": "tst2"} +common_data = {"common1": "cmn1", "common2": "cmn2"} packages = [Package("app-test1", "test", "1"), Package("app-test1", "test", "2"), Package("app-test1", "test1", "1"), Package("app-test2", "test2", "1")] for package in packages: orig_db.add_package(package, ebuild_data) +orig_db.set_common_data("app-test1", common_data) +full_data = dict(ebuild_data) +full_data.update(common_data) orig_db.write() os.system("cd " + orig_tempdir.name + " && tar cvzf good.tar.gz db") @@ -61,6 +65,8 @@ class TestPackageDB(BaseTest): srv.join() test_db.read() self.assertEqual(orig_db.database, test_db.database) +self.assertEqual(orig_db.get_common_data("app-test1"), test_db.get_common_data("app-test1")) +self.assertEqual(orig_db.get_common_data("app-test2"), test_db.get_common_data("app-test2")) self.assertEqual(set(test_db.list_categories()), set(["app-test1", "app-test2"])) self.assertTrue(test_db.in_category("app-test1", "test")) self.assertFalse(test_db.in_category("app-test2", "test")) @@ -71,7 +77,7 @@ class TestPackageDB(BaseTest): self.assertRaises(InvalidKeyError, test_db.list_package_versions, "app-test1", "invalid") self.assertEqual(set(test_db.list_package_versions("app-test1", "test")), set(['1', '2'])) self.assertEqual(set(test_db.list_all_packages()), set(packages)) -self.assertEqual(test_db.get_package_description(packages[0]), ebuild_data) +self.assertEqual(test_db.get_package_description(packages[0]), full_data) self.assertRaises(KeyError, test_db.get_package_description, Package("invalid", "invalid", "1")) self.assertEqual
[gentoo-commits] proj/g-sorcery:dev commit in: tests/, g_sorcery/
commit: d01b0ddac70c475225b73f36a4b863598653b0b0 Author: Jauhien Piatlicki gentoo org> AuthorDate: Fri Apr 17 15:02:38 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Fri Apr 17 17:10:47 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=d01b0dda [g_sorcery/db_layout] fix BSON support and add tests for it g_sorcery/db_layout.py | 6 ++- tests/server.py | 4 +- tests/test_PackageDB.py | 133 ++-- 3 files changed, 80 insertions(+), 63 deletions(-) diff --git a/g_sorcery/db_layout.py b/g_sorcery/db_layout.py index 1f4514c..da779bd 100644 --- a/g_sorcery/db_layout.py +++ b/g_sorcery/db_layout.py @@ -40,7 +40,7 @@ SUPPORTED_FILE_FORMATS = {JSON_FILE_SUFFIX: CategoryJSON} # bson module is optional, we should check if it is installed try: -from g_sorcery.file_bson.file_bson import FileBSON +from .file_bson.file_bson import FileBSON class CategoryBSON(FileBSON): """ @@ -172,9 +172,13 @@ def get_layout(metadata): return (CategoryJSON, [file_name(CATEGORIES_FILE_NAME)]) elif layout_version == 1: category_format = metadata['category_format'] +wrong_fmt = True try: category_cls = SUPPORTED_FILE_FORMATS[category_format] +wrong_fmt = False except KeyError: +pass +if wrong_fmt: raise DBLayoutError("unsupported packages file format: " + category_format) return (category_cls, [file_name(CATEGORIES_FILE_NAME), file_name(METADATA_FILE_NAME)]) else: diff --git a/tests/server.py b/tests/server.py index 5cb813f..51d49b7 100644 --- a/tests/server.py +++ b/tests/server.py @@ -38,10 +38,10 @@ def HTTPRequestHandlerGenerator(direct): class Server(threading.Thread): -def __init__(self, directory): +def __init__(self, directory, port=8080): super(Server, self).__init__() HTTPServer.allow_reuse_address = True -server_address = ('127.0.0.1', 8080) +server_address = ('127.0.0.1', port) self.httpd = HTTPServer(server_address, HTTPRequestHandlerGenerator(directory)) def run(self): diff --git a/tests/test_PackageDB.py b/tests/test_PackageDB.py index 2a67385..221be00 100644 --- a/tests/test_PackageDB.py +++ b/tests/test_PackageDB.py @@ -16,6 +16,7 @@ import time import unittest from g_sorcery.compatibility import TemporaryDirectory +from g_sorcery.db_layout import JSON_FILE_SUFFIX, BSON_FILE_SUFFIX from g_sorcery.exceptions import IntegrityError, InvalidKeyError, SyncError from g_sorcery.g_collections import Package from g_sorcery.package_db import PackageDB @@ -23,6 +24,14 @@ from g_sorcery.package_db import PackageDB from tests.base import BaseTest from tests.server import Server +SUPPORTED_FILE_FORMATS = [JSON_FILE_SUFFIX] +# bson module is optional, we should check if it is installed +try: +from g_sorcery.file_bson.file_bson import FileBSON +SUPPORTED_FILE_FORMATS.append(BSON_FILE_SUFFIX) +except ImportError as e: +pass + class TestDB(PackageDB): def get_real_db_uri(self, db_uri): @@ -32,66 +41,70 @@ class TestDB(PackageDB): class TestPackageDB(BaseTest): def test_functionality(self): -orig_tempdir = TemporaryDirectory() -orig_path = os.path.join(orig_tempdir.name, "db") -os.makedirs(orig_path) -orig_db = PackageDB(orig_path) -orig_db.add_category("app-test1") -orig_db.add_category("app-test2") -ebuild_data = {"test1": "tst1", "test2": "tst2"} -common_data = {"common1": "cmn1", "common2": "cmn2"} -packages = [Package("app-test1", "test", "1"), Package("app-test1", "test", "2"), -Package("app-test1", "test1", "1"), Package("app-test2", "test2", "1")] -for package in packages: -orig_db.add_package(package, ebuild_data) -orig_db.set_common_data("app-test1", common_data) -full_data = dict(ebuild_data) -full_data.update(common_data) - -orig_db.write() -os.system("cd " + orig_tempdir.name + " && tar cvzf good.tar.gz db") -os.system("echo invalid >> " + orig_tempdir.name + "/db/app-test1/packages.json") -os.system("cd " + orig_tempdir.name + " && tar cvzf dummy.tar.gz db") - -test_db = TestDB(self.tempdir.name) -self.assertRaises(SyncError, test_db.sync, "127.0.0.1:8080") - -srv = Server(orig_tempdir.name) -srv.start() -self.assertRaises(IntegrityError, test_db.sync, "127.0.0.1:8080") -os.system("cd " + orig_tempdir.name + " && mv good.tar.gz dummy.tar.gz") -test_db.sync("127.0.0.1:8080") -srv.shutdown() -srv.join() -test_db.read() -self.assertEqual(orig_db.database, test_db.database) -self.assertEqual(orig_db.get_common_data("app-test1"), test_db.get_common_data("app-test1")) -self.assertEqual(or
[gentoo-commits] proj/g-sorcery:dev commit in: gs_db_tool/, /, scripts/, tests/, g_sorcery/bson/, g_sorcery/
commit: e6fcf51f824bd476bec6106b7ad478e73be34cc5 Author: Jauhien Piatlicki gentoo org> AuthorDate: Thu Apr 16 22:23:16 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Thu Apr 16 22:23:16 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=e6fcf51f change DB structure move to new DB layout with versioning support bson format for category files g_sorcery/bson/__init__.py | 1 + g_sorcery/bson/bson.py | 47 + g_sorcery/db_layout.py | 283 g_sorcery/exceptions.py| 12 +- g_sorcery/fileutils.py | 62 -- g_sorcery/package_db.py| 455 + g_sorcery/serialization.py | 98 -- gs_db_tool/gs_db_tool.py | 10 +- scripts/all_pythons.sh | 2 +- setup.py | 29 ++- tests/test_PackageDB.py| 10 +- 11 files changed, 707 insertions(+), 302 deletions(-) diff --git a/g_sorcery/bson/__init__.py b/g_sorcery/bson/__init__.py new file mode 100644 index 000..4265cc3 --- /dev/null +++ b/g_sorcery/bson/__init__.py @@ -0,0 +1 @@ +#!/usr/bin/env python diff --git a/g_sorcery/bson/bson.py b/g_sorcery/bson/bson.py new file mode 100644 index 000..fdb8bb9 --- /dev/null +++ b/g_sorcery/bson/bson.py @@ -0,0 +1,47 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +""" +bson.py +~~~ + +bson file format support + +:copyright: (c) 2015 by Jauhien Piatlicki +:license: GPL-2, see LICENSE for more details. +""" + +import bson + +from g_sorcery.exceptions import FileJSONError +from g_sorcery.fileutils import FileJSONData +from g_sorcery.serialization import from_raw_serializable, to_raw_serializable + +class FileBSON(FileJSONData): +""" +Class for BSON files. Supports custom JSON serialization +provided by g_sorcery.serialization. +""" +def read_content(self): +""" +Read BSON file. +""" +content = {} +bcnt = None +with open(self.path, 'rb') as f: +bcnt = f.read() +if not bcnt: +raise FileJSONError('failed to read: ', self.path) +rawcnt = bson.BSON.decode(bcnt) +content = from_raw_serializable(rawcnt) +return content + + +def write_content(self, content): +""" +Write BSON file. +""" +rawcnt = to_raw_serializable(content) +bcnt = bson.BSON.encode(rawcnt) +with open(self.path, 'wb') as f: +f.write(bcnt) diff --git a/g_sorcery/db_layout.py b/g_sorcery/db_layout.py new file mode 100644 index 000..fe6f281 --- /dev/null +++ b/g_sorcery/db_layout.py @@ -0,0 +1,283 @@ +#!/usr/bin/env python +# -*- coding: utf-8 -*- + +""" +db_layout.py + + +package database file layout + +:copyright: (c) 2013-2015 by Jauhien Piatlicki +:license: GPL-2, see LICENSE for more details. +""" + +import hashlib +import os +import shutil + +from .exceptions import DBLayoutError, DBStructureError, FileJSONError, IntegrityError +from .fileutils import FileJSON, hash_file + +CATEGORIES_FILE_NAME = 'categories' +MANIFEST_FILE_NAME = 'manifest' +METADATA_FILE_NAME = 'metadata' +PACKAGES_FILE_NAME = 'packages' + +JSON_FILE_SUFFIX = 'json' +BSON_FILE_SUFFIX = 'bson' + +class CategoryJSON(FileJSON): +""" +Category file in JSON format. +""" +def __init__(self, directory, category): +super(CategoryJSON, self).__init__(os.path.join(os.path.abspath(directory), category), + file_name(PACKAGES_FILE_NAME, JSON_FILE_SUFFIX)) + + +SUPPORTED_FILE_FORMATS = {JSON_FILE_SUFFIX: CategoryJSON} + + +# bson module is optional, we should check if it is installed +try: +from g_sorcery.bson.bson import FileBSON + +class CategoryBSON(FileBSON): +""" +Category file in BSON format. +""" +def __init__(self, directory, category): +super(CategoryBSON, self).__init__(os.path.join(os.path.abspath(directory), category), + file_name(PACKAGES_FILE_NAME, BSON_FILE_SUFFIX)) + +SUPPORTED_FILE_FORMATS[BSON_FILE_SUFFIX] = CategoryBSON + +except ImportError as e: +pass + + +def file_name(name, suffix=JSON_FILE_SUFFIX): +""" +Return file name based on name and suffix. +""" +return name + '.' + suffix + + +class Manifest(FileJSON): +""" +Manifest file. +""" + +def __init__(self, directory): +super(Manifest, self).__init__(os.path.abspath(directory), file_name(MANIFEST_FILE_NAME)) + +def check(self): +""" +Check manifest. +""" +manifest = self.read() + +result = True +errors = [] + +names = [file_name(CATEGORIES_FILE_NAME)] +for name in names: +if not name in manifest: +raise DBLayoutError('Bad manifest: no ' + name + ' entry') + +for name, value in manif
[gentoo-commits] proj/g-sorcery:dev commit in: /
commit: 84e5836cd7c47683b78c24783c2fb62c49dc4b8a Author: Jauhien Piatlicki gentoo org> AuthorDate: Sat Apr 18 16:41:27 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Sat Apr 18 16:41:27 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=84e5836c [setup] fix copyright setup.py | 3 +++ 1 file changed, 3 insertions(+) diff --git a/setup.py b/setup.py index 95b715e..68ad410 100644 --- a/setup.py +++ b/setup.py @@ -8,6 +8,9 @@ installation script :copyright: (c) 2013-2015 by Jauhien Piatlicki +:copyright: (c) 2014 by Brian Dolbec +(code for conditional module installation +is taken from the layman project) :license: GPL-2, see LICENSE for more details. """
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/
commit: bbe16a229b42e96f847246d82e8f1bcff4d60a82 Author: Jauhien Piatlicki gentoo org> AuthorDate: Fri Apr 17 20:59:50 2015 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Fri Apr 17 20:59:50 2015 + URL:https://gitweb.gentoo.org/proj/g-sorcery.git/commit/?id=bbe16a22 [g_sorcery/db_layout,g_sorcery/package_db] better version/format checking, fix exceptions g_sorcery/db_layout.py | 4 +++- g_sorcery/package_db.py | 19 +-- 2 files changed, 16 insertions(+), 7 deletions(-) diff --git a/g_sorcery/db_layout.py b/g_sorcery/db_layout.py index da779bd..8eb81da 100644 --- a/g_sorcery/db_layout.py +++ b/g_sorcery/db_layout.py @@ -26,6 +26,8 @@ PACKAGES_FILE_NAME = 'packages' JSON_FILE_SUFFIX = 'json' BSON_FILE_SUFFIX = 'bson' +SUPPORTED_DB_LAYOUTS=[0, 1] + class CategoryJSON(FileJSON): """ Category file in JSON format. @@ -182,7 +184,7 @@ def get_layout(metadata): raise DBLayoutError("unsupported packages file format: " + category_format) return (category_cls, [file_name(CATEGORIES_FILE_NAME), file_name(METADATA_FILE_NAME)]) else: -raise DBLayoutError("unsupported DB layout version: " + layout_version) +raise DBLayoutError("unsupported DB layout version: " + str(layout_version)) class DBLayout(object): diff --git a/g_sorcery/package_db.py b/g_sorcery/package_db.py index 5374ae5..4c67fdc 100644 --- a/g_sorcery/package_db.py +++ b/g_sorcery/package_db.py @@ -18,12 +18,13 @@ import portage from .compatibility import basestring, py2k, TemporaryDirectory -from .db_layout import DBLayout, JSON_FILE_SUFFIX -from .exceptions import DBError, DBStructureError, InvalidKeyError, SyncError +from .db_layout import DBLayout, JSON_FILE_SUFFIX, SUPPORTED_DB_LAYOUTS, SUPPORTED_FILE_FORMATS +from .exceptions import DBError, DBLayoutError, DBStructureError, InvalidKeyError, SyncError from .fileutils import FileJSON, load_remote_file, copy_all, wget from .g_collections import Package from .logger import Logger +SUPPORTED_DB_STRUCTURES=[0, 1] class PackageDB(object): """ @@ -142,11 +143,17 @@ class PackageDB(object): if preferred_layout_version == 0 \ and preferred_db_version != 0: -raise DBStructureError("Wrong DB version: " + preferred_db_version + \ +raise DBStructureError("Wrong DB version: " + str(preferred_db_version) + \ ", with DB layout version 0 it can be only 0") -if not preferred_db_version in [0, 1]: -raise DBStructureError("Unsupported DB version: " + preferred_db_version) +if not preferred_db_version in SUPPORTED_DB_STRUCTURES: +raise DBStructureError("Unsupported DB version: " + str(preferred_db_version)) + +if not preferred_layout_version in SUPPORTED_DB_LAYOUTS: +raise DBLayoutError("unsupported DB layout version: " + str(preferred_layout_version)) + +if not preferred_category_format in SUPPORTED_FILE_FORMATS: +raise DBLayoutError("unsupported packages file format: " + preferred_category_format) self.logger = Logger() self.directory = os.path.abspath(directory) @@ -261,7 +268,7 @@ class PackageDB(object): elif db_version == 1: pass else: -raise DBStructureError("Unsupported DB version: " + db_version) +raise DBStructureError("Unsupported DB version: " + str(db_version)) def add_category(self, category, description=None):
[gentoo-commits] proj/g-sorcery:dev commit in: /
commit: d05d6f623f33ab04c1ffdb76f11a27e931310b9d Author: Jauhien Piatlicki zertisa com> AuthorDate: Mon Sep 29 19:36:36 2014 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Mon Sep 29 19:36:36 2014 + URL: http://sources.gentoo.org/gitweb/?p=proj/g-sorcery.git;a=commit;h=d05d6f62 Merge branch 'heroxbd/g-sorcery:master' into dev g_sorcery/ebuild.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
[gentoo-commits] proj/g-sorcery:dev commit in: g_sorcery/
commit: 2b33a61b659b35280616715fb9c522ef111520c7 Author: Benda Xu gentoo org> AuthorDate: Thu May 8 01:03:47 2014 + Commit: Jauhien Piatlicki gentoo org> CommitDate: Thu May 8 01:03:47 2014 + URL: http://sources.gentoo.org/gitweb/?p=proj/g-sorcery.git;a=commit;h=2b33a61b g_sorcery/ebuild.py: fix typo --- g_sorcery/ebuild.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/g_sorcery/ebuild.py b/g_sorcery/ebuild.py index c7fc463..31782f0 100644 --- a/g_sorcery/ebuild.py +++ b/g_sorcery/ebuild.py @@ -154,7 +154,7 @@ class DefaultEbuildGenerator(EbuildGenerator): Default ebuild generator. Takes a layout dictinary that describes ebuild structure and generates -an ebuild temlate basing on it. +an ebuild template basing on it. Layout has entries for vars and inherited eclasses. Each entry is a list. Entries are processed in the following order:
[gentoo-commits] proj/g-sorcery:dev commit in: /
commit: 561e590173dbc44720c57df5c656a7e857a0f667 Author: Jauhien Piatlicki gentoo org> AuthorDate: Sat May 10 02:03:05 2014 + Commit: Jauhien Piatlicki gmail com> CommitDate: Sat May 10 02:03:05 2014 + URL: http://git.overlays.gentoo.org/gitweb/?p=proj/g-sorcery.git;a=commit;h=561e5901 update README --- README.md | 7 --- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/README.md b/README.md index 4b25590..f0070a8 100644 --- a/README.md +++ b/README.md @@ -46,16 +46,17 @@ databases with information about available software and so on. Installation and using == -You should emerge **app-portage/layman-**. +You will need **app-portage/layman-** (when you emerge a backend you are +interested in it will be pulled in authomatically). Add `jauhien` overlay: **layman -a jauhien**. -Emerge g-sorcery: **emerge -va g-sorcery**. - Currently 2 backends are available: **gs-elpa** and **gs-pypi**. Here is an example of using gs-elpa backend. +Emerge backend you want to use: **emerge -va gs-elpa**. + There are two ways of using **gs-elpa**: * use it with **layman**
[gentoo-commits] proj/g-sorcery:dev commit in: gs_pypi/data/, /, docs/, bin/, gs_pypi/
commit: e342f62e8cc2265b6085700de85c8a0eea2c0b98 Author: Jauhien Piatlicki gentoo org> AuthorDate: Sat May 10 01:49:12 2014 + Commit: Jauhien Piatlicki gmail com> CommitDate: Sat May 10 01:49:12 2014 + URL: http://git.overlays.gentoo.org/gitweb/?p=proj/g-sorcery.git;a=commit;h=e342f62e gs-pypi moved to another repo --- bin/gs-pypi| 3 - bin/gs-pypi-generate-db| 18 --- docs/Makefile | 2 +- docs/gs-pypi.8 | 153 -- docs/gs-pypi.8.rst | 130 -- gs-pypi-overlays.xml | 14 -- gs-pypi.json | 46 --- gs_pypi/__init__.py| 2 - gs_pypi/backend.py | 35 - gs_pypi/data/gs-pypi.eclass| 15 --- gs_pypi/ebuild.py | 67 -- gs_pypi/gs_pypi_generate_db.py | 60 - gs_pypi/pypi_db.py | 289 - setup.py | 12 +- 14 files changed, 5 insertions(+), 841 deletions(-) diff --git a/bin/gs-pypi b/bin/gs-pypi deleted file mode 100644 index fbaabd4..000 --- a/bin/gs-pypi +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/bash - -g-sorcery gs-pypi $@ diff --git a/bin/gs-pypi-generate-db b/bin/gs-pypi-generate-db deleted file mode 100755 index 81710ec..000 --- a/bin/gs-pypi-generate-db +++ /dev/null @@ -1,18 +0,0 @@ -#!/usr/bin/env python -# -*- coding: utf-8 -*- - -""" -gs-pypi-generate-db -~~~ - -PyPI database generation - -:copyright: (c) 2013 by Jauhien Piatlicki -:license: GPL-2, see LICENSE for more details. -""" - -import sys -from gs_pypi import gs_pypi_generate_db - -if __name__ == "__main__": -sys.exit(gs_pypi_generate_db.main()) diff --git a/docs/Makefile b/docs/Makefile index 5c25523..6c47097 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -1,7 +1,7 @@ HTML_SOURCES=developer_instructions HTML_DOCS=$(HTML_SOURCES:=.html) -MAN_SOURCES=g-sorcery g-sorcery.cfg gs-pypi +MAN_SOURCES=g-sorcery g-sorcery.cfg MANS=$(MAN_SOURCES:=.8) RST2HTML=rst2html.py diff --git a/docs/gs-pypi.8 b/docs/gs-pypi.8 deleted file mode 100644 index 7536605..000 --- a/docs/gs-pypi.8 +++ /dev/null @@ -1,153 +0,0 @@ -.\" Man page generated from reStructuredText. -. -.TH GS-PYPI 8 "2013-08-04" "0.1" "g-sorcery" -.SH NAME -gs-pypi \- manage overlays for PYPI repository -. -.nr rst2man-indent-level 0 -. -.de1 rstReportMargin -\\$1 \\n[an-margin] -level \\n[rst2man-indent-level] -level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] -- -\\n[rst2man-indent0] -\\n[rst2man-indent1] -\\n[rst2man-indent2] -.. -.de1 INDENT -.\" .rstReportMargin pre: -. RS \\$1 -. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] -. nr rst2man-indent-level +1 -.\" .rstReportMargin post: -.. -.de UNINDENT -. RE -.\" indent \\n[an-margin] -.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] -.nr rst2man-indent-level -1 -.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] -.in \\n[rst2man-indent\\n[rst2man-indent-level]]u -.. -.SH SYNOPSIS -.sp -\fBgs\-pypi\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBsync\fP -.sp -\fBgs\-pypi\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBlist\fP -.sp -\fBgs\-pypi\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBgenerate\fP \fIPACKAGE\fP -.sp -\fBgs\-pypi\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBinstall\fP \fIPACKAGE\fP -.sp -\fBgs\-pypi\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBgenerate\-tree\fP [\fB\-d\fP] -.SH DESCRIPTION -.sp -\fBgs\-pypi\fP is an ebuild generator for PYPI PYTHON repository. -.sp -There are two ways of using \fBgs\-pypi\fP: -.INDENT 0.0 -.INDENT 3.5 -.INDENT 0.0 -.IP \(bu 2 -use it with \fBlayman\fP -.sp -In this case all you need to do is install \fBlayman\-\fP and \fBg\-sorcery\fP. -Then you should just run \fIlayman \-L\fP as -root and find an overlay you want. Type of overlay will be -displayed as \fIg\-sorcery\fP. Then you add this overlay as -usual. It\(aqs all you need to do and it\(aqs the recommended way of -using \fBgs\-pypi\fP. -.IP \(bu 2 -use it as stand\-alone tool -.sp -In this case you should create an overlay (see \fBportage\fP documentation), sync it and populate -it with one or more ebuilds. Then ebuilds could be installed by emerge or by \fBgs\-pypi\fP tool. -.UNINDENT -.UNINDENT -.UNINDENT -.SH OPTIONS -.INDENT 0.0 -.TP -.B \fB\-\-overlay\fP \fIOVERLAY\fP, \fB\-o\fP \fIOVERLAY\fP -Overlay directory. This option is mandatory if there is no -\fBdefault_overlay\fP entry in a backend config. -.TP -.B \fB\-\-repository\fP \fIREPO\fP, \fB\-r\fP \fIREPO\fP -Repository name. This option is not mandatory. If present should be \fBctan\fP. -.UNINDENT -.SH COMMANDS -.INDENT 0.0 -.TP -.B \fBsync\fP -Synchronize a repository database. -.TP -.B \fBlist\fP -List packages available in a repository. -.TP -.B \fBgenerate\fP -Generate a given ebuild and all its dependencies. -.TP -.B \
[gentoo-commits] proj/g-sorcery:dev commit in: docs/, bin/
commit: fa96bd7d15b4c84a32a965360ee2085f1f19f59c Author: Jauhien Piatlicki gentoo org> AuthorDate: Sat May 10 01:18:20 2014 + Commit: Jauhien Piatlicki gmail com> CommitDate: Sat May 10 01:18:20 2014 + URL: http://git.overlays.gentoo.org/gitweb/?p=proj/g-sorcery.git;a=commit;h=fa96bd7d fix: remove rest of gs-elpa --- bin/gs-elpa | 3 --- docs/Makefile | 2 +- 2 files changed, 1 insertion(+), 4 deletions(-) diff --git a/bin/gs-elpa b/bin/gs-elpa deleted file mode 100755 index ce096fa..000 --- a/bin/gs-elpa +++ /dev/null @@ -1,3 +0,0 @@ -#!/bin/bash - -g-sorcery gs-elpa $@ diff --git a/docs/Makefile b/docs/Makefile index cba7bc8..8af5b2c 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -1,7 +1,7 @@ HTML_SOURCES=developer_instructions HTML_DOCS=$(HTML_SOURCES:=.html) -MAN_SOURCES=g-sorcery g-sorcery.cfg gs-elpa gs-pypi gs-ctan +MAN_SOURCES=g-sorcery g-sorcery.cfg gs-pypi gs-ctan MANS=$(MAN_SOURCES:=.8) RST2HTML=rst2html.py
[gentoo-commits] proj/g-sorcery:dev commit in: /, gs_elpa/data/, docs/, gs_elpa/
commit: 5e3998bed044a5a1b6802aa4f51caeea5837f18b Author: Jauhien Piatlicki gentoo org> AuthorDate: Sat May 10 01:15:53 2014 + Commit: Jauhien Piatlicki gmail com> CommitDate: Sat May 10 01:15:53 2014 + URL: http://git.overlays.gentoo.org/gitweb/?p=proj/g-sorcery.git;a=commit;h=5e3998be gs-elpa moved to another repo --- docs/gs-elpa.8 | 165 docs/gs-elpa.8.rst | 144 -- gs-elpa-overlays.xml| 34 - gs-elpa.json| 54 --- gs_elpa/__init__.py | 2 - gs_elpa/backend.py | 37 -- gs_elpa/data/gs-elpa.eclass | 63 - gs_elpa/ebuild.py | 65 - gs_elpa/elpa_db.py | 130 -- 9 files changed, 694 deletions(-) diff --git a/docs/gs-elpa.8 b/docs/gs-elpa.8 deleted file mode 100644 index cd71794..000 --- a/docs/gs-elpa.8 +++ /dev/null @@ -1,165 +0,0 @@ -.\" Man page generated from reStructuredText. -. -.TH GS-ELPA 8 "2013-08-04" "0.1" "g-sorcery" -.SH NAME -gs-elpa \- manage overlays for ELPA repositories -. -.nr rst2man-indent-level 0 -. -.de1 rstReportMargin -\\$1 \\n[an-margin] -level \\n[rst2man-indent-level] -level margin: \\n[rst2man-indent\\n[rst2man-indent-level]] -- -\\n[rst2man-indent0] -\\n[rst2man-indent1] -\\n[rst2man-indent2] -.. -.de1 INDENT -.\" .rstReportMargin pre: -. RS \\$1 -. nr rst2man-indent\\n[rst2man-indent-level] \\n[an-margin] -. nr rst2man-indent-level +1 -.\" .rstReportMargin post: -.. -.de UNINDENT -. RE -.\" indent \\n[an-margin] -.\" old: \\n[rst2man-indent\\n[rst2man-indent-level]] -.nr rst2man-indent-level -1 -.\" new: \\n[rst2man-indent\\n[rst2man-indent-level]] -.in \\n[rst2man-indent\\n[rst2man-indent-level]]u -.. -.SH SYNOPSIS -.sp -\fBgs\-elpa\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBsync\fP -.sp -\fBgs\-elpa\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBlist\fP -.sp -\fBgs\-elpa\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBgenerate\fP \fIPACKAGE\fP -.sp -\fBgs\-elpa\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBinstall\fP \fIPACKAGE\fP -.sp -\fBgs\-elpa\fP \fB\-o\fP \fIOVERLAY\fP [\fB\-r\fP \fIREPO\fP] \fBgenerate\-tree\fP [\fB\-d\fP] -.SH DESCRIPTION -.sp -\fBgs\-elpa\fP is an ebuild generator for ELPA repositories \-\- repositories with -elisp packages for emacs. -.sp -There are two ways of using \fBgs\-elpa\fP: -.INDENT 0.0 -.INDENT 3.5 -.INDENT 0.0 -.IP \(bu 2 -use it with \fBlayman\fP -.sp -In this case all you need to do is install \fBlayman\-\fP and \fBg\-sorcery\fP. -Then you should just run \fIlayman \-L\fP as -root and find an overlay you want. Type of overlay will be -displayed as \fIg\-sorcery\fP. Then you add this overlay as -usual. It\(aqs all you need to do and it\(aqs the recommended way of -using \fBgs\-elpa\fP. -.IP \(bu 2 -use it as stand\-alone tool -.sp -In this case you should create an overlay (see \fBportage\fP documentation), sync it and populate -it with one or more ebuilds. Then ebuilds could be installed by emerge or by \fBgs\-elpa\fP tool. -.UNINDENT -.UNINDENT -.UNINDENT -.SH OPTIONS -.INDENT 0.0 -.TP -.B \fB\-\-overlay\fP \fIOVERLAY\fP, \fB\-o\fP \fIOVERLAY\fP -Overlay directory. This option is mandatory if there is no -\fBdefault_overlay\fP entry in a backend config. -.TP -.B \fB\-\-repository\fP \fIREPO\fP, \fB\-r\fP \fIREPO\fP -Repository name. Can be one of \fBgnu\-elpa\fP, \fBmarmalade\fP, \fBmelpa\fP. -.UNINDENT -.SH COMMANDS -.INDENT 0.0 -.TP -.B \fBsync\fP -Synchronize a repository database. -.TP -.B \fBlist\fP -List packages available in a repository. -.TP -.B \fBgenerate\fP -Generate a given ebuild and all its dependencies. -.TP -.B \fBinstall\fP -Generate and install an ebuild using your package mangler. -.TP -.B \fBgenerate\-tree\fP -Generate entire overlay structure. Without option \fB\-d\fP after -this command sources are not fetched during generation and there -are no entries for them in Manifest files. -.UNINDENT -.SH FILES -.INDENT 0.0 -.TP -.B \fB/etc/g\-sorcery/gs\-elpa.json\fP -Backend config. -.TP -.B \fB/etc/layman/overlays/gs\-elpa\-overlays.xml\fP -List of available repositories. -.UNINDENT -.SH EXAMPLES -.INDENT 0.0 -.TP -.B Using gs\-elpa with layman -Execute -.sp -\fBlayman \-L\fP -.sp -Find there an overlay you need (there are -3 gs\-elpa overlays currently: gnu\-elpa, marmalade and melpa). -Add, e.g. -.sp -\fBlayman \-a gnu\-elpa \-a marmalade\fP -.sp -Emerge any package from it, e.g. -.sp -\fBemerge \-va clojure\-mode\fP -.TP -.B Generating user ebuilds in user overlay -Create new user overlay. Run -.sp -\fBgs\-elpa \-o\fP \fIOVERLAY_DIRECTORY\fP \fB\-r gnu\-elpa\fP \fBsync\fP -.sp -List packages: -.sp -\fBgs\-elpa \-o\fP \fIOVERLAY_DIRECTORY\fP \fB\-r gnu\-elpa\fP \fBlist\fP -.sp -Install any package you want: -.sp -\fBgs\-elpa \-o\fP \fIOVERLAY_DIRECTORY\fP \fB\-r gnu\-elpa\fP \fBins
[gentoo-commits] proj/g-sorcery:dev commit in: /
commit: e2eba33ba1eccf351fc8264a00bbba817f179285 Author: Jauhien Piatlicki gentoo org> AuthorDate: Sat May 10 00:59:32 2014 + Commit: Jauhien Piatlicki gmail com> CommitDate: Sat May 10 00:59:32 2014 + URL: http://git.overlays.gentoo.org/gitweb/?p=proj/g-sorcery.git;a=commit;h=e2eba33b prepare to split gs-elpa to another package --- setup.py | 11 --- 1 file changed, 4 insertions(+), 7 deletions(-) diff --git a/setup.py b/setup.py index 10f0461..9a1291a 100644 --- a/setup.py +++ b/setup.py @@ -6,19 +6,16 @@ setup(name = 'g-sorcery', version = '0.1_alpha', description = 'g-sorcery framework for automated ebuild generators', author= 'Jauhien Piatlicki', - author_email = 'piatli...@gmail.com', - packages = ['g_sorcery', 'gs_db_tool', 'gs_elpa', 'gs_ctan', 'gs_pypi'], + author_email = 'jauh...@gentoo.org', + packages = ['g_sorcery', 'gs_db_tool', 'gs_ctan', 'gs_pypi'], package_data = {'g_sorcery': ['data/*'], 'gs_pypi': ['data/*'], - 'gs_elpa': ['data/*'], 'gs_ctan': ['data/*']}, - scripts = ['bin/g-sorcery', 'bin/gs-db-tool', 'bin/gs-elpa', + scripts = ['bin/g-sorcery', 'bin/gs-db-tool', 'bin/gs-ctan', 'bin/gs-pypi-generate-db', 'bin/gs-pypi'], - data_files= [('/etc/g-sorcery/', ['gs-elpa.json']), - ('/etc/g-sorcery/', ['gs-ctan.json']), + data_files= [('/etc/g-sorcery/', ['gs-ctan.json']), ('/etc/g-sorcery/', ['gs-pypi.json']), ('/etc/g-sorcery/', ['g-sorcery.cfg']), - ('/etc/layman/overlays/', ['gs-elpa-overlays.xml']), ('/etc/layman/overlays/', ['gs-ctan-overlays.xml']), ('/etc/layman/overlays/', ['gs-pypi-overlays.xml'])], license = 'GPL',