Hello community, here is the log from the commit of package benji for openSUSE:Factory checked in at 2020-09-22 21:11:08 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/benji (Old) and /work/SRC/openSUSE:Factory/.benji.new.4249 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "benji" Tue Sep 22 21:11:08 2020 rev:10 rq:835931 version:0.12.0 Changes: -------- --- /work/SRC/openSUSE:Factory/benji/benji.changes 2020-09-07 21:35:54.473396100 +0200 +++ /work/SRC/openSUSE:Factory/.benji.new.4249/benji.changes 2020-09-22 21:11:37.283952489 +0200 @@ -1,0 +2,12 @@ +Mon Sep 21 20:59:41 UTC 2020 - Michael Vetter <[email protected]> + +- Update to 0.12.0: + * Fix type conversion bug in benji storage-usage command (issue #96) + +------------------------------------------------------------------- +Mon Sep 21 20:59:21 UTC 2020 - Michael Vetter <[email protected]> + +- Update to 0.11.0: + * Fix SQL generation for benji storage-usage command (issue #95) + +------------------------------------------------------------------- Old: ---- v0.10.0.tar.gz New: ---- v0.12.0.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ benji.spec ++++++ --- /var/tmp/diff_new_pack.08bHrI/_old 2020-09-22 21:11:38.175953271 +0200 +++ /var/tmp/diff_new_pack.08bHrI/_new 2020-09-22 21:11:38.187953281 +0200 @@ -17,7 +17,7 @@ Name: benji -Version: 0.10.0 +Version: 0.12.0 Release: 0 Summary: Deduplicating block based backup software License: LGPL-3.0-only ++++++ v0.10.0.tar.gz -> v0.12.0.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/benji-0.10.0/CHANGES.md new/benji-0.12.0/CHANGES.md --- old/benji-0.10.0/CHANGES.md 2020-09-05 13:36:45.000000000 +0200 +++ new/benji-0.12.0/CHANGES.md 2020-09-21 13:08:50.000000000 +0200 @@ -1,3 +1,11 @@ +## 0.12.0, 21.09.2020 + +* Fix type conversion bug in `benji storage-usage` command (issue #96) + +## 0.11.0, 18.09.2020 + +* Fix SQL generation for `benji storage-usage` command (issue #95) + ## 0.10.0, 05.09.2020 Notable changes: diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/benji-0.10.0/docs/source/configuration.rst new/benji-0.12.0/docs/source/configuration.rst --- old/benji-0.10.0/docs/source/configuration.rst 2020-09-05 13:36:45.000000000 +0200 +++ new/benji-0.12.0/docs/source/configuration.rst 2020-09-21 13:08:50.000000000 +0200 @@ -203,7 +203,7 @@ Number of reader threads when reading from a backup source. Also affects the internal read queue length. It is highly recommended to -increase this number to increase this number to get better concurrency +increase this number to get better concurrency and performance. * name: **simultaneousWrites** @@ -211,7 +211,7 @@ * default: ``1`` Number of writer threads when restoring a version. Also affects the internal write queue length. It is highly -recommended to increase this number to increase this number to get better concurrency and performance. +recommended to increase this number to get better concurrency and performance. I/O Module file ~~~~~~~~~~~~~~~~ @@ -224,15 +224,14 @@ Number of reader threads when reading from a backup source. Also affects the internal read queue length. It is highly recommended to -increase this number to increase this number to get better concurrency -and performance. +increase this number to get better concurrency and performance. * name: **simultaneousWrites** * type: integer * default: ``1`` Number of writer threads when restoring a version. Also affects the internal write queue length. It is highly -recommended to increase this number to increase this number to get better concurrency and performance. +recommended to increase this number to get better concurrency and performance. I/O Module rbd ~~~~~~~~~~~~~~ @@ -264,7 +263,7 @@ Number of reader threads when reading from a backup source. Also affects the internal read queue length. It is highly recommended to -increase this number to increase this number to get better concurrency +increase this number to get better concurrency and performance. * name: **simultaneousWrites** @@ -272,7 +271,7 @@ * default: ``1`` Number of writer threads when restoring a version. Also affects the internal write queue length. It is highly -recommended to increase this number to increase this number to get better concurrency and performance. +recommended to increase this number to get better concurrency and performance. * name: **cephConfigFile** * type: string @@ -485,8 +484,8 @@ from base64 import b64encode from Crypto.PublicKey import ECC key = ECC.generate(curve='NIST P-384') - public_key = b64encode(key.public_key().export_key(format='DER', compress=True)).decode('ascii')) - private_key = b64encode(key.export_key(format='DER', compress=True).decode('ascii')) + public_key = b64encode(key.public_key().export_key(format='DER', compress=True)).decode('ascii') + private_key = b64encode(key.export_key(format='DER', compress=True)).decode('ascii') Storage Modules @@ -499,21 +498,21 @@ * default: ``1`` Number of reader threads when reading from a storage. Also affects the internal read queue length. It is highly -recommended to increase this number to increase this number to get better concurrency and performance. +recommended to increase this number to get better concurrency and performance. * name: **simultaneousWrites** * type: integer * default: ``1`` Number of writer threads when writing to a storage. Also affects the internal write queue length. It is highly -recommended to increase this number to increase this number to get better concurrency and performance. +recommended to increase this number to get better concurrency and performance. * name: **simultaneousRemovals** * type: integer * default: ``1`` Number of removal threads when removing blocks from a storage. Also affects the internal queue length. It is highly -recommended to increase this number to increase this number to get better concurrency and performance. +recommended to increase this number to get better concurrency and performance. * name: **bandwidthRead** * type: integer diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/benji-0.10.0/setup.py new/benji-0.12.0/setup.py --- old/benji-0.10.0/setup.py 2020-09-05 13:36:45.000000000 +0200 +++ new/benji-0.12.0/setup.py 2020-09-21 13:08:50.000000000 +0200 @@ -30,6 +30,7 @@ Operating System :: POSIX Programming Language :: Python :: 3.6 Programming Language :: Python :: 3.7 +Programming Language :: Python :: 3.8 Topic :: System :: Archiving :: Backup """[:-1].split('\n'), keywords='backup', diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/benji-0.10.0/src/benji/_static_version.py new/benji-0.12.0/src/benji/_static_version.py --- old/benji-0.10.0/src/benji/_static_version.py 2020-09-05 13:36:45.000000000 +0200 +++ new/benji-0.12.0/src/benji/_static_version.py 2020-09-21 13:08:50.000000000 +0200 @@ -8,5 +8,5 @@ version = "__use_git__" # These values are only set if the distribution was created with 'git archive' -refnames = "HEAD -> master, tag: v0.10.0" -git_hash = "894738d" +refnames = "HEAD -> master, tag: v0.12.0" +git_hash = "636db46" diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/benji-0.10.0/src/benji/database.py new/benji-0.12.0/src/benji/database.py --- old/benji-0.10.0/src/benji/database.py 2020-09-05 13:36:45.000000000 +0200 +++ new/benji-0.12.0/src/benji/database.py 2020-09-21 13:08:50.000000000 +0200 @@ -610,7 +610,7 @@ virtual = virtual.filter(Version.id.in_(version_ids)) for _, storage_name, size in virtual.all(): - usage[storage_name]['virtual'] = size + usage[storage_name]['virtual'] = int(size) # func.sum()/SUM() returns type Decimal sq_block, sq_storage, sq_version = aliased(Block), aliased(Storage), aliased(Version) share_count_overall_sq = Session.query(func.count('*')).select_from(sq_block).join(sq_version).join( @@ -622,7 +622,8 @@ Storage.id.label('storage_id'), Storage.name.label('storage_name'), Block.uid, Block.size.label('size'), func.count('*').label('share_count_subset'), share_count_overall_sq.label('share_count_overall')).select_from(Block).join(Version).join(Storage).filter( - (Block.uid_left != None) & (Block.uid_right != None)).group_by(Storage.id, Storage.name, Block.uid) + (Block.uid_left != None) & (Block.uid_right != None)).group_by(Storage.id, Storage.name, Block.uid, + Block.size) if version_ids is not None: share_count_query = share_count_query.filter(Block.version_id.in_(version_ids)) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/benji-0.10.0/src/benji/schemas/v1/benji.transform.aes_256_gcm_ecc.yaml new/benji-0.12.0/src/benji/schemas/v1/benji.transform.aes_256_gcm_ecc.yaml --- old/benji-0.10.0/src/benji/schemas/v1/benji.transform.aes_256_gcm_ecc.yaml 1970-01-01 01:00:00.000000000 +0100 +++ new/benji-0.12.0/src/benji/schemas/v1/benji.transform.aes_256_gcm_ecc.yaml 2020-09-21 13:08:50.000000000 +0200 @@ -0,0 +1,15 @@ +configuration: + required: True + schema: + eccKey: + type: string + empty: False + required: True + eccCurve: + type: string + empty: False + default: NIST P-384 + allowed: + - NIST P-256 + - NIST P-384 + - NIST P-521 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/benji-0.10.0/src/benji/tests/test_database.py new/benji-0.12.0/src/benji/tests/test_database.py --- old/benji-0.10.0/src/benji/tests/test_database.py 2020-09-05 13:36:45.000000000 +0200 +++ new/benji-0.12.0/src/benji/tests/test_database.py 2020-09-21 13:08:50.000000000 +0200 @@ -425,6 +425,79 @@ block_size=4 * 1024 * 4096) self.assertEqual(math.ceil(version.size / version.block_size), version.blocks_count) + def test_storage_usage(self): + Storage.sync('s-1', storage_id=1) + for i in range(2): + version = Version.create(version_uid=VersionUid(f'v{i + 1}'), + volume='backup-name', + snapshot='snapshot-name.{}'.format(i), + size=32 * 1024 * 4096, + storage_id=1, + block_size=1024 * 4096) + + blocks: List[Dict[str, Any]] = [] + # shared + for idx in range(0, 6): + uid = BlockUid(1, idx) + blocks.append({ + 'idx': idx, + 'uid_left': uid.left, + 'uid_right': uid.right, + 'checksum': None, + 'size': 1024 * 4096, + 'valid': True + }) + # sparse + for idx in range(6, 13): + uid = BlockUid(i + 1, idx) + blocks.append({ + 'idx': idx, + 'uid_left': None, + 'uid_right': None, + 'checksum': None, + 'size': 1024 * 4096, + 'valid': True + }) + # exclusive + for idx in range(13, 25): + uid = BlockUid(i + 1, idx) + blocks.append({ + 'idx': idx, + 'uid_left': uid.left, + 'uid_right': uid.right, + 'checksum': None, + 'size': 1024 * 4096, + 'valid': True + }) + # exclusive deduplicated + for idx in range(25, 32): + uid = BlockUid(i + 1, idx - 7) + blocks.append({ + 'idx': idx, + 'uid_left': uid.left, + 'uid_right': uid.right, + 'checksum': None, + 'size': 1024 * 4096, + 'valid': True + }) + version.create_blocks(blocks=blocks) + + for uid in ('v1', 'v2'): + usage = Version.storage_usage(f'uid == "{uid}"') + self.assertIsInstance(usage.get('s-1', None), dict) + usage_s_1 = usage['s-1'] + check_value_matrix = ( + ('virtual', 32 * 1024 * 4096), + ('shared', 6 * 1024 * 4096), + ('sparse', 7 * 1024 * 4096), + ('exclusive', 19 * 1024 * 4096), + ('deduplicated_exclusive', (19 - 7) * 1024 * 4096), + ) + for field, amount in check_value_matrix: + value = usage_s_1.get(field, None) + self.assertIsInstance(value, int) + self.assertEqual(amount, value) + class DatabaseBackendTestSQLLite(DatabaseBackendTestCase, TestCase): diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/benji-0.10.0/tests/postgresql-setup/docker-compose.yaml new/benji-0.12.0/tests/postgresql-setup/docker-compose.yaml --- old/benji-0.10.0/tests/postgresql-setup/docker-compose.yaml 2020-09-05 13:36:45.000000000 +0200 +++ new/benji-0.12.0/tests/postgresql-setup/docker-compose.yaml 2020-09-21 13:08:50.000000000 +0200 @@ -3,7 +3,7 @@ services: db: - image: postgres:10 + image: postgres:12 restart: always environment: POSTGRES_USER: benji
