Hello community, here is the log from the commit of package python-toml for openSUSE:Factory checked in at 2020-06-10 00:34:37 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Comparing /work/SRC/openSUSE:Factory/python-toml (Old) and /work/SRC/openSUSE:Factory/.python-toml.new.3606 (New) ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
Package is "python-toml" Wed Jun 10 00:34:37 2020 rev:5 rq:812543 version:0.10.1 Changes: -------- --- /work/SRC/openSUSE:Factory/python-toml/python-toml.changes 2020-03-27 00:20:57.248124800 +0100 +++ /work/SRC/openSUSE:Factory/.python-toml.new.3606/python-toml.changes 2020-06-10 00:34:42.196800666 +0200 @@ -1,0 +2,6 @@ +Mon Jun 8 10:32:52 UTC 2020 - Ondřej Súkup <[email protected]> + +- update to 0.10.1 + * update testdata + +------------------------------------------------------------------- Old: ---- toml-0.10.0.tar.gz toml-test-39bb76d6.tar.gz New: ---- toml-0.10.1.tar.gz toml-test-280497f.tar.gz ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ Other differences: ------------------ ++++++ python-toml.spec ++++++ --- /var/tmp/diff_new_pack.5XiY1j/_old 2020-06-10 00:34:49.616820569 +0200 +++ /var/tmp/diff_new_pack.5XiY1j/_new 2020-06-10 00:34:49.620820580 +0200 @@ -18,16 +18,17 @@ %{?!python_module:%define python_module() python-%{**} python3-%{**}} Name: python-toml -Version: 0.10.0 +Version: 0.10.1 Release: 0 Summary: Python module which parses and emits TOML License: MIT URL: https://github.com/uiri/toml Source: https://files.pythonhosted.org/packages/source/t/toml/toml-%{version}.tar.gz # Untagged test data https://github.com/uiri/toml/issues/232 -Source1: https://github.com/BurntSushi/toml-test/archive/39bb76d6.tar.gz#/toml-test-39bb76d6.tar.gz +Source1: https://github.com/BurntSushi/toml-test/archive/280497f.tar.gz#/toml-test-280497f.tar.gz # Missing file https://github.com/uiri/toml/pull/231 Source2: https://raw.githubusercontent.com/uiri/toml/%{version}/test.toml +BuildRequires: %{python_module numpy} BuildRequires: %{python_module pytest} BuildRequires: %{python_module setuptools} BuildRequires: coreutils ++++++ test.toml ++++++ --- /var/tmp/diff_new_pack.5XiY1j/_old 2020-06-10 00:34:49.656820676 +0200 +++ /var/tmp/diff_new_pack.5XiY1j/_new 2020-06-10 00:34:49.660820687 +0200 @@ -40,6 +40,12 @@ "omega" ] +[meeting] +[meeting.inspace] +time = 10:00:00 +[meeting.nospace] +time=10:00:00 + [[fruit]] name = "apple" ++++++ toml-0.10.0.tar.gz -> toml-0.10.1.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/LICENSE new/toml-0.10.1/LICENSE --- old/toml-0.10.0/LICENSE 2018-09-15 05:27:12.000000000 +0200 +++ new/toml-0.10.1/LICENSE 2020-05-14 05:53:07.000000000 +0200 @@ -1,11 +1,12 @@ The MIT License -Copyright 2013-2018 William Pearson +Copyright 2013-2019 William Pearson Copyright 2015-2016 Julien Enselme Copyright 2016 Google Inc. Copyright 2017 Samuel Vasko Copyright 2017 Nate Prewitt Copyright 2017 Jack Evans +Copyright 2019 Filippo Broggini Permission is hereby granted, free of charge, to any person obtaining a copy of this software and associated documentation files (the "Software"), to deal diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/MANIFEST.in new/toml-0.10.1/MANIFEST.in --- old/toml-0.10.0/MANIFEST.in 2018-10-04 04:04:17.000000000 +0200 +++ new/toml-0.10.1/MANIFEST.in 2019-10-18 03:54:44.000000000 +0200 @@ -2,4 +2,5 @@ include README.rst include toml.pyi include tox.ini +include test.toml recursive-include tests *.py *.sh diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/PKG-INFO new/toml-0.10.1/PKG-INFO --- old/toml-0.10.0/PKG-INFO 2018-10-04 04:32:10.000000000 +0200 +++ new/toml-0.10.1/PKG-INFO 2020-05-14 05:55:37.000000000 +0200 @@ -1,12 +1,11 @@ Metadata-Version: 1.1 Name: toml -Version: 0.10.0 +Version: 0.10.1 Summary: Python Library for Tom's Obvious, Minimal Language Home-page: https://github.com/uiri/toml Author: William Pearson Author-email: [email protected] License: MIT -Description-Content-Type: UNKNOWN Description: **** TOML **** @@ -123,6 +122,22 @@ For more functions, view the API Reference below. + Note + ---- + + For Numpy users, by default the data types ``np.floatX`` will not be translated to floats by toml, but will instead be encoded as strings. To get around this, specify the ``TomlNumpyEncoder`` when saving your data. + + .. code:: pycon + + >>> import toml + >>> import numpy as np + >>> a = np.arange(0, 10, dtype=np.double) + >>> output = {'a': a} + >>> toml.dumps(output) + 'a = [ "0.0", "1.0", "2.0", "3.0", "4.0", "5.0", "6.0", "7.0", "8.0", "9.0",]\n' + >>> toml.dumps(output, encoder=toml.TomlNumpyEncoder()) + 'a = [ 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0,]\n' + API Reference ============= @@ -157,12 +172,13 @@ * ``TomlDecodeError``: When an error occurs while decoding the TOML-formatted string - ``toml.dump(o, f)`` + ``toml.dump(o, f, encoder=None)`` Write a dictionary to a file containing TOML-formatted data :Args: * ``o``: An object to be converted into TOML * ``f``: A File descriptor where the TOML-formatted output should be stored + * ``encoder``: An instance of ``TomlEncoder`` (or subclass) for encoding the object. If ``None``, will default to ``TomlEncoder`` :Returns: A string containing the TOML-formatted data corresponding to object ``o`` @@ -170,15 +186,18 @@ :Raises: * ``TypeError``: When anything other than file descriptor is passed - ``toml.dumps(o)`` + ``toml.dumps(o, encoder=None)`` Create a TOML-formatted string from an input object :Args: * ``o``: An object to be converted into TOML + * ``encoder``: An instance of ``TomlEncoder`` (or subclass) for encoding the object. If ``None``, will default to ``TomlEncoder`` :Returns: A string containing the TOML-formatted data corresponding to object ``o`` + + Licensing ========= @@ -200,5 +219,6 @@ Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 +Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/README.rst new/toml-0.10.1/README.rst --- old/toml-0.10.0/README.rst 2018-10-04 04:04:18.000000000 +0200 +++ new/toml-0.10.1/README.rst 2019-10-18 03:54:44.000000000 +0200 @@ -114,6 +114,22 @@ For more functions, view the API Reference below. +Note +---- + +For Numpy users, by default the data types ``np.floatX`` will not be translated to floats by toml, but will instead be encoded as strings. To get around this, specify the ``TomlNumpyEncoder`` when saving your data. + +.. code:: pycon + + >>> import toml + >>> import numpy as np + >>> a = np.arange(0, 10, dtype=np.double) + >>> output = {'a': a} + >>> toml.dumps(output) + 'a = [ "0.0", "1.0", "2.0", "3.0", "4.0", "5.0", "6.0", "7.0", "8.0", "9.0",]\n' + >>> toml.dumps(output, encoder=toml.TomlNumpyEncoder()) + 'a = [ 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0,]\n' + API Reference ============= @@ -148,12 +164,13 @@ * ``TomlDecodeError``: When an error occurs while decoding the TOML-formatted string -``toml.dump(o, f)`` +``toml.dump(o, f, encoder=None)`` Write a dictionary to a file containing TOML-formatted data :Args: * ``o``: An object to be converted into TOML * ``f``: A File descriptor where the TOML-formatted output should be stored + * ``encoder``: An instance of ``TomlEncoder`` (or subclass) for encoding the object. If ``None``, will default to ``TomlEncoder`` :Returns: A string containing the TOML-formatted data corresponding to object ``o`` @@ -161,15 +178,18 @@ :Raises: * ``TypeError``: When anything other than file descriptor is passed -``toml.dumps(o)`` +``toml.dumps(o, encoder=None)`` Create a TOML-formatted string from an input object :Args: * ``o``: An object to be converted into TOML + * ``encoder``: An instance of ``TomlEncoder`` (or subclass) for encoding the object. If ``None``, will default to ``TomlEncoder`` :Returns: A string containing the TOML-formatted data corresponding to object ``o`` + + Licensing ========= diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/setup.cfg new/toml-0.10.1/setup.cfg --- old/toml-0.10.0/setup.cfg 2018-10-04 04:32:10.000000000 +0200 +++ new/toml-0.10.1/setup.cfg 2020-05-14 05:55:37.000000000 +0200 @@ -5,6 +5,7 @@ license_file = LICENSE [egg_info] -tag_build = +tag_svn_revision = 0 tag_date = 0 +tag_build = diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/setup.py new/toml-0.10.1/setup.py --- old/toml-0.10.0/setup.py 2018-10-04 03:56:21.000000000 +0200 +++ new/toml-0.10.1/setup.py 2020-01-03 21:37:49.000000000 +0100 @@ -18,6 +18,7 @@ packages=['toml'], license="MIT", long_description=readme_string, + python_requires=">=2.6, !=3.0.*, !=3.1.*, !=3.2.*", classifiers=[ 'Development Status :: 5 - Production/Stable', 'Intended Audience :: Developers', @@ -33,6 +34,7 @@ 'Programming Language :: Python :: 3.5', 'Programming Language :: Python :: 3.6', 'Programming Language :: Python :: 3.7', + 'Programming Language :: Python :: 3.8', 'Programming Language :: Python :: Implementation :: CPython', 'Programming Language :: Python :: Implementation :: PyPy', ] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/test.toml new/toml-0.10.1/test.toml --- old/toml-0.10.0/test.toml 1970-01-01 01:00:00.000000000 +0100 +++ new/toml-0.10.1/test.toml 2020-01-03 21:13:50.000000000 +0100 @@ -0,0 +1,71 @@ +# This is a TOML document. Boom. + +a = "C:\\Users\\n" + +title = "TOML Example" +the-void = [[[[[]]]]] +mixed = [[1, +2], ["a", "b"], [1.0, 2.0]] +avogadro = 6.23e23 + +[owner] +name = "Tom \\ / Preston-Werner" +organization = "GitHub" +bio = "GitHub Cofounder & CEO\nLikes tater tots and beer." +dob = 1979-05-27T07:32:00Z # First class dates? Why not? + +[database] +server = "192.168.1.1" +ports = [ 8001, 8001, 8002 ] +connection_max = 5000 +enabled = true +test = [["a"], ["b"], ["c"]] + +[servers] + + # You can indent as you please. Tabs or spaces. TOML don't care. + [servers.alpha] + ip = "10.0.0.1" + dc = "eqdc10" + + [servers.beta] + ip = "\u000a\u1000\u1000\u0002" + dc = "eqdc10" + +[clients] +data = [ ["gamma", "delta"], [1, 2] ] # just an update to make sure parsers support it + +# Line breaks are OK when inside arrays +hosts = [ + "alpha", + "omega" +] + +[meeting] +[meeting.inspace] +time = 10:00:00 +[meeting.nospace] +time=10:00:00 + +[[fruit]] + name = "apple" + + [fruit.physical] + color = "red" + shape = "round" + + [[fruit.variety]] + name = "red delicious" + + #[fruit.variety] + # name = "granny smith" + +[[fruit]] + name = "banana" + + [[fruit.variety]] + name = "plantain" + + [[fruit.variety]] + name = "whatever" + thing=34 + "l.mao" = 42 diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/tests/decoding_test.sh new/toml-0.10.1/tests/decoding_test.sh --- old/toml-0.10.0/tests/decoding_test.sh 2017-08-28 03:19:47.000000000 +0200 +++ new/toml-0.10.1/tests/decoding_test.sh 2019-10-18 03:54:44.000000000 +0200 @@ -1,3 +1,4 @@ #!/bin/sh +export PYTHONPATH=`pwd` python tests/decoding_test.py diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/tests/test_api.py new/toml-0.10.1/tests/test_api.py --- old/toml-0.10.0/tests/test_api.py 2018-10-04 03:56:21.000000000 +0200 +++ new/toml-0.10.1/tests/test_api.py 2020-05-14 05:36:35.000000000 +0200 @@ -3,6 +3,7 @@ import pytest import os import sys +from decimal import Decimal from toml.decoder import InlineTableDict @@ -48,7 +49,21 @@ for f in os.listdir(valid_dir): if not f.endswith("toml"): continue - toml.dumps(toml.load(open(os.path.join(valid_dir, f)))) + with open(os.path.join(valid_dir, f)) as fh: + toml.dumps(toml.load(fh)) + + +def test_circular_ref(): + a = {} + b = {} + b['c'] = 4 + b['self'] = b + a['b'] = b + with pytest.raises(ValueError): + toml.dumps(a) + + with pytest.raises(ValueError): + toml.dumps(b) def test__dict(): @@ -87,6 +102,40 @@ assert o == toml.loads(toml.dumps(o, encoder=encoder)) +def test_numpy_floats(): + import numpy as np + + encoder = toml.TomlNumpyEncoder() + d = {'a': np.array([1, .3], dtype=np.float64)} + o = toml.loads(toml.dumps(d, encoder=encoder)) + assert o == toml.loads(toml.dumps(o, encoder=encoder)) + + d = {'a': np.array([1, .3], dtype=np.float32)} + o = toml.loads(toml.dumps(d, encoder=encoder)) + assert o == toml.loads(toml.dumps(o, encoder=encoder)) + + d = {'a': np.array([1, .3], dtype=np.float16)} + o = toml.loads(toml.dumps(d, encoder=encoder)) + assert o == toml.loads(toml.dumps(o, encoder=encoder)) + + +def test_numpy_ints(): + import numpy as np + + encoder = toml.TomlNumpyEncoder() + d = {'a': np.array([1, 3], dtype=np.int64)} + o = toml.loads(toml.dumps(d, encoder=encoder)) + assert o == toml.loads(toml.dumps(o, encoder=encoder)) + + d = {'a': np.array([1, 3], dtype=np.int32)} + o = toml.loads(toml.dumps(d, encoder=encoder)) + assert o == toml.loads(toml.dumps(o, encoder=encoder)) + + d = {'a': np.array([1, 3], dtype=np.int16)} + o = toml.loads(toml.dumps(d, encoder=encoder)) + assert o == toml.loads(toml.dumps(o, encoder=encoder)) + + def test_ordered(): from toml import ordered as toml_ordered encoder = toml_ordered.TomlOrderedEncoder() @@ -102,13 +151,23 @@ assert o == toml.loads(toml.dumps(o)) +def test_decimal(): + PLACES = Decimal(10) ** -4 + + d = {"a": Decimal("0.1")} + o = toml.loads(toml.dumps(d)) + assert o == toml.loads(toml.dumps(o)) + assert Decimal(o["a"]).quantize(PLACES) == d["a"].quantize(PLACES) + + def test_invalid_tests(): invalid_dir = "toml-test/tests/invalid/" for f in os.listdir(invalid_dir): if not f.endswith("toml"): continue with pytest.raises(toml.TomlDecodeError): - toml.load(open(os.path.join(invalid_dir, f))) + with open(os.path.join(invalid_dir, f)) as fh: + toml.load(fh) def test_exceptions(): @@ -119,12 +178,12 @@ toml.load(2) try: - FileNotFoundError + FNFError = FileNotFoundError except NameError: # py2 - FileNotFoundError = IOError + FNFError = IOError - with pytest.raises(FileNotFoundError): + with pytest.raises(FNFError): toml.load([]) @@ -142,17 +201,19 @@ def test_dump(): + from collections import OrderedDict f = FakeFile() g = FakeFile() h = FakeFile() toml.dump(TEST_DICT, f) - toml.dump(toml.load(f), g) - toml.dump(toml.load(g), h) + toml.dump(toml.load(f, _dict=OrderedDict), g) + toml.dump(toml.load(g, _dict=OrderedDict), h) assert g.written == h.written def test_paths(): toml.load("test.toml") + toml.load(b"test.toml") import sys if (3, 4) <= sys.version_info: import pathlib @@ -169,3 +230,44 @@ def test_commutativity(): o = toml.loads(toml.dumps(TEST_DICT)) assert o == toml.loads(toml.dumps(o)) + + +def test_pathlib(): + if (3, 4) <= sys.version_info: + import pathlib + o = {"root": {"path": pathlib.Path("/home/edgy")}} + test_str = """[root] +path = "/home/edgy" +""" + assert test_str == toml.dumps(o, encoder=toml.TomlPathlibEncoder()) + + +def test_comment_preserve_decoder_encoder(): + test_str = """[[products]] +name = "Nail" +sku = 284758393 +# This is a comment +color = "gray" # Hello World +# name = { first = 'Tom', last = 'Preston-Werner' } +# arr7 = [ +# 1, 2, 3 +# ] +# lines = ''' +# The first newline is +# trimmed in raw strings. +# All other whitespace +# is preserved. +# ''' + +[animals] +color = "gray" # col +fruits = "apple" # a = [1,2,3] +a = 3 +b-comment = "a is 3" +""" + + s = toml.dumps(toml.loads(test_str, + decoder=toml.TomlPreserveCommentDecoder()), + encoder=toml.TomlPreserveCommentEncoder()) + + assert len(s) == len(test_str) and sorted(test_str) == sorted(s) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/toml/__init__.py new/toml-0.10.1/toml/__init__.py --- old/toml-0.10.0/toml/__init__.py 2018-10-04 03:56:21.000000000 +0200 +++ new/toml-0.10.1/toml/__init__.py 2020-05-14 05:47:29.000000000 +0200 @@ -6,16 +6,20 @@ from toml import encoder from toml import decoder -__version__ = "0.10.0" +__version__ = "0.10.1" _spec_ = "0.5.0" load = decoder.load loads = decoder.loads TomlDecoder = decoder.TomlDecoder TomlDecodeError = decoder.TomlDecodeError +TomlPreserveCommentDecoder = decoder.TomlPreserveCommentDecoder dump = encoder.dump dumps = encoder.dumps TomlEncoder = encoder.TomlEncoder TomlArraySeparatorEncoder = encoder.TomlArraySeparatorEncoder TomlPreserveInlineDictEncoder = encoder.TomlPreserveInlineDictEncoder +TomlNumpyEncoder = encoder.TomlNumpyEncoder +TomlPreserveCommentEncoder = encoder.TomlPreserveCommentEncoder +TomlPathlibEncoder = encoder.TomlPathlibEncoder diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/toml/common.py new/toml-0.10.1/toml/common.py --- old/toml-0.10.0/toml/common.py 1970-01-01 01:00:00.000000000 +0100 +++ new/toml-0.10.1/toml/common.py 2018-10-27 07:09:05.000000000 +0200 @@ -0,0 +1,6 @@ +# content after the \ +escapes = ['0', 'b', 'f', 'n', 'r', 't', '"'] +# What it should be replaced by +escapedchars = ['\0', '\b', '\f', '\n', '\r', '\t', '\"'] +# Used for substitution +escape_to_escapedchars = dict(zip(_escapes, _escapedchars)) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/toml/decoder.py new/toml-0.10.1/toml/decoder.py --- old/toml-0.10.0/toml/decoder.py 2018-10-04 03:56:21.000000000 +0200 +++ new/toml-0.10.1/toml/decoder.py 2020-05-14 05:36:35.000000000 +0200 @@ -24,7 +24,7 @@ def _ispath(p): - if isinstance(p, basestring): + if isinstance(p, (bytes, basestring)): return True return _detect_pathlib_path(p) @@ -44,7 +44,7 @@ FNFError = IOError -TIME_RE = re.compile("([0-9]{2}):([0-9]{2}):([0-9]{2})(\.([0-9]{3,6}))?") +TIME_RE = re.compile(r"([0-9]{2}):([0-9]{2}):([0-9]{2})(\.([0-9]{3,6}))?") class TomlDecodeError(ValueError): @@ -66,6 +66,27 @@ _number_with_underscores = re.compile('([0-9])(_([0-9]))*') +class CommentValue(object): + def __init__(self, val, comment, beginline, _dict): + self.val = val + separator = "\n" if beginline else " " + self.comment = separator + comment + self._dict = _dict + + def __getitem__(self, key): + return self.val[key] + + def __setitem__(self, key, value): + self.val[key] = value + + def dump(self, dump_value_func): + retstr = dump_value_func(self.val) + if isinstance(self.val, self._dict): + return self.comment + "\n" + unicode(retstr) + else: + return unicode(retstr) + self.comment + + def _strictly_valid_num(n): n = n.strip() if not n: @@ -96,6 +117,7 @@ f: Path to the file to open, array of files to read into single dict or a file descriptor _dict: (optional) Specifies the class of the returned toml dictionary + decoder: The decoder to use Returns: Parsed toml file represented as a dictionary @@ -120,9 +142,9 @@ "existing file.") raise FNFError(error_msg) if decoder is None: - decoder = TomlDecoder() + decoder = TomlDecoder(_dict) d = decoder.get_empty_table() - for l in f: + for l in f: # noqa: E741 if op.exists(l): d.update(load(l, _dict, decoder)) else: @@ -177,19 +199,30 @@ keygroup = False dottedkey = False keyname = 0 + key = '' + prev_key = '' + line_no = 1 + for i, item in enumerate(sl): if item == '\r' and sl[i + 1] == '\n': sl[i] = ' ' continue if keyname: + key += item if item == '\n': raise TomlDecodeError("Key name found without value." " Reached end of line.", original, i) if openstring: if item == openstrchar: - keyname = 2 - openstring = False - openstrchar = "" + oddbackslash = False + k = 1 + while i >= k and sl[i - k] == '\\': + oddbackslash = not oddbackslash + k += 1 + if not oddbackslash: + keyname = 2 + openstring = False + openstrchar = "" continue elif keyname == 1: if item.isspace(): @@ -220,6 +253,8 @@ continue if item == '=': keyname = 0 + prev_key = key[:-1].rstrip() + key = '' dottedkey = False else: raise TomlDecodeError("Found invalid character in key name: '" + @@ -272,12 +307,16 @@ if item == '#' and (not openstring and not keygroup and not arrayoftables): j = i + comment = "" try: while sl[j] != '\n': + comment += s[j] sl[j] = ' ' j += 1 except IndexError: break + if not openarr: + decoder.preserve_comment(line_no, prev_key, comment, beginline) if item == '[' and (not openstring and not keygroup and not arrayoftables): if beginline: @@ -308,12 +347,20 @@ sl[i] = ' ' else: beginline = True + line_no += 1 elif beginline and sl[i] != ' ' and sl[i] != '\t': beginline = False if not keygroup and not arrayoftables: if sl[i] == '=': raise TomlDecodeError("Found empty keyname. ", original, i) keyname = 1 + key += item + if keyname: + raise TomlDecodeError("Key name found without value." + " Reached end of file.", original, len(s)) + if openstring: # reached EOF and have an unterminated string + raise TomlDecodeError("Unterminated string found." + " Reached end of file.", original, len(s)) s = ''.join(sl) s = s.split('\n') multikey = None @@ -323,6 +370,9 @@ for idx, line in enumerate(s): if idx > 0: pos += len(s[idx - 1]) + 1 + + decoder.embed_comments(idx, currentlevel) + if not multilinestr or multibackslash or '\n' not in multilinestr: line = line.strip() if line == "" and (not multikey or multibackslash): @@ -333,9 +383,14 @@ else: multilinestr += line multibackslash = False - if len(line) > 2 and (line[-1] == multilinestr[0] and - line[-2] == multilinestr[0] and - line[-3] == multilinestr[0]): + closed = False + if multilinestr[0] == '[': + closed = line[-1] == ']' + elif len(line) > 2: + closed = (line[-1] == multilinestr[0] and + line[-2] == multilinestr[0] and + line[-3] == multilinestr[0]) + if closed: try: value, vtype = decoder.load_value(multilinestr) except ValueError as err: @@ -663,7 +718,8 @@ while len(pair[-1]) and (pair[-1][0] != ' ' and pair[-1][0] != '\t' and pair[-1][0] != "'" and pair[-1][0] != '"' and pair[-1][0] != '[' and pair[-1][0] != '{' and - pair[-1] != 'true' and pair[-1] != 'false'): + pair[-1].strip() != 'true' and + pair[-1].strip() != 'false'): try: float(pair[-1]) break @@ -671,6 +727,8 @@ pass if _load_date(pair[-1]) is not None: break + if TIME_RE.match(pair[-1]): + break i += 1 prev_val = pair[-1] pair = line.split('=', i) @@ -704,16 +762,10 @@ pair[0] = levels[-1].strip() elif (pair[0][0] == '"' or pair[0][0] == "'") and \ (pair[0][-1] == pair[0][0]): - pair[0] = pair[0][1:-1] - if len(pair[1]) > 2 and ((pair[1][0] == '"' or pair[1][0] == "'") and - pair[1][1] == pair[1][0] and - pair[1][2] == pair[1][0] and - not (len(pair[1]) > 5 and - pair[1][-1] == pair[1][0] and - pair[1][-2] == pair[1][0] and - pair[1][-3] == pair[1][0])): - k = len(pair[1]) - 1 - while k > -1 and pair[1][k] == '\\': + pair[0] = _unescape(pair[0][1:-1]) + k, koffset = self._load_line_multiline_str(pair[1]) + if k > -1: + while k > -1 and pair[1][k + koffset] == '\\': multibackslash = not multibackslash k -= 1 if multibackslash: @@ -734,6 +786,26 @@ else: currentlevel[pair[0]] = value + def _load_line_multiline_str(self, p): + poffset = 0 + if len(p) < 3: + return -1, poffset + if p[0] == '[' and (p.strip()[-1] != ']' and + self._load_array_isstrarray(p)): + newp = p[1:].strip().split(',') + while len(newp) > 1 and newp[-1][0] != '"' and newp[-1][0] != "'": + newp = newp[:-2] + [newp[-2] + ',' + newp[-1]] + newp = newp[-1] + poffset = len(p) - len(newp) + p = newp + if p[0] != '"' and p[0] != "'": + return -1, poffset + if p[1] != p[0] or p[2] != p[0]: + return -1, poffset + if len(p) > 5 and p[-1] == p[0] and p[-2] == p[0] and p[-3] == p[0]: + return -1, poffset + return len(p) - 1, poffset + def load_value(self, v, strictly_valid=True): if not v: raise ValueError("Empty value is invalid") @@ -769,7 +841,8 @@ pass if not oddbackslash: if closed: - raise ValueError("Stuff after closed string. WTF?") + raise ValueError("Found tokens after a closed " + + "string. Invalid TOML.") else: if not triplequote or triplequotecount > 1: closed = True @@ -857,15 +930,18 @@ break return not backslash + def _load_array_isstrarray(self, a): + a = a[1:-1].strip() + if a != '' and (a[0] == '"' or a[0] == "'"): + return True + return False + def load_array(self, a): atype = None retval = [] a = a.strip() if '[' not in a[1:-1] or "" != a[1:-1].split('[')[0].strip(): - strarray = False - tmpa = a[1:-1].strip() - if tmpa != '' and (tmpa[0] == '"' or tmpa[0] == "'"): - strarray = True + strarray = self._load_array_isstrarray(a) if not a[1:-1].strip().startswith('{'): a = a[1:-1].split(',') else: @@ -874,6 +950,7 @@ new_a = [] start_group_index = 1 end_group_index = 2 + open_bracket_count = 1 if a[start_group_index] == '{' else 0 in_str = False while end_group_index < len(a[1:]): if a[end_group_index] == '"' or a[end_group_index] == "'": @@ -884,9 +961,15 @@ in_str = not in_str backslash_index -= 1 in_str = not in_str + if not in_str and a[end_group_index] == '{': + open_bracket_count += 1 if in_str or a[end_group_index] != '}': end_group_index += 1 continue + elif a[end_group_index] == '}' and open_bracket_count > 1: + open_bracket_count -= 1 + end_group_index += 1 + continue # Increase end_group_index by 1 to get the closing bracket end_group_index += 1 @@ -943,3 +1026,27 @@ atype = ntype retval.append(nval) return retval + + def preserve_comment(self, line_no, key, comment, beginline): + pass + + def embed_comments(self, idx, currentlevel): + pass + + +class TomlPreserveCommentDecoder(TomlDecoder): + + def __init__(self, _dict=dict): + self.saved_comments = {} + super(TomlPreserveCommentDecoder, self).__init__(_dict) + + def preserve_comment(self, line_no, key, comment, beginline): + self.saved_comments[line_no] = (key, comment, beginline) + + def embed_comments(self, idx, currentlevel): + if idx not in self.saved_comments: + return + + key, comment, beginline = self.saved_comments[idx] + currentlevel[key] = CommentValue(currentlevel[key], comment, beginline, + self._dict) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/toml/encoder.py new/toml-0.10.1/toml/encoder.py --- old/toml-0.10.0/toml/encoder.py 2018-10-04 04:00:16.000000000 +0200 +++ new/toml-0.10.1/toml/encoder.py 2020-01-03 21:13:50.000000000 +0100 @@ -1,6 +1,7 @@ import datetime import re import sys +from decimal import Decimal from toml.decoder import InlineTableDict @@ -8,12 +9,13 @@ unicode = str -def dump(o, f): +def dump(o, f, encoder=None): """Writes out dict as toml to a file Args: o: Object to dump into toml f: File descriptor where the toml should be stored + encoder: The ``TomlEncoder`` to use for constructing the output string Returns: String containing the toml corresponding to dictionary @@ -24,7 +26,7 @@ if not f.write: raise TypeError("You can only dump an object to a file descriptor") - d = dumps(o) + d = dumps(o, encoder=encoder) f.write(d) return d @@ -34,11 +36,22 @@ Args: o: Object to dump into toml - - preserve: Boolean parameter. If true, preserve inline tables. + encoder: The ``TomlEncoder`` to use for constructing the output string Returns: String containing the toml corresponding to dict + + Examples: + ```python + >>> import toml + >>> output = { + ... 'a': "I'm a string", + ... 'b': ["I'm", "a", "list"], + ... 'c': 2400 + ... } + >>> toml.dumps(output) + 'a = "I\'m a string"\nb = [ "I\'m", "a", "list",]\nc = 2400\n' + ``` """ retval = "" @@ -46,7 +59,13 @@ encoder = TomlEncoder(o.__class__) addtoretval, sections = encoder.dump_sections(o, "") retval += addtoretval + outer_objs = [id(o)] while sections: + section_ids = [id(section) for section in sections] + for outer_obj in outer_objs: + if outer_obj in section_ids: + raise ValueError("Circular reference detected") + outer_objs += section_ids newsections = encoder.get_empty_table() for section in sections: addtoretval, addtosections = encoder.dump_sections( @@ -96,7 +115,7 @@ def _dump_float(v): - return "{0:.16}".format(v).replace("e+0", "e+").replace("e-0", "e-") + return "{}".format(v).replace("e+0", "e+").replace("e-0", "e-") def _dump_time(v): @@ -119,6 +138,7 @@ bool: lambda v: unicode(v).lower(), int: lambda v: v, float: _dump_float, + Decimal: _dump_float, datetime.datetime: lambda v: v.isoformat().replace('+00:00', 'Z'), datetime.time: _dump_time, datetime.date: lambda v: v.isoformat() @@ -169,10 +189,7 @@ section = unicode(section) qsection = section if not re.match(r'^[A-Za-z0-9_-]+$', section): - if '"' in section: - qsection = "'" + section + "'" - else: - qsection = '"' + section + '"' + qsection = _dump_str(section) if not isinstance(o[section], dict): arrayoftables = False if isinstance(o[section], list): @@ -248,3 +265,40 @@ t = s retval += "]" return retval + + +class TomlNumpyEncoder(TomlEncoder): + + def __init__(self, _dict=dict, preserve=False): + import numpy as np + super(TomlNumpyEncoder, self).__init__(_dict, preserve) + self.dump_funcs[np.float16] = _dump_float + self.dump_funcs[np.float32] = _dump_float + self.dump_funcs[np.float64] = _dump_float + self.dump_funcs[np.int16] = self._dump_int + self.dump_funcs[np.int32] = self._dump_int + self.dump_funcs[np.int64] = self._dump_int + + def _dump_int(self, v): + return "{}".format(int(v)) + + +class TomlPreserveCommentEncoder(TomlEncoder): + + def __init__(self, _dict=dict, preserve=False): + from toml.decoder import CommentValue + super(TomlPreserveCommentEncoder, self).__init__(_dict, preserve) + self.dump_funcs[CommentValue] = lambda v: v.dump(self.dump_value) + + +class TomlPathlibEncoder(TomlEncoder): + + def _dump_pathlib_path(self, v): + return _dump_str(str(v)) + + def dump_value(self, v): + if (3, 4) <= sys.version_info: + import pathlib + if isinstance(v, pathlib.PurePath): + v = str(v) + return super(TomlPathlibEncoder, self).dump_value(v) diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/toml.egg-info/PKG-INFO new/toml-0.10.1/toml.egg-info/PKG-INFO --- old/toml-0.10.0/toml.egg-info/PKG-INFO 2018-10-04 04:32:10.000000000 +0200 +++ new/toml-0.10.1/toml.egg-info/PKG-INFO 2020-05-14 05:55:35.000000000 +0200 @@ -1,12 +1,11 @@ Metadata-Version: 1.1 Name: toml -Version: 0.10.0 +Version: 0.10.1 Summary: Python Library for Tom's Obvious, Minimal Language Home-page: https://github.com/uiri/toml Author: William Pearson Author-email: [email protected] License: MIT -Description-Content-Type: UNKNOWN Description: **** TOML **** @@ -123,6 +122,22 @@ For more functions, view the API Reference below. + Note + ---- + + For Numpy users, by default the data types ``np.floatX`` will not be translated to floats by toml, but will instead be encoded as strings. To get around this, specify the ``TomlNumpyEncoder`` when saving your data. + + .. code:: pycon + + >>> import toml + >>> import numpy as np + >>> a = np.arange(0, 10, dtype=np.double) + >>> output = {'a': a} + >>> toml.dumps(output) + 'a = [ "0.0", "1.0", "2.0", "3.0", "4.0", "5.0", "6.0", "7.0", "8.0", "9.0",]\n' + >>> toml.dumps(output, encoder=toml.TomlNumpyEncoder()) + 'a = [ 0.0, 1.0, 2.0, 3.0, 4.0, 5.0, 6.0, 7.0, 8.0, 9.0,]\n' + API Reference ============= @@ -157,12 +172,13 @@ * ``TomlDecodeError``: When an error occurs while decoding the TOML-formatted string - ``toml.dump(o, f)`` + ``toml.dump(o, f, encoder=None)`` Write a dictionary to a file containing TOML-formatted data :Args: * ``o``: An object to be converted into TOML * ``f``: A File descriptor where the TOML-formatted output should be stored + * ``encoder``: An instance of ``TomlEncoder`` (or subclass) for encoding the object. If ``None``, will default to ``TomlEncoder`` :Returns: A string containing the TOML-formatted data corresponding to object ``o`` @@ -170,15 +186,18 @@ :Raises: * ``TypeError``: When anything other than file descriptor is passed - ``toml.dumps(o)`` + ``toml.dumps(o, encoder=None)`` Create a TOML-formatted string from an input object :Args: * ``o``: An object to be converted into TOML + * ``encoder``: An instance of ``TomlEncoder`` (or subclass) for encoding the object. If ``None``, will default to ``TomlEncoder`` :Returns: A string containing the TOML-formatted data corresponding to object ``o`` + + Licensing ========= @@ -200,5 +219,6 @@ Classifier: Programming Language :: Python :: 3.5 Classifier: Programming Language :: Python :: 3.6 Classifier: Programming Language :: Python :: 3.7 +Classifier: Programming Language :: Python :: 3.8 Classifier: Programming Language :: Python :: Implementation :: CPython Classifier: Programming Language :: Python :: Implementation :: PyPy diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/toml.egg-info/SOURCES.txt new/toml-0.10.1/toml.egg-info/SOURCES.txt --- old/toml-0.10.0/toml.egg-info/SOURCES.txt 2018-10-04 04:32:10.000000000 +0200 +++ new/toml-0.10.1/toml.egg-info/SOURCES.txt 2020-05-14 05:55:37.000000000 +0200 @@ -3,6 +3,7 @@ README.rst setup.cfg setup.py +test.toml toml.pyi tox.ini tests/__init__.py @@ -12,6 +13,7 @@ tests/decoding_test3.sh tests/test_api.py toml/__init__.py +toml/common.py toml/decoder.py toml/encoder.py toml/ordered.py diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/toml.pyi new/toml-0.10.1/toml.pyi --- old/toml-0.10.0/toml.pyi 2017-08-28 03:19:47.000000000 +0200 +++ new/toml-0.10.1/toml.pyi 2019-10-18 03:54:44.000000000 +0200 @@ -7,6 +7,14 @@ class TomlTz(datetime.tzinfo): def __init__(self, toml_offset: str) -> None: ... +class TomlDecoder(object):... + +class TomlEncoder(object):... + +class TomlPreserveCommentDecoder(TomlDecoder):... + +class TomlPreserveCommentEncoder(TomlEncoder):... + def load(f: Union[str, list, IO[str]], _dict: Type[MutableMapping[str, Any]] = ...) \ -> MutableMapping[str, Any]: ... diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-0.10.0/tox.ini new/toml-0.10.1/tox.ini --- old/toml-0.10.0/tox.ini 2018-07-14 19:07:51.000000000 +0200 +++ new/toml-0.10.1/tox.ini 2020-01-03 21:13:50.000000000 +0100 @@ -1,10 +1,11 @@ [tox] -envlist = py27, py33, py34, py35, py36, pypy +envlist = py27, py33, py34, py35, py36, py37, py38, pypy [testenv] deps = pytest pytest-cov + numpy commands=pytest tests {posargs} [testenv:check] ++++++ toml-test-39bb76d6.tar.gz -> toml-test-280497f.tar.gz ++++++ diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/README.md new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/README.md --- old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/README.md 2018-09-08 22:59:39.000000000 +0200 +++ new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/README.md 2020-05-14 05:18:01.000000000 +0200 @@ -17,7 +17,7 @@ Version: v0.4.0 (in sync with TOML) Compatible with TOML version -[v0.4.0](https://github.com/mojombo/toml/blob/master/versions/toml-v0.4.0.md) +[v0.4.0](https://github.com/toml-lang/toml/blob/v0.4.0/versions/en/toml-v0.4.0.md) Dependencies: [Go](http://golang.org). diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/invalid/multi-line-string-no-close.toml new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/invalid/multi-line-string-no-close.toml --- old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/invalid/multi-line-string-no-close.toml 1970-01-01 01:00:00.000000000 +0100 +++ new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/invalid/multi-line-string-no-close.toml 2020-05-14 05:18:01.000000000 +0200 @@ -0,0 +1,2 @@ +invalid = """ + this will fail diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/valid/nested-inline-table-array.json new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/valid/nested-inline-table-array.json --- old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/valid/nested-inline-table-array.json 1970-01-01 01:00:00.000000000 +0100 +++ new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/valid/nested-inline-table-array.json 2020-05-14 05:18:01.000000000 +0200 @@ -0,0 +1,7 @@ +{ + "a": [ + { + "b": {} + } + ] +} diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/valid/nested-inline-table-array.toml new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/valid/nested-inline-table-array.toml --- old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/valid/nested-inline-table-array.toml 1970-01-01 01:00:00.000000000 +0100 +++ new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/valid/nested-inline-table-array.toml 2020-05-14 05:18:01.000000000 +0200 @@ -0,0 +1 @@ +a = [ { b = {} } ] diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/valid/right-curly-brace-after-boolean.json new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/valid/right-curly-brace-after-boolean.json --- old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/valid/right-curly-brace-after-boolean.json 1970-01-01 01:00:00.000000000 +0100 +++ new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/valid/right-curly-brace-after-boolean.json 2020-05-14 05:18:01.000000000 +0200 @@ -0,0 +1,16 @@ +{ + "black":{ + "allow_prereleases":{ + "type":"bool", + "value":"true" + }, + "python":{ + "type":"string", + "value":">3.6" + }, + "version":{ + "type":"string", + "value":">=18.9b0" + } + } + } \ No newline at end of file diff -urN '--exclude=CVS' '--exclude=.cvsignore' '--exclude=.svn' '--exclude=.svnignore' old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/valid/right-curly-brace-after-boolean.toml new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/valid/right-curly-brace-after-boolean.toml --- old/toml-test-39bb76d631ba103a94b377aaf52c979456677fb1/tests/valid/right-curly-brace-after-boolean.toml 1970-01-01 01:00:00.000000000 +0100 +++ new/toml-test-280497fa5f12e43d7233aed0d74e07ca61ef176b/tests/valid/right-curly-brace-after-boolean.toml 2020-05-14 05:18:01.000000000 +0200 @@ -0,0 +1 @@ +black = { python=">3.6", version=">=18.9b0", allow_prereleases=true }
