Patch updated.
In article <CAEZNtZ+SKqiATLwDvzyVf=sulpght7usxhzeawnfextpwsb...@mail.gmail.com>
troycurti...@gmail.com writes:
On Thu, Jan 10, 2019 at 9:50 AM Yasuhito FUTATSUKI
<futat...@yf.bsdclub.org> wrote:
The patch attached modifies 4 kind of input argment translations.
(1) typemap(in) char * (with/without const modifiers); not allow NULL,
typemap(in) const char * MAY_BE_NULL; allows NULL
These had done by using 'parse=' typemap modifier, however there is no
PyArg_Parse() format unit to convert both of str and bytes in py3.
So I make a function svn_swig_py_string_to_cstring() in swigutil_py.c,
and use it in for new typemap definition.
consideration:
* For py2, my patch code uses svn_swig_py_string_to_cstring()
- It isn't allow Unicode for input, however 's' and 'z' format units
PyArg_Parse() Unicode in py2. If it is need to accept Unicode in py2,
it is need to fix. (use svn_swig_py_string_to_cstring() py3 only, or
add code to conversion for py2)
Yes I think you should support Unicode in this case, but it turns out
you are most of the way there. If you just remove the IS_PY3
conditional, it will support unicode! The "PyBytes_*" and "PyStr_*"
functions are wrappers provided by the py3c library. The names point
to the concept that they target, and then map it appropriately in Py2
and Py3. So
PyBytes: Sequence of byte values, e.g. "raw data"
In Py2: str
In Py3: bytes
PyStr: Character data
In Py2: Unicode
In Py3: str
Now it uses PyUnicode_Check() and PyStr_UTF8(), then it works in py2 and py3
without IS_PY3 conditional here.
- Difference of TypeError exception message. Pyrg_Parse() reports
argment by argnum in Python wrapper function. However it can't
determin in typemap definition code, so my patch code uses argment
symbol instead.
This is still left, if we treat it as a regression.
* For py3, it seems to need more kindly Exception message for
UnicodeEncodeError, which can be caused if input str contains surrogate
data (U+DC00 - U+DCFF).
I reconsidered it and abandoned, because of UnicodeEncodeError structure.
It is enought to analyse why it causes.
* test case for UnicodeEncodeError is needed
Added.
(2) in core.i, typemap for svn_stream_write
consideration:
* As this typemap doesn't seems to accept Unicode on py2, there seems to
be no regression like described in (1)
* As this typemap only used for svn_stream_write wrapper which have only
one char * arg, default UnicodeEncodeError message seems to be sufficient.
Not changed in this patch.
(3) typemap(in) apr_hash_t * (for various types)
These are using make_string_from_ob() for hash key string conversion,
and typemap(in) apr_hash_t *HASH_CSTRING uses it for hash value
conversion, too. Similarly typemap(in) apr_hash_t *PROPHASH uses
make_svn_strinf_from_ob() for hash value conversion
consideration:
* It seems some of API (e.g. svn_prop_diffs()) allows NULL for hash value,
but current implementation of conversion function doesn't allows. Isn't
it needed? (I added test for this case, but disabled until it make clear)
* test case for UnicodeEncodeError is needed (for both of hash key and
hash value)
Yes this looks like a good catch, the generic conversion function
should handle the NULL/None case.
I've fixed it. To fix this, check key and value separately in
svn_swig_py_*_from_dict().
Test case for UnicodeEncodeError is added, and it detect
make_svn_strinf_from_ob()
couldn't handle this case :) Yes, I've fixed it, of course.
(4) typemap(in) apr_array_header_t *STRINGLIST
This typemap is using svn_swig_py_unwrap_string() through the function
svn_swig_py_seq_to_array().
consideration:
* test case for UnicodeEncodeError is needed (for both of hash key and
hash value)
Test case for UnicodeEncodeError is added in tests/client.py.
And here is the start of my patch review:
I've intended to fix problems pointed out, except the comment for
svn_swig_py_string_to_cstring(). For svn_swig_py_string_to_cstring(),
I've added a comment for the reason of return type and why it seems to
sufficient not to duplicate string entity.
Thanks,
--
Yasuhito FUTATSUKI
On branch swig-py3: accept both of bytes/str for input char * arguments
* Replace typemap(in) for char * using 'parse' modifier with one using
function allows both of bytes/str in py2/py3 in libsvn_swig_py library
* Fix functions to convert Python objects into char pointer to accept
both of bytes/str object in py2/py3
* Fix to accept None as representation of NULL value on conversion
prop dict into apr_hash_t * and apr_array_header_t * of svn_props_t *
[In subversion/bindings/swig]
* core.i (%typemap(in) (const char *data, apr_size_t *len): Allow str
as well as bytes for data argment of svn_stream_write()
* include/svn_global.swg
(remove)(%typemap(in) char *, char const *, char * const,
char const * const): Move this typemap into include/svn_strings as
typemap (in) IN_STRING
* include/svn_string.swg (new)(%typemap(in) IN_STRING): replacement of
%typemap(in) char *, char const *, char * const, char const * const.
actual processing code is moved new svn_swig_py_string_to_cstring()
function in python/libsvn_swig_py/swigutil_py.c
* include/svn_types.swg (%typemap(in) const char *MAY_BE_NULL):
Move processing code into new svn_swig_py_string_to_cstring() function
in python/libsvn_swig_py/swigutil_py.c
* python/libsvn_swig_py/swigutil_py.c
(svn_swig_py_string_to_cstring): New function to convert Python
bytes or str into const char *, with better TypeError exception message
(svn_swig_py_string_type_exception): New function to construct
TypeError exception for new make_string_from_ob_maybe_null() function
(make_string_from_ob, make_svn_string_from_ob):
- Allow str as well as bytes for ob
- Don't raise TypeError exception because all callers don't expect it
(make_string_from_ob_maybe_null, make_svn_string_from_ob_maybe_null):
New function same as make_string_from_ob() and make_svn_string_fromob()
but allows None input represents NULL value and raise TypeError
if input value don't have appropriate type
(svn_swig_py_stringhash_from_dict, svn_swig_py_mergeinfo_from_dict,
svn_swig_py_proparray_from_dict, svn_swig_py_prophash_from_dict,
svn_swig_py_path_rev_hash_from_dict,
svn_swig_py_struct_ptr_hash_from_dict): separate check of key conversion
result and value conversion result
(svn_swig_py_proparray_from_dict, svn_swig_py_prophash_from_dict):
allow NULL for prop values
(svn_swig_py_unwrap_string):
- Allow str as well as bytes for source argument
(svn_swig_py_make_file):
- Allow str as well as bytes for py_file argument as file path
(svn_swig_py_auth_gnome_keyring_unlock_prompt_func):
- Use new function make_string_from_ob_maybe_null() instead of
make_string_from_ob() to check TypeError
- Report Python exception caused by Python callback function as
callback exception error
* python/libsvn_swig_py/swigutil_py.h
Expose new public function make_string_from_ob_maybe_null(), which is
used by typemap(in) char IN_STRING, typemap(in) const char *MAY_BY_NULL
* python/tests/client.py
(SubversionClientTestCase.log_entry_receiver_whole): New helper
callback function for new SubversionClientTestCase.test_log5_revprops
(SubversionClientTestCase.test_log5_revprops): new test for
typemap(in) apr_array_t *STRINGLIST and its helper function
svn_swig_py_unwrap_string()
* python/tests/core.py
(SubversionCoreTestCase.test_stream_write_exception):
- As unicode input is now valid, use int value as invalid input
- (Only for Python 3) Denote that input including surrogate causes
UnicodeEncodeError
(SubversionCoreTestCase.test_stream_write_str):(Only for Python 3)
New test case for svn_stream_write() to pass Str object as data argument
(SubversionCoreTestCase.test_stream_write_bytes):
Renamed from SubversionCoreTestCase.test_stream_write
* python/tests/run_all.py: Register new test module typemap
* python/tests/typemap.py: New unittest module for typemaps
(SubversionTypemapTestCase): New unittest subclass for unit test about
typemaps
(SubversionTypemapTestCase.test_char_ptr_in): New test case
(SubversionTypemapTestCase.test_char_ptr_in_unicode_exception):
New test case(Only for Python3)
(SubversionTypemapTestCase.test_char_ptr_may_be_null): New test case
(SubversionTypemapTestCase.test_char_ptr_may_be_null_unicode_exception):
New test case(Only for Python3)
(SubversionTypemapTestCase.test_make_string_from_ob): New test case
(SubversionTypemapTestCase.test_prophash_from_dict_null_value):
New test case
(SubversionTypemapTestCase.test_make-string_ob_unicode_exception):
New test case(Only for Python3)
(SubversionTypemapTestCase.test_make_svn_string_ob_unicode_exception):
New test case(Only for Python3)
(suite): New function to drive SubversionTypemapTestCase
Review by: troycurtisjr (anticipated)
diff --git a/subversion/bindings/swig/core.i b/subversion/bindings/swig/core.i
index c309d48070..bb94713154 100644
--- a/subversion/bindings/swig/core.i
+++ b/subversion/bindings/swig/core.i
@@ -442,18 +442,24 @@
*/
#ifdef SWIGPYTHON
%typemap(in) (const char *data, apr_size_t *len) ($*2_type temp) {
- char *tmpdata;
Py_ssize_t length;
- if (!PyBytes_Check($input)) {
- PyErr_SetString(PyExc_TypeError,
- "expecting a bytes object for the buffer");
- SWIG_fail;
+ if (PyBytes_Check($input)) {
+ if (PyBytes_AsStringAndSize($input, (char **)&$1, &length) == -1) {
+ SWIG_fail;
+ }
}
- if (PyBytes_AsStringAndSize($input, &tmpdata, &length) == -1) {
+ else if (PyUnicode_Check($input)) {
+ $1 = (char *)PyStr_AsUTF8AndSize($input, &length);
+ if (PyErr_Occurred()) {
+ SWIG_fail;
+ }
+ }
+ else {
+ PyErr_SetString(PyExc_TypeError,
+ "expecting a bytes or str object for the buffer");
SWIG_fail;
}
temp = ($*2_type)length;
- $1 = tmpdata;
$2 = ($2_ltype)&temp;
}
#endif
diff --git a/subversion/bindings/swig/include/svn_global.swg
b/subversion/bindings/swig/include/svn_global.swg
index f7364be28f..57b1e236db 100644
--- a/subversion/bindings/swig/include/svn_global.swg
+++ b/subversion/bindings/swig/include/svn_global.swg
@@ -142,13 +142,6 @@ static PyObject * _global_py_pool = NULL;
/* Python format specifiers. Use Python instead of SWIG to parse
these basic types, because Python reports better error messages
(with correct argument numbers). */
-#if defined(PY3)
-%typemap (in, parse="y")
- char *, char const *, char * const, char const * const "";
-#else
-%typemap (in, parse="s")
- char *, char const *, char * const, char const * const "";
-#endif
%typemap (in, parse="c") char "";
%typemap (in, fragment=SWIG_As_frag(long)) long
diff --git a/subversion/bindings/swig/include/svn_string.swg
b/subversion/bindings/swig/include/svn_string.swg
index 0fc64ebdcc..8be4c3d746 100644
--- a/subversion/bindings/swig/include/svn_string.swg
+++ b/subversion/bindings/swig/include/svn_string.swg
@@ -251,6 +251,26 @@ typedef struct svn_string_t svn_string_t;
}
#endif
+ /* -----------------------------------------------------------------------
+ Type: char * (input)
+*/
+#ifdef SWIGPYTHON
+%typemap (in) IN_STRING
+{
+ $1 = svn_swig_py_string_to_cstring($input, FALSE, "$symname", "$1_name");
+ if (PyErr_Occurred()) SWIG_fail;
+}
+
+%typemap (freearg) IN_STRING "";
+
+%apply IN_STRING {
+ const char *,
+ char *,
+ char const *,
+ char * const,
+ char const * const
+};
+#endif
/* -----------------------------------------------------------------------
define a way to return a 'const char *'
*/
diff --git a/subversion/bindings/swig/include/svn_types.swg
b/subversion/bindings/swig/include/svn_types.swg
index 319f7daa6b..7c933b1ac7 100644
--- a/subversion/bindings/swig/include/svn_types.swg
+++ b/subversion/bindings/swig/include/svn_types.swg
@@ -348,12 +348,8 @@ svn_ ## TYPE ## _swig_rb_closed(VALUE self)
#ifdef SWIGPYTHON
%typemap(in) const char *MAY_BE_NULL
{
- if ($input == Py_None) {
- $1 = NULL;
- } else {
- $1 = PyBytes_AsString($input);
- if ($1 == NULL) SWIG_fail;
- }
+ $1 = svn_swig_py_string_to_cstring($input, TRUE, "$symname", "$1_name");
+ if (PyErr_Occurred()) SWIG_fail;
}
#endif
diff --git a/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c
b/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c
index 8a4ec631be..e661bca95b 100644
--- a/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c
+++ b/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.c
@@ -506,6 +506,36 @@ void svn_swig_py_svn_exception(svn_error_t *error_chain)
/*** Helper/Conversion Routines ***/
+/* Function to get char * representation of bytes/str object. This is
+ the replacement of typemap(in, parse="s") and typemap(in, parse="z")
+ to accept both of bytes object and str object, and it assumes to be
+ used from those typemaps only.
+ Note: type of return value should be char const *, however, as SWIG
+ produces variables for C function without 'const' modifier, to avoid
+ ton of cast in SWIG produced C code we drop it from return value
+ types as well */
+char *svn_swig_py_string_to_cstring(PyObject *input, int maybe_null,
+ const char * funcsym, const char * argsym)
+{
+ char *retval = NULL;
+ if (PyBytes_Check(input))
+ {
+ retval = PyBytes_AsString(input);
+ }
+ else if (PyUnicode_Check(input))
+ {
+ retval = (char *)PyStr_AsUTF8(input);
+ }
+ else if (input != Py_None || ! maybe_null)
+ {
+ PyErr_Format(PyExc_TypeError,
+ "%s() argument %s must be bytes or str%s, not %s",
+ funcsym, argsym, maybe_null?" or None":"",
+ Py_TYPE(input)->tp_name);
+ }
+ return retval;
+}
+
/* Functions for making Python wrappers around Subversion structs */
static PyObject *make_ob_pool(void *pool)
{
@@ -540,32 +570,89 @@ static PyObject *make_ob_error(svn_error_t *err)
/***/
+static void svn_swig_py_string_type_exception(int maybe_null)
+{
+ PyErr_Format(PyExc_TypeError, "not a bytes or a str%s",
+ maybe_null?" or None":"");
+}
+
/* Conversion from Python single objects (not hashes/lists/etc.) to
Subversion types. */
static char *make_string_from_ob(PyObject *ob, apr_pool_t *pool)
{
+ /* caller should not expect to raise TypeError: check return value
+ whether it is NULL or not, if needed */
+ if (PyBytes_Check(ob))
+ {
+ return apr_pstrdup(pool, PyBytes_AsString(ob));
+ }
+ if (PyUnicode_Check(ob))
+ {
+ /* PyStr_AsUTF8() may cause UnicodeEncodeError,
+ but apr_pstrdup() allows NULL for s */
+ return apr_pstrdup(pool, PyStr_AsUTF8(ob));
+ }
+ return NULL;
+}
+
+static char *make_string_from_ob_maybe_null(PyObject *ob, apr_pool_t *pool)
+{
+ char * retval;
if (ob == Py_None)
- return NULL;
- if (! PyBytes_Check(ob))
{
- PyErr_SetString(PyExc_TypeError, "not a bytes object");
return NULL;
}
- return apr_pstrdup(pool, PyBytes_AsString(ob));
+ retval = make_string_from_ob(ob, pool);
+ if (!retval)
+ {
+ if (!PyErr_Occurred())
+ {
+ svn_swig_py_string_type_exception(TRUE);
+ }
+ }
+ return retval;
}
+
static svn_string_t *make_svn_string_from_ob(PyObject *ob, apr_pool_t *pool)
{
+ /* caller should not expect to raise TypeError: check return value
+ whether it is NULL or not, if needed */
+ if (PyBytes_Check(ob))
+ {
+ return svn_string_create(PyBytes_AsString(ob), pool);
+ }
+ if (PyUnicode_Check(ob))
+ {
+ /* PyStr_AsUTF8() may cause UnicodeEncodeError,
+ and svn_string_create() does not allows NULL for cstring */
+ const char *obstr = PyStr_AsUTF8(ob);
+ if (obstr)
+ {
+ return svn_string_create(obstr, pool);
+ }
+ }
+ return NULL;
+}
+
+static svn_string_t *make_svn_string_from_ob_maybe_null(PyObject *ob,
+ apr_pool_t *pool)
+{
+ svn_string_t * retval;
if (ob == Py_None)
- return NULL;
- if (! PyBytes_Check(ob))
{
- PyErr_SetString(PyExc_TypeError, "not a bytes object");
return NULL;
}
- return svn_string_create(PyBytes_AsString(ob), pool);
+ retval = make_svn_string_from_ob(ob, pool);
+ if (!retval)
+ {
+ if (!PyErr_Occurred())
+ {
+ svn_swig_py_string_type_exception(TRUE);
+ }
+ }
+ return retval;
}
-
/***/
static PyObject *convert_hash(apr_hash_t *hash,
@@ -1055,6 +1142,9 @@ PyObject
*svn_swig_py_changed_path2_hash_to_dict(apr_hash_t *hash)
return dict;
}
+#define TYPE_ERROR_DICT_STRING_KEY \
+ "dictionary keys aren't bytes or str objects"
+
apr_hash_t *svn_swig_py_stringhash_from_dict(PyObject *dict,
apr_pool_t *pool)
{
@@ -1079,11 +1169,18 @@ apr_hash_t *svn_swig_py_stringhash_from_dict(PyObject
*dict,
PyObject *key = PyList_GetItem(keys, i);
PyObject *value = PyDict_GetItem(dict, key);
const char *propname = make_string_from_ob(key, pool);
- const char *propval = make_string_from_ob(value, pool);
- if (! (propname && propval))
+ if (!propname)
+ {
+ if (!PyErr_Occurred())
+ {
+ PyErr_SetString(PyExc_TypeError, TYPE_ERROR_DICT_STRING_KEY);
+ }
+ Py_DECREF(keys);
+ return NULL;
+ }
+ const char *propval = make_string_from_ob_maybe_null(value, pool);
+ if (PyErr_Occurred())
{
- PyErr_SetString(PyExc_TypeError,
- "dictionary keys/values aren't strings");
Py_DECREF(keys);
return NULL;
}
@@ -1116,6 +1213,15 @@ apr_hash_t *svn_swig_py_mergeinfo_from_dict(PyObject
*dict,
PyObject *key = PyList_GetItem(keys, i);
PyObject *value = PyDict_GetItem(dict, key);
const char *pathname = make_string_from_ob(key, pool);
+ if (!pathname)
+ {
+ if (!PyErr_Occurred())
+ {
+ PyErr_SetString(PyExc_TypeError, TYPE_ERROR_DICT_STRING_KEY);
+ }
+ Py_DECREF(keys);
+ return NULL;
+ }
const svn_rangelist_t *ranges = svn_swig_py_seq_to_array(value,
sizeof(const svn_merge_range_t *),
svn_swig_py_unwrap_struct_ptr,
@@ -1123,10 +1229,10 @@ apr_hash_t *svn_swig_py_mergeinfo_from_dict(PyObject
*dict,
pool
);
- if (! (pathname && ranges))
+ if (!ranges)
{
PyErr_SetString(PyExc_TypeError,
- "dictionary keys aren't strings or values aren't
svn_merge_range_t *'s");
+ "dictionary values aren't svn_merge_range_t *'s");
Py_DECREF(keys);
return NULL;
}
@@ -1161,11 +1267,18 @@ apr_array_header_t
*svn_swig_py_proparray_from_dict(PyObject *dict,
PyObject *value = PyDict_GetItem(dict, key);
svn_prop_t *prop = apr_palloc(pool, sizeof(*prop));
prop->name = make_string_from_ob(key, pool);
- prop->value = make_svn_string_from_ob(value, pool);
- if (! (prop->name && prop->value))
+ if (! prop->name)
+ {
+ if (!PyErr_Occurred())
+ {
+ PyErr_SetString(PyExc_TypeError, TYPE_ERROR_DICT_STRING_KEY);
+ }
+ Py_DECREF(keys);
+ return NULL;
+ }
+ prop->value = make_svn_string_from_ob_maybe_null(value, pool);
+ if (PyErr_Occurred())
{
- PyErr_SetString(PyExc_TypeError,
- "dictionary keys/values aren't strings");
Py_DECREF(keys);
return NULL;
}
@@ -1199,11 +1312,18 @@ apr_hash_t *svn_swig_py_prophash_from_dict(PyObject
*dict,
PyObject *key = PyList_GetItem(keys, i);
PyObject *value = PyDict_GetItem(dict, key);
const char *propname = make_string_from_ob(key, pool);
- svn_string_t *propval = make_svn_string_from_ob(value, pool);
- if (! (propname && propval))
+ if (!propname)
+ {
+ if (!PyErr_Occurred())
+ {
+ PyErr_SetString(PyExc_TypeError, TYPE_ERROR_DICT_STRING_KEY);
+ }
+ Py_DECREF(keys);
+ return NULL;
+ }
+ svn_string_t *propval = make_svn_string_from_ob_maybe_null(value, pool);
+ if (PyErr_Occurred())
{
- PyErr_SetString(PyExc_TypeError,
- "dictionary keys/values aren't strings");
Py_DECREF(keys);
return NULL;
}
@@ -1241,8 +1361,10 @@ apr_hash_t
*svn_swig_py_path_revs_hash_from_dict(PyObject *dict,
if (!(path))
{
- PyErr_SetString(PyExc_TypeError,
- "dictionary keys aren't strings");
+ if (!PyErr_Occurred())
+ {
+ PyErr_SetString(PyExc_TypeError, TYPE_ERROR_DICT_STRING_KEY);
+ }
Py_DECREF(keys);
return NULL;
}
@@ -1296,8 +1418,10 @@ apr_hash_t
*svn_swig_py_struct_ptr_hash_from_dict(PyObject *dict,
if (!c_key)
{
- PyErr_SetString(PyExc_TypeError,
- "dictionary keys aren't strings");
+ if (!PyErr_Occurred())
+ {
+ PyErr_SetString(PyExc_TypeError, TYPE_ERROR_DICT_STRING_KEY);
+ }
Py_DECREF(keys);
return NULL;
}
@@ -1321,8 +1445,21 @@ svn_swig_py_unwrap_string(PyObject *source,
void *baton)
{
const char **ptr_dest = destination;
- *ptr_dest = PyBytes_AsString(source);
-
+ if (PyBytes_Check(source))
+ {
+ *ptr_dest = PyBytes_AsString(source);
+ }
+ else if (PyUnicode_Check(source))
+ {
+ *ptr_dest = PyStr_AsUTF8(source);
+ }
+ else
+ {
+ PyErr_Format(PyExc_TypeError,
+ "Expected bytes or str object, %s found",
+ Py_TYPE(source)->tp_name);
+ *ptr_dest = NULL;
+ }
if (*ptr_dest != NULL)
return 0;
else
@@ -1494,7 +1631,7 @@ commit_item_array_to_list(const apr_array_header_t *array)
}
-
+
/*** Errors ***/
/* Convert a given SubversionException to an svn_error_t. On failure returns
@@ -2546,14 +2683,25 @@ apr_file_t *svn_swig_py_make_file(PyObject *py_file,
{
apr_file_t *apr_file = NULL;
apr_status_t apr_err;
+ const char* fname = NULL;
if (py_file == NULL || py_file == Py_None)
return NULL;
+ /* check if input is a path */
if (PyBytes_Check(py_file))
+ {
+ fname = PyBytes_AsString(py_file);
+ }
+#if IS_PY3
+ else if (PyStr_Check(py_file))
+ {
+ fname = PyStr_AsUTF8(py_file);
+ }
+#endif
+ if (fname)
{
/* input is a path -- just open an apr_file_t */
- const char* fname = PyBytes_AsString(py_file);
apr_err = apr_file_open(&apr_file, fname,
APR_CREATE | APR_READ | APR_WRITE,
APR_OS_DEFAULT, pool);
@@ -3659,7 +3807,11 @@ svn_swig_py_auth_gnome_keyring_unlock_prompt_func(char
**keyring_passwd,
}
else
{
- *keyring_passwd = make_string_from_ob(result, pool);
+ *keyring_passwd = make_string_from_ob_maybe_null(result, pool);
+ if (PyErr_Occurred())
+ {
+ err = callback_exception_error();
+ }
Py_DECREF(result);
}
diff --git a/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.h
b/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.h
index 89e7c53c28..149ed81ff5 100644
--- a/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.h
+++ b/subversion/bindings/swig/python/libsvn_swig_py/swigutil_py.h
@@ -107,6 +107,17 @@ void svn_swig_py_svn_exception(svn_error_t *err);
+/* Function to get char * representation of bytes/str object. This is
+ the replacement of typemap(in, parse="s") and typemap(in, parse="z")
+ to accept both of bytes object and str object, and it assumes to be
+ used from those typemaps only.
+ Note: type of return value should be char const *, however, as SWIG
+ produces variables for C function without 'const' modifier, to avoid
+ ton of cast in SWIG produced C code we drop it from return value
+ types as well */
+char *svn_swig_py_string_to_cstring(PyObject *input, int maybe_null,
+ const char * funcsym, const char * argsym);
+
/* helper function to convert an apr_hash_t* (char* -> svnstring_t*) to
a Python dict */
PyObject *svn_swig_py_prophash_to_dict(apr_hash_t *hash);
diff --git a/subversion/bindings/swig/python/tests/client.py
b/subversion/bindings/swig/python/tests/client.py
index 88f7fcf5e4..cd9444cc7f 100644
--- a/subversion/bindings/swig/python/tests/client.py
+++ b/subversion/bindings/swig/python/tests/client.py
@@ -52,6 +52,10 @@ class SubversionClientTestCase(unittest.TestCase):
"""An implementation of svn_log_entry_receiver_t."""
self.received_revisions.append(log_entry.revision)
+ def log_entry_receiver_whole(self, log_entry, pool):
+ """An implementation of svn_log_entry_receiver_t, holds whole log
entries."""
+ self.received_log_entries.append(log_entry)
+
def setUp(self):
"""Set up authentication and client context"""
self.client_ctx = client.svn_client_create_context()
@@ -243,6 +247,33 @@ class SubversionClientTestCase(unittest.TestCase):
self.assertEqual(self.received_revisions, list(range(0, 5)))
+ def test_log5_revprops(self):
+ """Test svn_client_log5 revprops (for typemap(in) apr_array_t
*STRINGLIST)"""
+ directory = urljoin(self.repos_uri+b"/", b"trunk/dir1")
+ start = core.svn_opt_revision_t()
+ end = core.svn_opt_revision_t()
+ core.svn_opt_parse_revision(start, end, b"4:0")
+ rev_range = core.svn_opt_revision_range_t()
+ rev_range.start = start
+ rev_range.end = end
+
+ self.received_log_entries = []
+
+ # (Python 3: pass tuple of bytes and str mixture as revprops argment)
+ client.log5((directory,), start, (rev_range,), 1, True, False, False,
+ ('svn:author', b'svn:log'),
+ self.log_entry_receiver_whole, self.client_ctx)
+ self.assertEqual(len(self.received_log_entries), 1)
+ revprops = self.received_log_entries[0].revprops
+ self.assertEqual(revprops[b'svn:log'], b"More directories.")
+ self.assertEqual(revprops[b'svn:author'], b"john")
+ with self.assertRaises(KeyError):
+ commit_date = revprops['svn:date']
+ with self.assertRaises(UnicodeEncodeError):
+ client.log5((directory,), start, (rev_range,), 1, True, False, False,
+ (u'svn:\udc61uthor', b'svn:log'),
+ self.log_entry_receiver_whole, self.client_ctx)
+
def test_uuid_from_url(self):
"""Test svn_client_uuid_from_url on a file:// URL"""
self.assertTrue(isinstance(
diff --git a/subversion/bindings/swig/python/tests/core.py
b/subversion/bindings/swig/python/tests/core.py
index 6370fa5e1e..9bd82d6535 100644
--- a/subversion/bindings/swig/python/tests/core.py
+++ b/subversion/bindings/swig/python/tests/core.py
@@ -21,6 +21,9 @@
import unittest
import os
import tempfile
+import sys
+
+IS_PY3 = sys.version_info[0] >= 3
import svn.core, svn.client
import utils
@@ -220,13 +223,52 @@ class SubversionCoreTestCase(unittest.TestCase):
svn.core.svn_stream_close(stream)
def test_stream_write_exception(self):
- ostr_unicode = b'Python'.decode()
stream = svn.core.svn_stream_empty()
with self.assertRaises(TypeError):
- svn.core.svn_stream_write(stream, ostr_unicode)
+ svn.core.svn_stream_write(stream, 16)
+ # Check UnicodeEncodeError which can be caused only in Python 3
+ if IS_PY3:
+ o1_str = b'Python\x00\xa4\xd1\xa4\xa4\xa4\xbd\xa4\xf3\r\n'
+ ostr_unicode = o1_str.decode('ascii', 'surrogateescape')
+ with self.assertRaises(UnicodeEncodeError):
+ svn.core.svn_stream_write(stream, ostr_unicode)
svn.core.svn_stream_close(stream)
- def test_stream_write(self):
+ @unittest.skipUnless(IS_PY3, "test for Python 3 only")
+ def test_stream_write_str(self):
+ o1_str = 'Python\x00\xa4\xd1\xa4\xa4\xa4\xbd\xa4\xf3\r\n'
+ o2_str = ('subVersioN\x00'
+ '\xa4\xb5\xa4\xd6\xa4\xd0\xa1\xbc\xa4\xb8\xa4\xe7\xa4\xf3\n')
+ o3_str = 'swig\x00\xa4\xb9\xa4\xa6\xa4\xa3\xa4\xb0\rend'
+ out_str = o1_str + o2_str + o3_str
+ rewrite_str = 'Subversion'
+ fd, fname = tempfile.mkstemp()
+ os.close(fd)
+ try:
+ stream = svn.core.svn_stream_from_aprfile2(fname, False)
+ self.assertEqual(svn.core.svn_stream_write(stream, out_str),
+ len(out_str.encode('UTF-8')))
+ svn.core.svn_stream_seek(stream, None)
+ self.assertEqual(svn.core.svn_stream_read_full(stream, 4096),
+ out_str.encode('UTF-8'))
+ svn.core.svn_stream_seek(stream, None)
+ svn.core.svn_stream_skip(stream, len(o1_str.encode('UTF-8')))
+ self.assertEqual(svn.core.svn_stream_write(stream, rewrite_str),
+ len(rewrite_str.encode('UTF-8')))
+ svn.core.svn_stream_seek(stream, None)
+ self.assertEqual(
+ svn.core.svn_stream_read_full(stream, 4096),
+ (o1_str + rewrite_str
+ + o2_str[len(rewrite_str.encode('UTF-8')):]
+ + o3_str ).encode('UTF-8'))
+ svn.core.svn_stream_close(stream)
+ finally:
+ try:
+ os.remove(fname)
+ except OSError:
+ pass
+
+ def test_stream_write_bytes(self):
o1_str = b'Python\x00\xa4\xd1\xa4\xa4\xa4\xbd\xa4\xf3\r\n'
o2_str = (b'subVersioN\x00'
b'\xa4\xb5\xa4\xd6\xa4\xd0\xa1\xbc\xa4\xb8\xa4\xe7\xa4\xf3\n')
diff --git a/subversion/bindings/swig/python/tests/run_all.py
b/subversion/bindings/swig/python/tests/run_all.py
index 5cfd9d7536..3a042e012c 100644
--- a/subversion/bindings/swig/python/tests/run_all.py
+++ b/subversion/bindings/swig/python/tests/run_all.py
@@ -21,7 +21,7 @@
import sys
import unittest, setup_path
import mergeinfo, core, client, delta, checksum, pool, fs, ra, wc, repository,
\
- auth, trac.versioncontrol.tests
+ auth, trac.versioncontrol.tests, typemap
from svn.core import svn_cache_config_get, svn_cache_config_set
# Run all tests
@@ -47,6 +47,7 @@ def suite():
s.addTest(repository.suite())
s.addTest(auth.suite())
s.addTest(trac.versioncontrol.tests.suite())
+ s.addTest(typemap.suite())
return s
if __name__ == '__main__':
diff --git a/subversion/bindings/swig/python/tests/typemap.py
b/subversion/bindings/swig/python/tests/typemap.py
new file mode 100644
index 0000000000..7f6e839176
--- /dev/null
+++ b/subversion/bindings/swig/python/tests/typemap.py
@@ -0,0 +1,133 @@
+#
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+#
+import unittest
+import os
+import tempfile
+
+import svn.core
+
+class SubversionTypemapTestCase(unittest.TestCase):
+ """Test cases for the SWIG typemaps argments and return values transration"""
+
+ def test_char_ptr_in(self):
+ """Check %typemap(in) IN_STRING works correctly"""
+ self.assertEqual(svn.core.svn_path_canonicalize(b'foo'), b'foo')
+ self.assertEqual(svn.core.svn_dirent_join(b'foo', 'bar'), b'foo/bar')
+ with self.assertRaises(TypeError) as cm:
+ svn.core.svn_dirent_join(None, b'bar')
+ self.assertEqual(str(cm.exception),
+ "svn_dirent_join() argument base must be"
+ " bytes or str, not %s" % None.__class__.__name__)
+ with self.assertRaises(TypeError) as cm:
+ svn.core.svn_dirent_join(b'foo', self)
+ self.assertEqual(str(cm.exception),
+ "svn_dirent_join() argument component must be"
+ " bytes or str, not %s" % self.__class__.__name__)
+ with self.assertRaises(TypeError) as cm:
+ svn.core.svn_dirent_join('foo', 10)
+ self.assertEqual(str(cm.exception),
+ "svn_dirent_join() argument component must be"
+ " bytes or str, not int")
+
+ def test_char_ptr_in_unicode_exception(self):
+ """Check %typemap(in) IN_STRING handles unicode encode error correctly"""
+ with self.assertRaises(UnicodeEncodeError):
+ svn.core.svn_dirent_join(b'foo', u'\udc62\udc61\udc72')
+
+ def test_char_ptr_may_be_null(self):
+ """Check %typemap(in) char *MAY_BE_NULL works correctly"""
+ cfg = svn.core.svn_config_create2(False, False)
+ self.assertEqual(svn.core.svn_config_get(cfg, b'foo', b'bar', b'baz'),
+ b'baz')
+ self.assertEqual(svn.core.svn_config_get(cfg, b'foo', b'bar', 'baz'),
+ b'baz')
+ self.assertIsNone(svn.core.svn_config_get(cfg, b'foo', b'bar', None))
+ with self.assertRaises(TypeError) as cm:
+ svn.core.svn_config_get(cfg, b'foo', b'bar', self)
+ self.assertEqual(str(cm.exception),
+ "svn_config_get() argument default_value"
+ " must be bytes or str or None, not %s"
+ % self.__class__.__name__)
+
+ def test_char_ptr_may_be_null_unicode_exception(self):
+ """Check %typemap(in) char *MAY_BE_NULL handles unicode encode error
correctly"""
+ cfg = svn.core.svn_config_create2(False, False)
+ with self.assertRaises(UnicodeEncodeError):
+ svn.core.svn_config_get(cfg, u'f\udc6fo', b'bar', None)
+
+ def test_make_string_from_ob(self):
+ """Check make_string_from_ob and make_svn_string_from_ob work correctly"""
+ source_props = { b'a' : b'foo',
+ b'b' : 'foo',
+ 'c' : b'' }
+ target_props = { b'a' : '',
+ 'b' : 'bar',
+ b'c' : b'baz' }
+ expected = { b'a' : b'',
+ b'b' : b'bar',
+ b'c' : b'baz' }
+ self.assertEqual(svn.core.svn_prop_diffs(target_props, source_props),
+ expected)
+
+ def test_prophash_from_dict_null_value(self):
+ """Check make_svn_string_from_ob_maybe_null work correctly"""
+ source_props = { 'a' : 'foo',
+ 'b' : 'foo',
+ 'c' : None }
+ target_props = { 'a' : None,
+ 'b' : 'bar',
+ 'c' : 'baz' }
+ expected = { b'a' : None,
+ b'b' : b'bar',
+ b'c' : b'baz' }
+ self.assertEqual(svn.core.svn_prop_diffs(target_props, source_props),
+ expected)
+
+ def test_make_string_from_ob_unicode_exception(self):
+ """Check make_string_from_ob handles unicode encode error correctly"""
+ source_props = { b'a' : b'foo',
+ b'b' : u'foo',
+ u'\udc63' : b'' }
+ target_props = { b'a' : u'',
+ u'b' : u'bar',
+ b'c' : b'baz' }
+ with self.assertRaises(UnicodeEncodeError):
+ svn.core.svn_prop_diffs(target_props, source_props)
+
+ def test_make_svn_string_from_ob_unicode_exception(self):
+ """Check make_string_from_ob handles unicode encode error correctly"""
+ source_props = { b'a' : b'foo',
+ b'b' : 'foo',
+ u'c' : b'' }
+ target_props = { b'a' : u'',
+ u'b' : u'b\udc61r',
+ b'c' : b'baz' }
+ with self.assertRaises(UnicodeEncodeError):
+ svn.core.svn_prop_diffs(target_props, source_props)
+
+
+def suite():
+ return unittest.defaultTestLoader.loadTestsFromTestCase(
+ SubversionTypemapTestCase)
+
+if __name__ == '__main__':
+ runner = unittest.TextTestRunner()
+ runner.run(suite())
