Re: [PATCH] fix broken Python3 support
On Tue, Jun 14, 2011 at 06:06:51AM +0200, Bram Moolenaar wrote: Thanks for the update. I'll check it out later. I have this remark in the todo list, I suppose this patch is fixing it: Python3 interface doesn't handle utf-8 correctly? (Nov 2010, lilydjwg) Yes. -- Best regards, lilydjwg Linux Vim Python 我的博客 http://bit.ly/lilydjwg or http://goo.gl/y4Gsy -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: [PATCH] fix broken Python3 support
On Mon, Jun 13, 2011 at 03:54:04AM +0200, Bram Moolenaar wrote:
I have included the patch, but on my Ubuntu system I get this error:
E448: Could not load library function PyUnicode_UCSX_*
when doing something simple as :py3 print('hello')
Can you fix that?
Yes, just a typo preventing Python symbols from loading.
Later a user found that there was heavy memory leakage, and I tried hard
to fix them. It was partly introduced by my code, and I believe partly
had been there before. Attached patch should fix this too.
--
Best regards,
lilydjwg
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
diff --git a/src/if_py_both.h b/src/if_py_both.h
index f78eccb..72ec8b0 100644
--- a/src/if_py_both.h
+++ b/src/if_py_both.h
@@ -65,10 +65,10 @@ static struct PyMethodDef OutputMethods[] = {
OutputWrite(PyObject *self, PyObject *args)
{
int len;
-char *str;
+char *str = NULL;
int error = ((OutputObject *)(self))-error;
-if (!PyArg_ParseTuple(args, s#, str, len))
+if (!PyArg_ParseTuple(args, es#, p_enc, str, len))
return NULL;
Py_BEGIN_ALLOW_THREADS
@@ -76,6 +76,7 @@ OutputWrite(PyObject *self, PyObject *args)
writer((writefn)(error ? emsg : msg), (char_u *)str, len);
Python_Release_Vim();
Py_END_ALLOW_THREADS
+PyMem_Free(str);
Py_INCREF(Py_None);
return Py_None;
@@ -104,10 +105,10 @@ OutputWritelines(PyObject *self, PyObject *args)
for (i = 0; i n; ++i)
{
PyObject *line = PyList_GetItem(list, i);
- char *str;
+ char *str = NULL;
PyInt len;
- if (!PyArg_Parse(line, s#, str, len)) {
+ if (!PyArg_Parse(line, es#, p_enc, str, len)) {
PyErr_SetString(PyExc_TypeError, _(writelines() requires list of
strings));
Py_DECREF(list);
return NULL;
@@ -118,6 +119,7 @@ OutputWritelines(PyObject *self, PyObject *args)
writer((writefn)(error ? emsg : msg), (char_u *)str, len);
Python_Release_Vim();
Py_END_ALLOW_THREADS
+ PyMem_Free(str);
}
Py_DECREF(list);
@@ -681,6 +683,7 @@ StringToLine(PyObject *obj)
{
const char *str;
char *save;
+PyObject *bytes;
PyInt len;
PyInt i;
char *p;
@@ -691,8 +694,9 @@ StringToLine(PyObject *obj)
return NULL;
}
-str = PyString_AsString(obj);
-len = PyString_Size(obj);
+bytes = PyString_AsBytes(obj);
+str = PyString_AsString(bytes);
+len = PyString_Size(bytes);
/*
* Error checking: String must not contain newlines, as we
@@ -731,6 +735,7 @@ StringToLine(PyObject *obj)
}
save[i] = '\0';
+PyString_FreeBytes(bytes);
return save;
}
@@ -908,6 +913,193 @@ SetBufferLine(buf_T *buf, PyInt n, PyObject *line, PyInt
*len_change)
}
}
+/* Replace a range of lines in the specified buffer. The line numbers are in
+ * Vim format (1-based). The range is from lo up to, but not including, hi.
+ * The replacement lines are given as a Python list of string objects. The
+ * list is checked for validity and correct format. Errors are returned as a
+ * value of FAIL. The return value is OK on success.
+ * If OK is returned and len_change is not NULL, *len_change
+ * is set to the change in the buffer length.
+ */
+static int
+SetBufferLineList(buf_T *buf, PyInt lo, PyInt hi, PyObject *list, PyInt
*len_change)
+{
+/* First of all, we check the thpe of the supplied Python object.
+ * There are three cases:
+ * 1. NULL, or None - this is a deletion.
+ * 2. A list- this is a replacement.
+ * 3. Anything else - this is an error.
+ */
+if (list == Py_None || list == NULL)
+{
+ PyInt i;
+ PyInt n = (int)(hi - lo);
+ buf_T *savebuf = curbuf;
+
+ PyErr_Clear();
+ curbuf = buf;
+
+ if (u_savedel((linenr_T)lo, (long)n) == FAIL)
+ PyErr_SetVim(_(cannot save undo information));
+ else
+ {
+ for (i = 0; i n; ++i)
+ {
+ if (ml_delete((linenr_T)lo, FALSE) == FAIL)
+ {
+ PyErr_SetVim(_(cannot delete line));
+ break;
+ }
+ }
+ if (buf == curwin-w_buffer)
+ py_fix_cursor((linenr_T)lo, (linenr_T)hi, (linenr_T)-n);
+ deleted_lines_mark((linenr_T)lo, (long)i);
+ }
+
+ curbuf = savebuf;
+
+ if (PyErr_Occurred() || VimErrorCheck())
+ return FAIL;
+
+ if (len_change)
+ *len_change = -n;
+
+ return OK;
+}
+else if (PyList_Check(list))
+{
+ PyInt i;
+ PyInt new_len = PyList_Size(list);
+ PyInt old_len = hi - lo;
+ PyInt extra = 0; /* lines added to text, can be negative */
+ char**array;
+ buf_T *savebuf;
+
+ if (new_len
Re: [PATCH] fix broken Python3 support
lilydjwg wronte:
On Mon, Jun 13, 2011 at 03:54:04AM +0200, Bram Moolenaar wrote:
I have included the patch, but on my Ubuntu system I get this error:
E448: Could not load library function PyUnicode_UCSX_*
when doing something simple as :py3 print('hello')
Can you fix that?
Yes, just a typo preventing Python symbols from loading.
Later a user found that there was heavy memory leakage, and I tried hard
to fix them. It was partly introduced by my code, and I believe partly
had been there before. Attached patch should fix this too.
Thanks for the update. I'll check it out later.
I have this remark in the todo list, I suppose this patch is fixing it:
Python3 interface doesn't handle utf-8 correctly? (Nov 2010, lilydjwg)
--
hundred-and-one symptoms of being an internet addict:
179. You wonder why your household garbage can doesn't have an
empty recycle bin button.
/// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org///
\\\help me help AIDS victims -- http://ICCF-Holland.org///
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: [PATCH] fix broken Python3 support
lilydjwg wrote:
I made a patch to fix some bugs related to Vim's Python3 support. The
bugs I found are:
* vim.error is a `str` instead of an `Exception` object, so `except` or
`raise` it causes a `SystemError` exception
* buffer objects do not support slice assignment
* when exchange text between Vim and Python, multibyte texts become
gabage or cause Unicode Expceptions, etc
* `py3file` tries to read in the file as Unicode, sometimes causes
UnicodeDecodeException
The patch works on my Ubuntu Linux 32bit and Windows XP SP3 and should
apply to any Vim 7.3+ version.
I have included the patch, but on my Ubuntu system I get this error:
E448: Could not load library function PyUnicode_UCSX_*
when doing something simple as :py3 print('hello')
Can you fix that?
--
If Microsoft would build a car...
... Occasionally your car would die on the freeway for no
reason. You would have to pull over to the side of the road,
close all of the car windows, shut it off, restart it, and
reopen the windows before you could continue. For some reason
you would simply accept this.
/// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\
///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\ an exciting new programming language -- http://www.Zimbu.org///
\\\help me help AIDS victims -- http://ICCF-Holland.org///
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Re: [PATCH] fix broken Python3 support
Lilydjwg wrote: I made a patch to fix some bugs related to Vim's Python3 support. The bugs I found are: * vim.error is a `str` instead of an `Exception` object, so `except` or `raise` it causes a `SystemError` exception * buffer objects do not support slice assignment * when exchange text between Vim and Python, multibyte texts become gabage or cause Unicode Expceptions, etc * `py3file` tries to read in the file as Unicode, sometimes causes UnicodeDecodeException The patch works on my Ubuntu Linux 32bit and Windows XP SP3 and should apply to any Vim 7.3+ version. Sounds good. Can a few people try this out? -- Apologies for taking up the bandwidth with the apology. Anything else I can apologise for .. er no can't think of anything, sorry about that. Andy Hunt (Member of British Olympic Apology Squad) /// Bram Moolenaar -- b...@moolenaar.net -- http://www.Moolenaar.net \\\ ///sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\ \\\ an exciting new programming language -- http://www.Zimbu.org/// \\\help me help AIDS victims -- http://ICCF-Holland.org/// -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
Re: [PATCH] fix broken Python3 support
Sorry, I forgot the attachment.
--
Best regards,
lilydjwg
Linux Vim Python 我的博客
http://lilydjwg.is-programmer.com/
--
You received this message from the vim_dev maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
diff --git a/src/if_py_both.h b/src/if_py_both.h
index e435e3e..2e701a8 100644
--- a/src/if_py_both.h
+++ b/src/if_py_both.h
@@ -63,10 +63,10 @@ static struct PyMethodDef OutputMethods[] = {
OutputWrite(PyObject *self, PyObject *args)
{
int len;
-char *str;
+char *str = NULL;
int error = ((OutputObject *)(self))-error;
-if (!PyArg_ParseTuple(args, s#, str, len))
+if (!PyArg_ParseTuple(args, es#, p_enc, str, len))
return NULL;
Py_BEGIN_ALLOW_THREADS
@@ -74,6 +74,7 @@ OutputWrite(PyObject *self, PyObject *args)
writer((writefn)(error ? emsg : msg), (char_u *)str, len);
Python_Release_Vim();
Py_END_ALLOW_THREADS
+PyMem_Free(str);
Py_INCREF(Py_None);
return Py_None;
@@ -102,10 +103,10 @@ OutputWritelines(PyObject *self, PyObject *args)
for (i = 0; i n; ++i)
{
PyObject *line = PyList_GetItem(list, i);
- char *str;
+ char *str = NULL;
PyInt len;
- if (!PyArg_Parse(line, s#, str, len)) {
+ if (!PyArg_Parse(line, es#, p_enc, str, len)) {
PyErr_SetString(PyExc_TypeError, _(writelines() requires list of strings));
Py_DECREF(list);
return NULL;
@@ -116,6 +117,7 @@ OutputWritelines(PyObject *self, PyObject *args)
writer((writefn)(error ? emsg : msg), (char_u *)str, len);
Python_Release_Vim();
Py_END_ALLOW_THREADS
+ PyMem_Free(str);
}
Py_DECREF(list);
@@ -897,6 +899,193 @@ SetBufferLine(buf_T *buf, PyInt n, PyObject *line, PyInt *len_change)
}
}
+/* Replace a range of lines in the specified buffer. The line numbers are in
+ * Vim format (1-based). The range is from lo up to, but not including, hi.
+ * The replacement lines are given as a Python list of string objects. The
+ * list is checked for validity and correct format. Errors are returned as a
+ * value of FAIL. The return value is OK on success.
+ * If OK is returned and len_change is not NULL, *len_change
+ * is set to the change in the buffer length.
+ */
+static int
+SetBufferLineList(buf_T *buf, PyInt lo, PyInt hi, PyObject *list, PyInt *len_change)
+{
+/* First of all, we check the thpe of the supplied Python object.
+ * There are three cases:
+ * 1. NULL, or None - this is a deletion.
+ * 2. A list - this is a replacement.
+ * 3. Anything else - this is an error.
+ */
+if (list == Py_None || list == NULL)
+{
+ PyInt i;
+ PyInt n = (int)(hi - lo);
+ buf_T *savebuf = curbuf;
+
+ PyErr_Clear();
+ curbuf = buf;
+
+ if (u_savedel((linenr_T)lo, (long)n) == FAIL)
+ PyErr_SetVim(_(cannot save undo information));
+ else
+ {
+ for (i = 0; i n; ++i)
+ {
+ if (ml_delete((linenr_T)lo, FALSE) == FAIL)
+ {
+ PyErr_SetVim(_(cannot delete line));
+ break;
+ }
+ }
+ if (buf == curwin-w_buffer)
+ py_fix_cursor((linenr_T)lo, (linenr_T)hi, (linenr_T)-n);
+ deleted_lines_mark((linenr_T)lo, (long)i);
+ }
+
+ curbuf = savebuf;
+
+ if (PyErr_Occurred() || VimErrorCheck())
+ return FAIL;
+
+ if (len_change)
+ *len_change = -n;
+
+ return OK;
+}
+else if (PyList_Check(list))
+{
+ PyInt i;
+ PyInt new_len = PyList_Size(list);
+ PyInt old_len = hi - lo;
+ PyInt extra = 0; /* lines added to text, can be negative */
+ char **array;
+ buf_T *savebuf;
+
+ if (new_len == 0) /* avoid allocating zero bytes */
+ array = NULL;
+ else
+ {
+ array = (char **)alloc((unsigned)(new_len * sizeof(char *)));
+ if (array == NULL)
+ {
+ PyErr_NoMemory();
+ return FAIL;
+ }
+ }
+
+ for (i = 0; i new_len; ++i)
+ {
+ PyObject *line = PyList_GetItem(list, i);
+
+ array[i] = StringToLine(line);
+ if (array[i] == NULL)
+ {
+ while (i)
+ vim_free(array[--i]);
+ vim_free(array);
+ return FAIL;
+ }
+ }
+
+ savebuf = curbuf;
+
+ PyErr_Clear();
+ curbuf = buf;
+
+ if (u_save((linenr_T)(lo-1), (linenr_T)hi) == FAIL)
+ PyErr_SetVim(_(cannot save undo information));
+
+ /* If the size of the range is reducing (ie, new_len old_len) we
+ * need to delete some old_len. We do this at the start, by
+ * repeatedly deleting line lo.
+ */
+ if (!PyErr_Occurred())
+ {
+ for (i = 0; i old_len - new_len; ++i)
+ if (ml_delete((linenr_T)lo, FALSE) == FAIL)
+ {
+ PyErr_SetVim(_(cannot delete line));
+ break;
+ }
+ extra -= i;
+ }
+
+ /* For as long as possible, replace the existing old_len with the
+ * new old_len. This is a more efficient operation, as it requires
+ * less memory allocation and freeing.
+ */
+ if (!PyErr_Occurred())
+ {
+ for (i = 0; i old_len i new_len; ++i)
+ if (ml_replace((linenr_T)(lo+i), (char_u *)array[i], FALSE)
+ == FAIL)
+ {
+ PyErr_SetVim(_(cannot replace line));
[PATCH] fix broken Python3 support
Hi there, I made a patch to fix some bugs related to Vim's Python3 support. The bugs I found are: * vim.error is a `str` instead of an `Exception` object, so `except` or `raise` it causes a `SystemError` exception * buffer objects do not support slice assignment * when exchange text between Vim and Python, multibyte texts become gabage or cause Unicode Expceptions, etc * `py3file` tries to read in the file as Unicode, sometimes causes UnicodeDecodeException The patch works on my Ubuntu Linux 32bit and Windows XP SP3 and should apply to any Vim 7.3+ version. -- Best regards, lilydjwg Linux Vim Python 我的博客 http://lilydjwg.is-programmer.com/ -- You received this message from the vim_dev maillist. Do not top-post! Type your reply below the text you are replying to. For more information, visit http://www.vim.org/maillist.php
