Re: [Python-mode] looking for a discussion of things python(-mode) ish
Barry Warsaw wrote: On Sep 9, 2009, at 5:56 AM, Andreas Roehler wrote: checking for the triple-quoted-bug: with python.el, (nth 8 (syntax-ppss)) shows the correct result. Unfortunately with python-mode.el (nth 8 (syntax-ppss)) fails. From there I assume, setting the syntax-table properly might solve the bug I haven't looked at this stuff in ages. Has the syntax table grown more support for the kind of things Python's triple quoted strings needs? -Barry Hi Barry, after (put-text-property TRIPLE-QUOTE-START-LAST-CHAR-POS TRIPLE-QUOTE-END-FIRST-CHAR-POS 'py-mode-syntax-table '(15 . 34)) doublequotes may be inserted inside triple quoted string - no fontification-bug then. Similar thing in python.el AFAIU. Strange thing I don't understand in context with checks inside triple quoted string (TQS): text-properties-at - (face font-lock-string-face py-mode-syntax-table (15 . 34) fontified t) But (syntax-after pos) inside returns different syntax-classes, (7) or (2) Does it query an underlying syntax-table? Below my test-text.py, triple quoted string starts at pos 330 Used (put-text-property 330 347 'py-mode-syntax-table '(15 . 34)) Probably the right thing is: (put-text-property (copy-marker 330) (copy-marker 347) 'py-mode-syntax-table '(15 . 34)) Cheers Andreas ;; #! /usr/bin/env python # -*- coding: utf-8 -*- import re, sys, os, pdb, random, time import MySQLdb from urllib2 import Request, urlopen, URLError, HTTPError # Get the command line arguments args = sys.argv # pdb.set_trace() # Get the name of the file to count the words in filename = args[1] def usage(): print Usage: %s % ( os.path.basename(sys.argv[0])) def main(): if len(sys.argv)==1: usage() sys.exit() if __name__==__main__: main() ___ Python-mode mailing list Python-mode@python.org http://mail.python.org/mailman/listinfo/python-mode
[Python-mode] triple-quoted-string bug fixed
Hi Barry, diff attached against latest python-mode.el solves bug 328790, the triple string bug for me - checked with X- and GNU Emacs. Took some stuff from python.el, thanks towards its excellent author BTW. Did create a branch for it. Log now reads revno: 352 committer: Andreas Roehler andreas.roeh...@online.de branch nick: python-mode timestamp: Thu 2009-09-10 13:30:24 +0200 message: --fixes=lp:328790 Should I try bzr push lp:~a-roehler/python-mode/triple-quoted-string-bug-328790.el ? Cheers Andreas 11a12 387a389,468 ;; 2009-09-10 a.roeh...@web.de changed section start ;; from python.el, version 22.1 (defconst python-font-lock-syntactic-keywords ;; Make outer chars of matching triple-quote sequences into generic ;; string delimiters. Fixme: Is there a better way? ;; First avoid a sequence preceded by an odd number of backslashes. `((,(rx (not (any ?\\)) ?\\ (* (and ?\\ ?\\)) (group (syntax string-quote)) (backref 1) (group (backref 1))) (2 ,(string-to-syntax \))) ; dummy (,(rx (group (optional (any uUrR))) ; prefix gets syntax property (optional (any rR)) ; possible second prefix (group (syntax string-quote)) ; maybe gets property (backref 2) ; per first quote (group (backref 2))) ; maybe gets property (1 (python-quote-syntax 1)) (2 (python-quote-syntax 2)) (3 (python-quote-syntax 3))) ;; This doesn't really help. ;;; (,(rx (and ?\\ (group ?\n))) (1 )) )) (defun python-quote-syntax (n) Put `syntax-table' property correctly on triple quote. Used for syntactic keywords. N is the match number (1, 2 or 3). ;; Given a triple quote, we have to check the context to know ;; whether this is an opening or closing triple or whether it's ;; quoted anyhow, and should be ignored. (For that we need to do ;; the same job as `syntax-ppss' to be correct and it seems to be OK ;; to use it here despite initial worries.) We also have to sort ;; out a possible prefix -- well, we don't _have_ to, but I think it ;; should be treated as part of the string. ;; Test cases: ;; urar x='' # ;; x = ''' ' a ;; ''' ;; x '' x \ x (save-excursion (goto-char (match-beginning 0)) (cond ;; Consider property for the last char if in a fenced string. ((= n 3) (let* ((font-lock-syntactic-keywords nil) (syntax (syntax-ppss))) (when (eq t (nth 3 syntax)) ; after unclosed fence (goto-char (nth 8 syntax)) ; fence position (skip-chars-forward uUrR) ; skip any prefix ;; Is it a matching sequence? (if (eq (char-after) (char-after (match-beginning 2))) (eval-when-compile (string-to-syntax |)) ;; Consider property for initial char, accounting for prefixes. ((or (and (= n 2) ; leading quote (not prefix) (= (match-beginning 1) (match-end 1))) ; prefix is null (and (= n 1) ; prefix (/= (match-beginning 1) (match-end 1 ; non-empty (let ((font-lock-syntactic-keywords nil)) (unless (eq 'string (syntax-ppss-context (syntax-ppss))) (eval-when-compile (string-to-syntax |) ;; Otherwise (we're in a non-matching string) the property is ;; nil, which is OK. ))) (defsubst python-in-string/comment () Return non-nil if point is in a Python literal (a comment or string). ;; We don't need to save the match data. (nth 8 (syntax-ppss))) (defconst python-space-backslash-table (let ((table (copy-syntax-table py-mode-syntax-table))) (modify-syntax-entry ?\\ table) table) `python-mode-syntax-table' with backslash given whitespace syntax.) ;; 2009-09-10 a.roeh...@web.de changed section end 508d588 (put 'python-mode 'font-lock-defaults '(python-font-lock-keywords)) 730,771c810,875 (defvar py-mode-syntax-table nil Syntax table used in `python-mode' buffers.) (when (not py-mode-syntax-table) (setq py-mode-syntax-table (make-syntax-table)) (modify-syntax-entry ?\( () py-mode-syntax-table) (modify-syntax-entry ?\) )( py-mode-syntax-table) (modify-syntax-entry ?\[ (] py-mode-syntax-table) (modify-syntax-entry ?\] )[ py-mode-syntax-table) (modify-syntax-entry ?\{ (} py-mode-syntax-table) (modify-syntax-entry ?\} ){ py-mode-syntax-table) ;; Add operator symbols misassigned in the std table (modify-syntax-entry ?\$ . py-mode-syntax-table) (modify-syntax-entry ?\% . py-mode-syntax-table) (modify-syntax-entry ?\ . py-mode-syntax-table) (modify-syntax-entry ?\* . py-mode-syntax-table) (modify-syntax-entry ?\+ . py-mode-syntax-table) (modify-syntax-entry ?\- . py-mode-syntax-table) (modify-syntax-entry ?\/ . py-mode-syntax-table) (modify-syntax-entry ?\ . py-mode-syntax-table) (modify-syntax-entry ?\= . py-mode-syntax-table) (modify-syntax-entry ?\ . py-mode-syntax-table) (modify-syntax-entry ?\| . py-mode-syntax-table) ;; For historical reasons, underscore is
[Python-mode] triple-quoted-string bug fixed (2)
Hi, had to move `py-mode-syntax-table' still upward in file. Now no complaining any more. Thanks being patient... Andreas 11a12 387a389,487 ;; 2009-09-10 a.roeh...@web.de changed section start ;; from python.el, version 22.1 (defconst python-font-lock-syntactic-keywords ;; Make outer chars of matching triple-quote sequences into generic ;; string delimiters. Fixme: Is there a better way? ;; First avoid a sequence preceded by an odd number of backslashes. `((,(rx (not (any ?\\)) ?\\ (* (and ?\\ ?\\)) (group (syntax string-quote)) (backref 1) (group (backref 1))) (2 ,(string-to-syntax \))) ; dummy (,(rx (group (optional (any uUrR))) ; prefix gets syntax property (optional (any rR)) ; possible second prefix (group (syntax string-quote)) ; maybe gets property (backref 2) ; per first quote (group (backref 2))) ; maybe gets property (1 (python-quote-syntax 1)) (2 (python-quote-syntax 2)) (3 (python-quote-syntax 3))) ;; This doesn't really help. ;;; (,(rx (and ?\\ (group ?\n))) (1 )) )) (defun python-quote-syntax (n) Put `syntax-table' property correctly on triple quote. Used for syntactic keywords. N is the match number (1, 2 or 3). ;; Given a triple quote, we have to check the context to know ;; whether this is an opening or closing triple or whether it's ;; quoted anyhow, and should be ignored. (For that we need to do ;; the same job as `syntax-ppss' to be correct and it seems to be OK ;; to use it here despite initial worries.) We also have to sort ;; out a possible prefix -- well, we don't _have_ to, but I think it ;; should be treated as part of the string. ;; Test cases: ;; urar x='' # ;; x = ''' ' a ;; ''' ;; x '' x \ x (save-excursion (goto-char (match-beginning 0)) (cond ;; Consider property for the last char if in a fenced string. ((= n 3) (let* ((font-lock-syntactic-keywords nil) (syntax (syntax-ppss))) (when (eq t (nth 3 syntax)) ; after unclosed fence (goto-char (nth 8 syntax)) ; fence position (skip-chars-forward uUrR) ; skip any prefix ;; Is it a matching sequence? (if (eq (char-after) (char-after (match-beginning 2))) (eval-when-compile (string-to-syntax |)) ;; Consider property for initial char, accounting for prefixes. ((or (and (= n 2) ; leading quote (not prefix) (= (match-beginning 1) (match-end 1))) ; prefix is null (and (= n 1) ; prefix (/= (match-beginning 1) (match-end 1 ; non-empty (let ((font-lock-syntactic-keywords nil)) (unless (eq 'string (syntax-ppss-context (syntax-ppss))) (eval-when-compile (string-to-syntax |) ;; Otherwise (we're in a non-matching string) the property is ;; nil, which is OK. ))) (defvar py-mode-syntax-table (let ((table (make-syntax-table))) ;; Give punctuation syntax to ASCII that normally has symbol ;; syntax or has word syntax and isn't a letter. (let ((symbol (string-to-syntax _)) (sst (standard-syntax-table))) (dotimes (i 128) (unless (= i ?_) (if (equal symbol (aref sst i)) (modify-syntax-entry i . table) (modify-syntax-entry ?$ . table) (modify-syntax-entry ?% . table) ;; exceptions (modify-syntax-entry ?# table) (modify-syntax-entry ?\n table) (modify-syntax-entry ?' \ table) (modify-syntax-entry ?` $ table) table)) (defsubst python-in-string/comment () Return non-nil if point is in a Python literal (a comment or string). ;; We don't need to save the match data. (nth 8 (syntax-ppss))) (defconst python-space-backslash-table (let ((table (copy-syntax-table py-mode-syntax-table))) (modify-syntax-entry ?\\ table) table) `python-mode-syntax-table' with backslash given whitespace syntax.) ;; 2009-09-10 a.roeh...@web.de changed section end 508d607 (put 'python-mode 'font-lock-defaults '(python-font-lock-keywords)) 730,771c829,875 (defvar py-mode-syntax-table nil Syntax table used in `python-mode' buffers.) (when (not py-mode-syntax-table) (setq py-mode-syntax-table (make-syntax-table)) (modify-syntax-entry ?\( () py-mode-syntax-table) (modify-syntax-entry ?\) )( py-mode-syntax-table) (modify-syntax-entry ?\[ (] py-mode-syntax-table) (modify-syntax-entry ?\] )[ py-mode-syntax-table) (modify-syntax-entry ?\{ (} py-mode-syntax-table) (modify-syntax-entry ?\} ){ py-mode-syntax-table) ;; Add operator symbols misassigned in the std table (modify-syntax-entry ?\$ . py-mode-syntax-table) (modify-syntax-entry ?\% . py-mode-syntax-table) (modify-syntax-entry ?\ . py-mode-syntax-table) (modify-syntax-entry ?\* . py-mode-syntax-table) (modify-syntax-entry ?\+ . py-mode-syntax-table) (modify-syntax-entry ?\- . py-mode-syntax-table) (modify-syntax-entry ?\/ . py-mode-syntax-table)
Re: [Python-mode] triple-quoted-string bug fixed
Damn, that was fast. Thanks. On Thu, Sep 10, 2009 at 1:54 PM, Andreas Roehler andreas.roeh...@online.de wrote: Hi Barry, diff attached against latest python-mode.el solves bug 328790, the triple string bug for me - checked with X- and GNU Emacs. Took some stuff from python.el, thanks towards its excellent author BTW. Did create a branch for it. Log now reads revno: 352 committer: Andreas Roehler andreas.roeh...@online.de branch nick: python-mode timestamp: Thu 2009-09-10 13:30:24 +0200 message: --fixes=lp:328790 Should I try bzr push lp:~a-roehler/python-mode/triple-quoted-string-bug-328790.el ? Cheers Andreas ___ Python-mode mailing list Python-mode@python.org http://mail.python.org/mailman/listinfo/python-mode