[twill] Spurious parse error when processing tag without a tag Barry Hart Tue, 05 Aug 2008 13:31:55 -0700 Hi, At my company, we are starting to use Twill for server monitoring and load testing. We noticed a problem recently where Twill would report a "nested SELECTs" parse error on a page which looked fine to us. On further investigation, this was the HTML causing the problem. It is a 'quick links' drop-down which we include at the top of every page: <select py:if="not tg.identity.anonymous" id="quick_link_menu"> <option value="">Quick links</option> <option value="/promotion/list/create_new">Create a new promotion</option> <option value="/promotion/list/?type=planned&amp;dates.begin=&amp;dates.end=">Find a planned promotion</option> </select> Because events on this select tag control are processed on the browser side by JavaScript, it is not enclosed in a <form> tag. This HTML uncovered an apparent bug in twill/other_packages/_mechanize_dist. The problem stems from the following function in ClientForm.py. Because end_select() exits immediately if self._current_form is self._global_form, it leaves self._select set if there is a <select> tag outside of a <form> tag. def end_select(self): debug("") if self._current_form is self._global_form: return if self._select is None: raise ParseError("end of SELECT before start") if self._option is not None: self._end_option() self._select = None Thus, any subsequent <select> tag will cause a parse error in start_select(): def start_select(self, attrs): debug("%s", attrs) if self._select is not None: raise ParseError("nested SELECTs") if self._textarea is not None: raise ParseError("SELECT inside TEXTAREA") d = {} for key, val in attrs: d[key] = val self._select = d self._add_label(d) self._append_select_control({"__select": d}) A possible fix is to to clear self._select here: if self._current_form is self._global_form: self._select = None # NEW CODE return or you can monkeypatch around it, as I'm doing: # Patch the form parser class to ensure end select tags are processed correctly from twill.other_packages._mechanize_dist import ClientForm form_parser_class = b._browser._factory.soup_factory._forms_factory.form_parser_class old_end_select = form_parser_class.end_select def end_select(self): result = old_end_select(self) self._select = None return result form_parser_class.end_select = end_select Either of these fixes works but with one apparent side effect - now all the form indices are shifted by 1, e.g. form 1 becomes form 2. If either of these changes looks okay, could one of them be incorporated in Twill or mechanize? Thanks, Barry Hart _______________________________________________ twill mailing list twill@lists.idyll.org http://lists.idyll.org/listinfo/twill Previous message View by thread View by date Next message Reply via email to

[Barry Hart]

Hi,
 
At my company, we are starting to use Twill for server monitoring and load
testing.
 
We noticed a problem recently where Twill would report a "nested SELECTs"
parse error on a page which looked fine to us. On further investigation,
this was the HTML causing the problem. It is a 'quick links' drop-down
which we include at the top of every page:
 
      <select py:if="not tg.identity.anonymous" id="quick_link_menu">  
        <option value="">Quick links</option>  
        <option value="/promotion/list/create_new">Create a new
promotion</option>  
        <option
value="/promotion/list/?type=planned&amp;dates.begin=&amp;dates.end=">Find
a planned promotion</option>
      </select>

 
Because events on this select tag control are processed on the browser
side by JavaScript, it is not enclosed in a <form> tag. This HTML
uncovered an apparent bug in twill/other_packages/_mechanize_dist. The
problem stems from the following function in ClientForm.py. Because
end_select() exits immediately if self._current_form is self._global_form,
it leaves self._select set if there is a <select> tag outside of a <form>
tag.
 
    def end_select(self):
        debug("")
        if self._current_form is self._global_form:
            return
        if self._select is None:
            raise ParseError("end of SELECT before start")
 
        if self._option is not None:
            self._end_option()
 
        self._select = None

 
 Thus, any subsequent <select> tag will cause a parse error in
start_select():
 
    def start_select(self, attrs):
        debug("%s", attrs)
        if self._select is not None:
            raise ParseError("nested SELECTs")
        if self._textarea is not None:
            raise ParseError("SELECT inside TEXTAREA")
        d = {}
        for key, val in attrs:
            d[key] = val
 
        self._select = d
        self._add_label(d)
 
        self._append_select_control({"__select": d})

 
A possible fix is to to clear self._select here:
        if self._current_form is self._global_form:
            self._select = None # NEW CODE
            return

 
or you can monkeypatch around it, as I'm doing:
 
# Patch the form parser class to ensure end select tags are processed
correctly
from twill.other_packages._mechanize_dist import ClientForm
form_parser_class =
b._browser._factory.soup_factory._forms_factory.form_parser_class
old_end_select = form_parser_class.end_select
def end_select(self):
    result = old_end_select(self)
    self._select = None
    return result
form_parser_class.end_select = end_select

 
Either of these fixes works but with one apparent side effect - now all
the form indices are shifted by 1, e.g. form 1 becomes form 2.
 
If either of these changes looks okay, could one of them be incorporated
in Twill or mechanize?
 
Thanks,
Barry Hart
 

_______________________________________________
twill mailing list
twill@lists.idyll.org
http://lists.idyll.org/listinfo/twill