Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
Hi Dan, Plus, as I stated earlier, libxml will be required to handle Wispr responses (and hotspot 2.0), which are legitimate XML trees. So, why not just add it as a dependency right now? that is a funny comment since within ConnMan, we are using GMarkup to handle the WISPr login. And actually if you are looking at anything of the HTML except the WISPr XML you are doing something wrong. Do not bother parsing the HTML page. Either operators add the WISPr XML somewhere or they end up hiding it in comment section. I am also confused what Hotspot 2.0 has to do with XML. Am I reading a different specification than you are? Yeah I think GMarkup can handle most of the XML stuff. But if we want to better handle random hotspots in the future (like my hotel one) we may need to do some HTML parsing. I don't necessarily think that would require XML though, perhaps just regex. if your hotel does not support WISPr you are most certainly on the loosing end with trying to figure out anything. We have never bothered to try to handle these. They just trigger the opening of a browser. There exists in theory another protocol besides WISPr that early Boingo hotspots used, but I have not seen it in a long time. I think the last one was Swisscom and they also support WISPr. So rather pointless trying to chase that one down. Only trick part is the the XML from WISPr is not always present on the first page the hostpot returns. Sometimes it is on the second or later. One example for such a setup is Orange in France. However all the hotspot do use a proper HTTP Location header and thus no need to parse HTML ever. Regards Marcel ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
One note: The original patch was supposed to handle 511 http status code, not wispr. see http://tools.ietf.org/html/rfc6585 2013/2/19 Marcel Holtmann mar...@holtmann.org Hi Dan, Plus, as I stated earlier, libxml will be required to handle Wispr responses (and hotspot 2.0), which are legitimate XML trees. So, why not just add it as a dependency right now? that is a funny comment since within ConnMan, we are using GMarkup to handle the WISPr login. And actually if you are looking at anything of the HTML except the WISPr XML you are doing something wrong. Do not bother parsing the HTML page. Either operators add the WISPr XML somewhere or they end up hiding it in comment section. I am also confused what Hotspot 2.0 has to do with XML. Am I reading a different specification than you are? Yeah I think GMarkup can handle most of the XML stuff. But if we want to better handle random hotspots in the future (like my hotel one) we may need to do some HTML parsing. I don't necessarily think that would require XML though, perhaps just regex. if your hotel does not support WISPr you are most certainly on the loosing end with trying to figure out anything. We have never bothered to try to handle these. They just trigger the opening of a browser. There exists in theory another protocol besides WISPr that early Boingo hotspots used, but I have not seen it in a long time. I think the last one was Swisscom and they also support WISPr. So rather pointless trying to chase that one down. Only trick part is the the XML from WISPr is not always present on the first page the hostpot returns. Sometimes it is on the second or later. One example for such a setup is Orange in France. However all the hotspot do use a proper HTTP Location header and thus no need to parse HTML ever. Regards Marcel -- Jonh Wendell http://www.bani.com.br ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
Hi Jonh, One note: The original patch was supposed to handle 511 http status code, not wispr. see http://tools.ietf.org/html/rfc6585 I do not remember any portal using 511 status. At least the ones we tested have not used it. However we have seen hotspots that are using 200 success status code and still just give you a portal page. Trying to rely on the HTTP status is pretty much a bad idea. You need something a bit more trust worthy. The ConnMan support for WISPr actually uses special vendor HTTP headers to confirm the connection. Regards Marcel ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
Hi Jonh, Plus, as I stated earlier, libxml will be required to handle Wispr responses (and hotspot 2.0), which are legitimate XML trees. So, why not just add it as a dependency right now? that is a funny comment since within ConnMan, we are using GMarkup to handle the WISPr login. And actually if you are looking at anything of the HTML except the WISPr XML you are doing something wrong. Do not bother parsing the HTML page. Either operators add the WISPr XML somewhere or they end up hiding it in comment section. I am also confused what Hotspot 2.0 has to do with XML. Am I reading a different specification than you are? Regards Marcel ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
On Tue, 2013-02-19 at 01:45 +0100, Marcel Holtmann wrote: Hi Jonh, Plus, as I stated earlier, libxml will be required to handle Wispr responses (and hotspot 2.0), which are legitimate XML trees. So, why not just add it as a dependency right now? that is a funny comment since within ConnMan, we are using GMarkup to handle the WISPr login. And actually if you are looking at anything of the HTML except the WISPr XML you are doing something wrong. Do not bother parsing the HTML page. Either operators add the WISPr XML somewhere or they end up hiding it in comment section. I am also confused what Hotspot 2.0 has to do with XML. Am I reading a different specification than you are? Yeah I think GMarkup can handle most of the XML stuff. But if we want to better handle random hotspots in the future (like my hotel one) we may need to do some HTML parsing. I don't necessarily think that would require XML though, perhaps just regex. Dan ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
Hi Dan. 2013/2/11 Dan Williams d...@redhat.com On Mon, 2013-02-11 at 15:30 -0200, Jonh Wendell wrote: In these patches I want to fix the 511-http-status. As it's something new, of course most hotspots don't use that (including my employer). Almost all of them rely on 30X Moved with help of the Wispr 'pseudo-protocol'. It's on my TODO list to work on those scenarios. That would touch only code in NMConnectivity object. Indeed, for that cases, we would use xml parsing as wispr is xml. Thanks for the clarification. In any case, would a regex be possible here instead of the XML for now? It might be less code and would certainly be less error-prone when checking broken HTML which is quite common. I'm not sure if parsing a HTML with regex is less error-prone. There are lots of traps in mal-formed HTML, or even if legitimate HTML structs like img title=displays src=big.gif So, I'd prefer to rely on libxml which is supposed to be more smart than a simple regex. In the patch, I'm using the flags 'HTML_PARSE_NOERROR | HTML_PARSE_NOWARNING' so that libxml doesn't complain about broken HTML. In fact, I've tested here some broken HTML and it was parsed successfully by libxml. Plus, as I stated earlier, libxml will be required to handle Wispr responses (and hotspot 2.0), which are legitimate XML trees. So, why not just add it as a dependency right now? Dan 2013/2/11 Dan Williams d...@redhat.com On Mon, 2013-02-11 at 17:10 +0100, Bastien Nocera wrote: On Mon, 2013-02-11 at 10:06 -0600, Dan Williams wrote: On Mon, 2013-02-11 at 12:09 -0200, Jonh Wendell wrote: From: Jonh Wendell jonh.wend...@oiwifi.com.br libsoup already depends on libxml2 but we need to explicitly link to it. At least we already theoretically required it; though is it possible to use GMarkup here instead of libxml2? GMarkup would be somewhat simpler, though it's only a subset. Given how it's used, there's probably little reason this couldn't be a regexp. Both GMarkup and libxml2 would choke on broken, slightly broken, and very very broken HTML files. Yeah, and I've heard that for example, some hotspots literally just append raw XML to the end of the HTTP request outside the XML. I think we need to be somewhat more robust here and XML parsing may not be the way to get there? Also, we may need to add special cases for various hotspots, which might require regex and not just XML parsing. Dan -- Jonh Wendell http://www.bani.com.br -- Jonh Wendell http://www.bani.com.br ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
[PATCH 2/5] connectivity: Add libxml2 as a dependency
From: Jonh Wendell jonh.wend...@oiwifi.com.br
libsoup already depends on libxml2 but we need to explicitly link
to it.
Signed-off-by: Jonh Wendell jonh.wend...@oiwifi.com.br
---
configure.ac| 14 +++---
src/Makefile.am | 4 ++--
2 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/configure.ac b/configure.ac
index 916561a..49a6203 100644
--- a/configure.ac
+++ b/configure.ac
@@ -573,15 +573,23 @@ fi
AC_DEFINE_UNQUOTED(KERNEL_FIRMWARE_DIR, $KERNEL_FIRMWARE_DIR, [Define to
path of the kernel firmware directory])
AC_SUBST(KERNEL_FIRMWARE_DIR)
+PKG_CHECK_MODULES(LIBXML, [libxml-2.0], [have_libxml=yes],[have_libxml=no])
PKG_CHECK_MODULES(LIBSOUP, [libsoup-2.4 = 2.26],
[have_libsoup=yes],[have_libsoup=no])
+if test x$have_libsoup = xyes -a x$have_libxml = xyes; then
+can_have_connectivity=yes
+else
+can_have_connectivity=no
+fi
AC_ARG_ENABLE(concheck, AS_HELP_STRING([--enable-concheck], [enable
connectivity checking support]),
- [enable_concheck=${enableval}],
[enable_concheck=${have_libsoup}])
+ [enable_concheck=${enableval}],
[enable_concheck=${can_have_connectivity}])
if (test ${enable_concheck} = yes); then
- if test x$have_libsoup = xno; then
- AC_MSG_ERROR(Connectivity checking requires libsoup development
headers)
+ if test x$can_have_connectivity = xno; then
+ AC_MSG_ERROR(Connectivity checking requires libsoup and libxml2
development headers)
fi
AC_SUBST(LIBSOUP_CFLAGS)
AC_SUBST(LIBSOUP_LIBS)
+ AC_SUBST(LIBXML_CFLAGS)
+ AC_SUBST(LIBXML_LIBS)
AC_DEFINE(WITH_CONCHECK, 1, [Define if you want connectivity checking
support])
else
AC_DEFINE(WITH_CONCHECK, 0, [Define if you want connectivity checking
support])
diff --git a/src/Makefile.am b/src/Makefile.am
index 9d306a7..54eaea9 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -316,7 +316,7 @@ NetworkManager_CPPFLAGS = \
-DARP_DEBUG
if WITH_CONCHECK
-NetworkManager_CPPFLAGS += $(LIBSOUP_CFLAGS)
+NetworkManager_CPPFLAGS += $(LIBSOUP_CFLAGS) $(LIBXML_CFLAGS)
endif
if SESSION_TRACKING_CK
@@ -350,7 +350,7 @@ NetworkManager_LDADD = \
$(LIBDL)
if WITH_CONCHECK
-NetworkManager_LDADD += $(LIBSOUP_LIBS)
+NetworkManager_LDADD += $(LIBSOUP_LIBS) $(LIBXML_LIBS)
endif
NetworkManager_LDFLAGS = -rdynamic
--
1.8.1.2
___
networkmanager-list mailing list
networkmanager-list@gnome.org
https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
On Mon, 2013-02-11 at 12:09 -0200, Jonh Wendell wrote:
From: Jonh Wendell jonh.wend...@oiwifi.com.br
libsoup already depends on libxml2 but we need to explicitly link
to it.
At least we already theoretically required it; though is it possible to
use GMarkup here instead of libxml2? GMarkup would be somewhat simpler,
though it's only a subset.
Dan
Signed-off-by: Jonh Wendell jonh.wend...@oiwifi.com.br
---
configure.ac| 14 +++---
src/Makefile.am | 4 ++--
2 files changed, 13 insertions(+), 5 deletions(-)
diff --git a/configure.ac b/configure.ac
index 916561a..49a6203 100644
--- a/configure.ac
+++ b/configure.ac
@@ -573,15 +573,23 @@ fi
AC_DEFINE_UNQUOTED(KERNEL_FIRMWARE_DIR, $KERNEL_FIRMWARE_DIR, [Define to
path of the kernel firmware directory])
AC_SUBST(KERNEL_FIRMWARE_DIR)
+PKG_CHECK_MODULES(LIBXML, [libxml-2.0], [have_libxml=yes],[have_libxml=no])
PKG_CHECK_MODULES(LIBSOUP, [libsoup-2.4 = 2.26],
[have_libsoup=yes],[have_libsoup=no])
+if test x$have_libsoup = xyes -a x$have_libxml = xyes; then
+can_have_connectivity=yes
+else
+can_have_connectivity=no
+fi
AC_ARG_ENABLE(concheck, AS_HELP_STRING([--enable-concheck], [enable
connectivity checking support]),
- [enable_concheck=${enableval}],
[enable_concheck=${have_libsoup}])
+ [enable_concheck=${enableval}],
[enable_concheck=${can_have_connectivity}])
if (test ${enable_concheck} = yes); then
- if test x$have_libsoup = xno; then
- AC_MSG_ERROR(Connectivity checking requires libsoup development
headers)
+ if test x$can_have_connectivity = xno; then
+ AC_MSG_ERROR(Connectivity checking requires libsoup and libxml2
development headers)
fi
AC_SUBST(LIBSOUP_CFLAGS)
AC_SUBST(LIBSOUP_LIBS)
+ AC_SUBST(LIBXML_CFLAGS)
+ AC_SUBST(LIBXML_LIBS)
AC_DEFINE(WITH_CONCHECK, 1, [Define if you want connectivity checking
support])
else
AC_DEFINE(WITH_CONCHECK, 0, [Define if you want connectivity checking
support])
diff --git a/src/Makefile.am b/src/Makefile.am
index 9d306a7..54eaea9 100644
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -316,7 +316,7 @@ NetworkManager_CPPFLAGS = \
-DARP_DEBUG
if WITH_CONCHECK
-NetworkManager_CPPFLAGS += $(LIBSOUP_CFLAGS)
+NetworkManager_CPPFLAGS += $(LIBSOUP_CFLAGS) $(LIBXML_CFLAGS)
endif
if SESSION_TRACKING_CK
@@ -350,7 +350,7 @@ NetworkManager_LDADD = \
$(LIBDL)
if WITH_CONCHECK
-NetworkManager_LDADD += $(LIBSOUP_LIBS)
+NetworkManager_LDADD += $(LIBSOUP_LIBS) $(LIBXML_LIBS)
endif
NetworkManager_LDFLAGS = -rdynamic
___
networkmanager-list mailing list
networkmanager-list@gnome.org
https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
On Mon, 2013-02-11 at 17:10 +0100, Bastien Nocera wrote: On Mon, 2013-02-11 at 10:06 -0600, Dan Williams wrote: On Mon, 2013-02-11 at 12:09 -0200, Jonh Wendell wrote: From: Jonh Wendell jonh.wend...@oiwifi.com.br libsoup already depends on libxml2 but we need to explicitly link to it. At least we already theoretically required it; though is it possible to use GMarkup here instead of libxml2? GMarkup would be somewhat simpler, though it's only a subset. Given how it's used, there's probably little reason this couldn't be a regexp. Both GMarkup and libxml2 would choke on broken, slightly broken, and very very broken HTML files. Yeah, and I've heard that for example, some hotspots literally just append raw XML to the end of the HTTP request outside the XML. I think we need to be somewhat more robust here and XML parsing may not be the way to get there? Also, we may need to add special cases for various hotspots, which might require regex and not just XML parsing. Dan ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
On Mon, 2013-02-11 at 10:06 -0600, Dan Williams wrote: On Mon, 2013-02-11 at 12:09 -0200, Jonh Wendell wrote: From: Jonh Wendell jonh.wend...@oiwifi.com.br libsoup already depends on libxml2 but we need to explicitly link to it. At least we already theoretically required it; though is it possible to use GMarkup here instead of libxml2? GMarkup would be somewhat simpler, though it's only a subset. Given how it's used, there's probably little reason this couldn't be a regexp. Both GMarkup and libxml2 would choke on broken, slightly broken, and very very broken HTML files. ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
In these patches I want to fix the 511-http-status. As it's something new, of course most hotspots don't use that (including my employer). Almost all of them rely on 30X Moved with help of the Wispr 'pseudo-protocol'. It's on my TODO list to work on those scenarios. That would touch only code in NMConnectivity object. Indeed, for that cases, we would use xml parsing as wispr is xml. 2013/2/11 Dan Williams d...@redhat.com On Mon, 2013-02-11 at 17:10 +0100, Bastien Nocera wrote: On Mon, 2013-02-11 at 10:06 -0600, Dan Williams wrote: On Mon, 2013-02-11 at 12:09 -0200, Jonh Wendell wrote: From: Jonh Wendell jonh.wend...@oiwifi.com.br libsoup already depends on libxml2 but we need to explicitly link to it. At least we already theoretically required it; though is it possible to use GMarkup here instead of libxml2? GMarkup would be somewhat simpler, though it's only a subset. Given how it's used, there's probably little reason this couldn't be a regexp. Both GMarkup and libxml2 would choke on broken, slightly broken, and very very broken HTML files. Yeah, and I've heard that for example, some hotspots literally just append raw XML to the end of the HTTP request outside the XML. I think we need to be somewhat more robust here and XML parsing may not be the way to get there? Also, we may need to add special cases for various hotspots, which might require regex and not just XML parsing. Dan -- Jonh Wendell http://www.bani.com.br ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
Re: [PATCH 2/5] connectivity: Add libxml2 as a dependency
On Mon, 2013-02-11 at 15:30 -0200, Jonh Wendell wrote: In these patches I want to fix the 511-http-status. As it's something new, of course most hotspots don't use that (including my employer). Almost all of them rely on 30X Moved with help of the Wispr 'pseudo-protocol'. It's on my TODO list to work on those scenarios. That would touch only code in NMConnectivity object. Indeed, for that cases, we would use xml parsing as wispr is xml. Thanks for the clarification. In any case, would a regex be possible here instead of the XML for now? It might be less code and would certainly be less error-prone when checking broken HTML which is quite common. Dan 2013/2/11 Dan Williams d...@redhat.com On Mon, 2013-02-11 at 17:10 +0100, Bastien Nocera wrote: On Mon, 2013-02-11 at 10:06 -0600, Dan Williams wrote: On Mon, 2013-02-11 at 12:09 -0200, Jonh Wendell wrote: From: Jonh Wendell jonh.wend...@oiwifi.com.br libsoup already depends on libxml2 but we need to explicitly link to it. At least we already theoretically required it; though is it possible to use GMarkup here instead of libxml2? GMarkup would be somewhat simpler, though it's only a subset. Given how it's used, there's probably little reason this couldn't be a regexp. Both GMarkup and libxml2 would choke on broken, slightly broken, and very very broken HTML files. Yeah, and I've heard that for example, some hotspots literally just append raw XML to the end of the HTTP request outside the XML. I think we need to be somewhat more robust here and XML parsing may not be the way to get there? Also, we may need to add special cases for various hotspots, which might require regex and not just XML parsing. Dan -- Jonh Wendell http://www.bani.com.br ___ networkmanager-list mailing list networkmanager-list@gnome.org https://mail.gnome.org/mailman/listinfo/networkmanager-list
