I will port your code in Scala :-)
Best.
Julien.
2013/5/2 Jona Christopher Sahnwaldt <[email protected]>
> On 2 May 2013 20:58, Julien Plu <[email protected]>
> wrote:
> > Ok, so if I well understood what you mean is writing this inside the
> > dump/pom.xml :
> >
> > <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
> > <jvmArg>-Dhttp.proxyPort=80</jvmArg>
> > <jvmArg>-Dhttp.proxyUser=foo</jvmArg>
> > <jvmArg>-Dhttp.proxyPassword=bar</jvmArg>
> >
> > And retrieve the value of http.proxyUser and http.proxyPassword with
> this :
> >
> > String user = System.getProperty("http.proxyUser", "");
> > String password = System.getProperty("http.proxyPassword", "");
> >
> > For using them inside the ProxyAuthenticator class. If it's that so ok
> I'm
> > totally agree with, and I will modify my code for taking this way :-)
>
> Exactly. :-)
>
> As I said, you should look at the code at
>
> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
> There are some security issues. Basically, you can just copy that
> code, or port it to Scala if you want.
>
> >
> > Best.
> >
> > Julien.
> >
> >
> > 2013/5/2 Jona Christopher Sahnwaldt <[email protected]>
> >>
> >> On 2 May 2013 17:34, Julien Plu <[email protected]>
> >> wrote:
> >> >>Use the system properties http.proxyUser and http.proxyPassword.
> >> >>Usually, system properties are problematic, but in this case, they're
> >> >>ok: 1. we use system properties for the proxy settings anyway, 2. we
> >> >>don't have to change the download config, and 3. we can more easily
> >> >>reuse the solution elsewhere.
> >> >
> >> > I think I badly explained myself. You have to give your proxy logs, so
> >> > you
> >> > have to write them somewhere, and into the code is the worst solution
> >> > ever,
> >> > and if you pass them through the jvm args like "-Dhttp.proxyUser=xxx
> >> > -Dhttp.proxyPassword=xxx" it doesn't works.
> >>
> >> I know that simply setting these system properties doesn't help. You
> >> need the Authenticator.setAuthenticator(...) code that we're
> >> discussing here. The question is: how does that code get the username
> >> and password?
> >>
> >> One possibility is what you did - add config parameters, read them,
> >> pass them to the code. That works, but wherever you want to enable
> >> proxy authentication, you have to change the code that handles the
> >> configuration.
> >>
> >> Another possibility is: the code reads the system properties
> >> http.proxyUser and http.proxyPassword:
> >>
> >> String user = System.getProperty("http.proxyUser", "");
> >> String password = System.getProperty("http.proxyPassword", "");
> >>
> >> In Java, you can set and read whichever system property you want.
> >> (Well, it depends on your security settings, but that doesn't matter
> >> in our case.) These two properties are ignored by the Java classes
> >> that handle proxy requests, that's correct. But that doesn't mean we
> >> can't use them for our own purposes.
> >>
> >> For more details, please see the code at
> >>
> >>
> >>
> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
> >>
> >> I think that's the best and safest implementation of setting the
> >> authenticator. There's more to this issue than it seems. If I
> >> understand correctly, the Authenticator is used for *all* HTTP
> >> requests, not just for proxy requests.
> >>
> >> > I tested it. So a good solution
> >> > is to write them into a property file.
> >>
> >> The problem with this is that we have to change the configuration of
> >> every piece of code in the framework that uses HTTP. When the code
> >> that sets the authenticator reads the system properties, that's not
> >> necessary.
> >>
> >> System properties are often not a good choice for configuration,
> >> mostly because they affect the whole system and there is no way for
> >> different modules within one JVM to use different settings. But in our
> >> case, the system is rather small - there are no sub-modules - and it's
> >> very unlikely that we want different proxy settings.
> >>
> >> Additionally, the user has to configure the proxy host and port in the
> >> pom.xml anyway. I think it's a lot more intuitive to have
> >>
> >> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
> >> <jvmArg>-Dhttp.proxyPort=80</jvmArg>
> >> <jvmArg>-Dhttp.proxyUser=foo</jvmArg>
> >> <jvmArg>-Dhttp.proxyPassword=bar</jvmArg>
> >>
> >> in pom.xml than to have
> >>
> >> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
> >> <jvmArg>-Dhttp.proxyPort=80</jvmArg>
> >>
> >> in pom.xml and
> >>
> >> proxyUser=foo
> >> proxyPassword=bar
> >>
> >> in some other configuration file.
> >>
> >> As for the proxy settings for the extraction: I was wrong, you
> >> probably do need them there too. We don't really make http calls, but
> >> some code deep down in Java's XML handling probably does. It's silly
> >> and we should turn it off, but that requires a bit of coding for which
> >> we didn't have time yet.
> >>
> >> http://sourceforge.net/mailarchive/message.php?msg_id=30590876
> >>
> >> JC
> >>
> >> >
> >> > Next, create a "util" class is, indeed, a good way to reuse it easily.
> >> >
> >> > Best.
> >> >
> >> > Julien.
> >> >
> >> >
> >> > 2013/5/2 Jona Christopher Sahnwaldt <[email protected]>
> >> >>
> >> >> P.S.: ProxyAuthentication is a good class name as well. :-)
> >> >>
> >> >> On 2 May 2013 17:04, Jona Christopher Sahnwaldt <[email protected]>
> >> >> wrote:
> >> >> > On 2 May 2013 12:55, Julien Plu <
> [email protected]>
> >> >> > wrote:
> >> >> >> Ok, now that works like a charm. I added just two properties
> inside
> >> >> >> "download.minimal.properties" file which are : "user" and
> "password"
> >> >> >> and the
> >> >> >> authentication is made only is these parameters are set. And I
> >> >> >> created
> >> >> >> a
> >> >> >> class "ProxyAuthentication.scala".
> >> >> >>
> >> >> >
> >> >> > A few recommendations:
> >> >> >
> >> >> > - Use the system properties http.proxyUser and http.proxyPassword.
> >> >> > Usually, system properties are problematic, but in this case,
> they're
> >> >> > ok: 1. we use system properties for the proxy settings anyway, 2.
> we
> >> >> > don't have to change the download config, and 3. we can more easily
> >> >> > reuse the solution elsewhere. See below.
> >> >> >
> >> >> > - I just googled a bit and think we should make the implementation
> a
> >> >> > bit safer:
> >> >> >
> >> >> >
> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
> >> >> >
> >> >> > - Put the code in a new util class ProxyUtils.scala in the core
> >> >> > module
> >> >> > in the util package
> >> >> >
> >> >> >
> >> >> >
> https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction/util
> >> >> > , or you can just use ProxyUtils.java and create a new util package
> >> >> > in
> >> >> >
> >> >> >
> >> >> >
> https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction
> >> >> > , then you won't have to port the code to Scala.
> >> >> >
> >> >> > We make HTTP calls in a few other places, so we will have to call
> the
> >> >> > method from there as well, so we should have an implementation in
> the
> >> >> > core module.
> >> >> >
> >> >> >> I have to do the same thing for "Extraction.scala" class or this
> >> >> >> class
> >> >> >> doesn't need an internet connexion ?
> >> >> >
> >> >> > I'm pretty sure it doesn't. I don't think we make any requests
> during
> >> >> > extraction.
> >> >> >
> >> >> > But there are other places. You can look in the pom.xml files, they
> >> >> > should contain Scala launchers for all main classes that we have.
> For
> >> >> > example, all the launchers in core/pom.xml download some stuff over
> >> >> > HTTP, so if you want you can add a call to your new method in each
> of
> >> >> > their main() methods. When your code uses the system properties,
> you
> >> >> > just have to add one line:
> ProxyUtils.configureProxyAuthenticator();
> >> >> >
> >> >> > Cheers,
> >> >> > JC
> >> >> >
> >> >> >
> >> >> >
> >> >> >>
> >> >> >>
> >> >> >> 2013/4/30 Julien Plu <[email protected]>
> >> >> >>>
> >> >> >>> That's a really good idea ! I will do it thursday because now I
> >> >> >>> finished
> >> >> >>> my working day and tomorrow is a national day off in France.
> >> >> >>>
> >> >> >>> I have also try to solve some bugs in the extraction with
> >> >> >>> separators.
> >> >> >>>
> >> >> >>> Best.
> >> >> >>>
> >> >> >>> Julien.
> >> >> >>>
> >> >> >>>
> >> >> >>> 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> >> >> >>>>
> https://github.com/dbpedia/extraction-framework/blob/master/dump/src/main/scala/org/dbpedia/extraction/dump/download/Download.scala
> >> >> >>>>
> >> >> >>>> It would be cool if you send a pull request when you are done!
> As
> >> >> >>>> a
> >> >> >>>> first step, you can simply copy & paste the code from SO and
> >> >> >>>> hard-code
> >> >> >>>> your settings to test if that solves your problem at all, but it
> >> >> >>>> would
> >> >> >>>> be nice if you could make the code a little smarter so others
> can
> >> >> >>>> use
> >> >> >>>> it: call System.getProperty("http.proxyUser") and "...Password"
> >> >> >>>> and
> >> >> >>>> only install the authenticator if both are set. Good luck!
> >> >> >>>>
> >> >> >>>> On 30 April 2013 19:47, Julien Plu
> >> >> >>>> <[email protected]>
> >> >> >>>> wrote:
> >> >> >>>> > Ok so apparently it's normal that it's doesn't works with
> >> >> >>>> > command
> >> >> >>>> > line,
> >> >> >>>> > I
> >> >> >>>> > have to modify the main. So which file contains the main
> >> >> >>>> > function ?
> >> >> >>>> >
> >> >> >>>> > Best.
> >> >> >>>> >
> >> >> >>>> > Julien.
> >> >> >>>> >
> >> >> >>>> >
> >> >> >>>> > 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
> >> >> >>>> >>
> >> >> >>>> >> Maybe you have to tweak the code a bit:
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >>
> >> >> >>>> >>
> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java
> >> >> >>>> >>
> >> >> >>>> >> On 30 April 2013 19:28, Julien Plu
> >> >> >>>> >> <[email protected]>
> >> >> >>>> >> wrote:
> >> >> >>>> >> > Doesn't works :-(
> >> >> >>>> >> >
> >> >> >>>> >> > Best
> >> >> >>>> >> >
> >> >> >>>> >> > Julien.
> >> >> >>>> >> >
> >> >> >>>> >> >
> >> >> >>>> >> > 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
> >> >> >>>> >> >>
> >> >> >>>> >> >> Hi Julien,
> >> >> >>>> >> >>
> >> >> >>>> >> >> try adding the settings for https as well, e.g.
> >> >> >>>> >> >>
> >> >> >>>> >> >> <jvmArg>-Dhttps.proxyHost=myproxyhost</jvmArg>
> >> >> >>>> >> >>
> >> >> >>>> >> >> Might help, I'm not sure.
> >> >> >>>> >> >>
> >> >> >>>> >> >> JC
> >> >> >>>> >> >>
> >> >> >>>> >> >> On 30 April 2013 16:57, Julien Plu
> >> >> >>>> >> >> <[email protected]>
> >> >> >>>> >> >> wrote:
> >> >> >>>> >> >> > Hi,
> >> >> >>>> >> >> >
> >> >> >>>> >> >> > I try to use the extraction framework on my Windows
> >> >> >>>> >> >> > machine,
> >> >> >>>> >> >> > but
> >> >> >>>> >> >> > I
> >> >> >>>> >> >> > fall
> >> >> >>>> >> >> > on a
> >> >> >>>> >> >> > 407 http proxy authentication error during the dumps
> >> >> >>>> >> >> > download. I
> >> >> >>>> >> >> > put
> >> >> >>>> >> >> > these
> >> >> >>>> >> >> > lines inside the dump/pom.xml :
> >> >> >>>> >> >> >
> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyHost=myproxyhost</jvmArg>
> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyPort=port</jvmArg>
> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyUser=myuser</jvmArg>
> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyPassword=mypass</jvmArg>
> >> >> >>>> >> >> >
> >> >> >>>> >> >> > Even this :
> >> >> >>>> >> >> >
> >> >> >>>> >> >> > <jvmArg>-Djava.net.useSystemProxies=true</jvmArg>
> >> >> >>>> >> >> >
> >> >> >>>> >> >> > Doesn't work.
> >> >> >>>> >> >> >
> >> >> >>>> >> >> > There is something more to use ?
> >> >> >>>> >> >> >
> >> >> >>>> >> >> > Best.
> >> >> >>>> >> >> >
> >> >> >>>> >> >> > Julien.
> >> >> >>>> >> >> >
> >> >> >>>> >> >> >
> >> >> >>>> >> >> >
> >> >> >>>> >> >> >
> >> >> >>>> >> >> >
> >> >> >>>> >> >> >
> >> >> >>>> >> >> >
> ------------------------------------------------------------------------------
> >> >> >>>> >> >> > Introducing AppDynamics Lite, a free troubleshooting
> tool
> >> >> >>>> >> >> > for
> >> >> >>>> >> >> > Java/.NET
> >> >> >>>> >> >> > Get 100% visibility into your production application -
> at
> >> >> >>>> >> >> > no
> >> >> >>>> >> >> > cost.
> >> >> >>>> >> >> > Code-level diagnostics for performance bottlenecks with
> >> >> >>>> >> >> > <2%
> >> >> >>>> >> >> > overhead
> >> >> >>>> >> >> > Download for free and get started troubleshooting in
> >> >> >>>> >> >> > minutes.
> >> >> >>>> >> >> > http://p.sf.net/sfu/appdyn_d2d_ap1
> >> >> >>>> >> >> > _______________________________________________
> >> >> >>>> >> >> > Dbpedia-discussion mailing list
> >> >> >>>> >> >> > [email protected]
> >> >> >>>> >> >> >
> >> >> >>>> >> >> >
> >> >> >>>> >> >> >
> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
> >> >> >>>> >> >> >
> >> >> >>>> >> >
> >> >> >>>> >> >
> >> >> >>>> >
> >> >> >>>> >
> >> >> >>>
> >> >> >>>
> >> >> >>
> >> >
> >> >
> >
> >
>
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Dbpedia-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
