Ok, so if I well understood what you mean is writing this inside the
dump/pom.xml :
<jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
<jvmArg>-Dhttp.proxyPort=80</jvmArg>
<jvmArg>-Dhttp.proxyUser=foo</jvmArg>
<jvmArg>-Dhttp.proxyPassword=bar</jvmArg>
And retrieve the value of http.proxyUser and http.proxyPassword with this :
String user = System.getProperty("http.proxyUser", "");
String password = System.getProperty("http.proxyPassword", "");
For using them inside the ProxyAuthenticator class. If it's that so ok I'm
totally agree with, and I will modify my code for taking this way :-)
Best.
Julien.
2013/5/2 Jona Christopher Sahnwaldt <[email protected]>
> On 2 May 2013 17:34, Julien Plu <[email protected]>
> wrote:
> >>Use the system properties http.proxyUser and http.proxyPassword.
> >>Usually, system properties are problematic, but in this case, they're
> >>ok: 1. we use system properties for the proxy settings anyway, 2. we
> >>don't have to change the download config, and 3. we can more easily
> >>reuse the solution elsewhere.
> >
> > I think I badly explained myself. You have to give your proxy logs, so
> you
> > have to write them somewhere, and into the code is the worst solution
> ever,
> > and if you pass them through the jvm args like "-Dhttp.proxyUser=xxx
> > -Dhttp.proxyPassword=xxx" it doesn't works.
>
> I know that simply setting these system properties doesn't help. You
> need the Authenticator.setAuthenticator(...) code that we're
> discussing here. The question is: how does that code get the username
> and password?
>
> One possibility is what you did - add config parameters, read them,
> pass them to the code. That works, but wherever you want to enable
> proxy authentication, you have to change the code that handles the
> configuration.
>
> Another possibility is: the code reads the system properties
> http.proxyUser and http.proxyPassword:
>
> String user = System.getProperty("http.proxyUser", "");
> String password = System.getProperty("http.proxyPassword", "");
>
> In Java, you can set and read whichever system property you want.
> (Well, it depends on your security settings, but that doesn't matter
> in our case.) These two properties are ignored by the Java classes
> that handle proxy requests, that's correct. But that doesn't mean we
> can't use them for our own purposes.
>
> For more details, please see the code at
>
>
> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
>
> I think that's the best and safest implementation of setting the
> authenticator. There's more to this issue than it seems. If I
> understand correctly, the Authenticator is used for *all* HTTP
> requests, not just for proxy requests.
>
> > I tested it. So a good solution
> > is to write them into a property file.
>
> The problem with this is that we have to change the configuration of
> every piece of code in the framework that uses HTTP. When the code
> that sets the authenticator reads the system properties, that's not
> necessary.
>
> System properties are often not a good choice for configuration,
> mostly because they affect the whole system and there is no way for
> different modules within one JVM to use different settings. But in our
> case, the system is rather small - there are no sub-modules - and it's
> very unlikely that we want different proxy settings.
>
> Additionally, the user has to configure the proxy host and port in the
> pom.xml anyway. I think it's a lot more intuitive to have
>
> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
> <jvmArg>-Dhttp.proxyPort=80</jvmArg>
> <jvmArg>-Dhttp.proxyUser=foo</jvmArg>
> <jvmArg>-Dhttp.proxyPassword=bar</jvmArg>
>
> in pom.xml than to have
>
> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
> <jvmArg>-Dhttp.proxyPort=80</jvmArg>
>
> in pom.xml and
>
> proxyUser=foo
> proxyPassword=bar
>
> in some other configuration file.
>
> As for the proxy settings for the extraction: I was wrong, you
> probably do need them there too. We don't really make http calls, but
> some code deep down in Java's XML handling probably does. It's silly
> and we should turn it off, but that requires a bit of coding for which
> we didn't have time yet.
>
> http://sourceforge.net/mailarchive/message.php?msg_id=30590876
>
> JC
>
> >
> > Next, create a "util" class is, indeed, a good way to reuse it easily.
> >
> > Best.
> >
> > Julien.
> >
> >
> > 2013/5/2 Jona Christopher Sahnwaldt <[email protected]>
> >>
> >> P.S.: ProxyAuthentication is a good class name as well. :-)
> >>
> >> On 2 May 2013 17:04, Jona Christopher Sahnwaldt <[email protected]>
> wrote:
> >> > On 2 May 2013 12:55, Julien Plu <[email protected]>
> >> > wrote:
> >> >> Ok, now that works like a charm. I added just two properties inside
> >> >> "download.minimal.properties" file which are : "user" and "password"
> >> >> and the
> >> >> authentication is made only is these parameters are set. And I
> created
> >> >> a
> >> >> class "ProxyAuthentication.scala".
> >> >>
> >> >
> >> > A few recommendations:
> >> >
> >> > - Use the system properties http.proxyUser and http.proxyPassword.
> >> > Usually, system properties are problematic, but in this case, they're
> >> > ok: 1. we use system properties for the proxy settings anyway, 2. we
> >> > don't have to change the download config, and 3. we can more easily
> >> > reuse the solution elsewhere. See below.
> >> >
> >> > - I just googled a bit and think we should make the implementation a
> >> > bit safer:
> >> >
> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
> >> >
> >> > - Put the code in a new util class ProxyUtils.scala in the core module
> >> > in the util package
> >> >
> >> >
> https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction/util
> >> > , or you can just use ProxyUtils.java and create a new util package in
> >> >
> >> >
> https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction
> >> > , then you won't have to port the code to Scala.
> >> >
> >> > We make HTTP calls in a few other places, so we will have to call the
> >> > method from there as well, so we should have an implementation in the
> >> > core module.
> >> >
> >> >> I have to do the same thing for "Extraction.scala" class or this
> class
> >> >> doesn't need an internet connexion ?
> >> >
> >> > I'm pretty sure it doesn't. I don't think we make any requests during
> >> > extraction.
> >> >
> >> > But there are other places. You can look in the pom.xml files, they
> >> > should contain Scala launchers for all main classes that we have. For
> >> > example, all the launchers in core/pom.xml download some stuff over
> >> > HTTP, so if you want you can add a call to your new method in each of
> >> > their main() methods. When your code uses the system properties, you
> >> > just have to add one line: ProxyUtils.configureProxyAuthenticator();
> >> >
> >> > Cheers,
> >> > JC
> >> >
> >> >
> >> >
> >> >>
> >> >>
> >> >> 2013/4/30 Julien Plu <[email protected]>
> >> >>>
> >> >>> That's a really good idea ! I will do it thursday because now I
> >> >>> finished
> >> >>> my working day and tomorrow is a national day off in France.
> >> >>>
> >> >>> I have also try to solve some bugs in the extraction with
> separators.
> >> >>>
> >> >>> Best.
> >> >>>
> >> >>> Julien.
> >> >>>
> >> >>>
> >> >>> 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
> >> >>>>
> >> >>>>
> >> >>>>
> >> >>>>
> https://github.com/dbpedia/extraction-framework/blob/master/dump/src/main/scala/org/dbpedia/extraction/dump/download/Download.scala
> >> >>>>
> >> >>>> It would be cool if you send a pull request when you are done! As a
> >> >>>> first step, you can simply copy & paste the code from SO and
> >> >>>> hard-code
> >> >>>> your settings to test if that solves your problem at all, but it
> >> >>>> would
> >> >>>> be nice if you could make the code a little smarter so others can
> use
> >> >>>> it: call System.getProperty("http.proxyUser") and "...Password" and
> >> >>>> only install the authenticator if both are set. Good luck!
> >> >>>>
> >> >>>> On 30 April 2013 19:47, Julien Plu
> >> >>>> <[email protected]>
> >> >>>> wrote:
> >> >>>> > Ok so apparently it's normal that it's doesn't works with command
> >> >>>> > line,
> >> >>>> > I
> >> >>>> > have to modify the main. So which file contains the main
> function ?
> >> >>>> >
> >> >>>> > Best.
> >> >>>> >
> >> >>>> > Julien.
> >> >>>> >
> >> >>>> >
> >> >>>> > 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
> >> >>>> >>
> >> >>>> >> Maybe you have to tweak the code a bit:
> >> >>>> >>
> >> >>>> >>
> >> >>>> >>
> >> >>>> >>
> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java
> >> >>>> >>
> >> >>>> >> On 30 April 2013 19:28, Julien Plu
> >> >>>> >> <[email protected]>
> >> >>>> >> wrote:
> >> >>>> >> > Doesn't works :-(
> >> >>>> >> >
> >> >>>> >> > Best
> >> >>>> >> >
> >> >>>> >> > Julien.
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >> > 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
> >> >>>> >> >>
> >> >>>> >> >> Hi Julien,
> >> >>>> >> >>
> >> >>>> >> >> try adding the settings for https as well, e.g.
> >> >>>> >> >>
> >> >>>> >> >> <jvmArg>-Dhttps.proxyHost=myproxyhost</jvmArg>
> >> >>>> >> >>
> >> >>>> >> >> Might help, I'm not sure.
> >> >>>> >> >>
> >> >>>> >> >> JC
> >> >>>> >> >>
> >> >>>> >> >> On 30 April 2013 16:57, Julien Plu
> >> >>>> >> >> <[email protected]>
> >> >>>> >> >> wrote:
> >> >>>> >> >> > Hi,
> >> >>>> >> >> >
> >> >>>> >> >> > I try to use the extraction framework on my Windows
> machine,
> >> >>>> >> >> > but
> >> >>>> >> >> > I
> >> >>>> >> >> > fall
> >> >>>> >> >> > on a
> >> >>>> >> >> > 407 http proxy authentication error during the dumps
> >> >>>> >> >> > download. I
> >> >>>> >> >> > put
> >> >>>> >> >> > these
> >> >>>> >> >> > lines inside the dump/pom.xml :
> >> >>>> >> >> >
> >> >>>> >> >> > <jvmArg>-Dhttp.proxyHost=myproxyhost</jvmArg>
> >> >>>> >> >> > <jvmArg>-Dhttp.proxyPort=port</jvmArg>
> >> >>>> >> >> > <jvmArg>-Dhttp.proxyUser=myuser</jvmArg>
> >> >>>> >> >> > <jvmArg>-Dhttp.proxyPassword=mypass</jvmArg>
> >> >>>> >> >> >
> >> >>>> >> >> > Even this :
> >> >>>> >> >> >
> >> >>>> >> >> > <jvmArg>-Djava.net.useSystemProxies=true</jvmArg>
> >> >>>> >> >> >
> >> >>>> >> >> > Doesn't work.
> >> >>>> >> >> >
> >> >>>> >> >> > There is something more to use ?
> >> >>>> >> >> >
> >> >>>> >> >> > Best.
> >> >>>> >> >> >
> >> >>>> >> >> > Julien.
> >> >>>> >> >> >
> >> >>>> >> >> >
> >> >>>> >> >> >
> >> >>>> >> >> >
> >> >>>> >> >> >
> >> >>>> >> >> >
> ------------------------------------------------------------------------------
> >> >>>> >> >> > Introducing AppDynamics Lite, a free troubleshooting tool
> for
> >> >>>> >> >> > Java/.NET
> >> >>>> >> >> > Get 100% visibility into your production application - at
> no
> >> >>>> >> >> > cost.
> >> >>>> >> >> > Code-level diagnostics for performance bottlenecks with <2%
> >> >>>> >> >> > overhead
> >> >>>> >> >> > Download for free and get started troubleshooting in
> minutes.
> >> >>>> >> >> > http://p.sf.net/sfu/appdyn_d2d_ap1
> >> >>>> >> >> > _______________________________________________
> >> >>>> >> >> > Dbpedia-discussion mailing list
> >> >>>> >> >> > [email protected]
> >> >>>> >> >> >
> >> >>>> >> >> >
> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
> >> >>>> >> >> >
> >> >>>> >> >
> >> >>>> >> >
> >> >>>> >
> >> >>>> >
> >> >>>
> >> >>>
> >> >>
> >
> >
>
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Dbpedia-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
