Done ! See the pull request that I submited.
Best.
Julien.
2013/5/2 Julien Plu <[email protected]>
> I will port your code in Scala :-)
>
> Best.
>
> Julien.
>
>
> 2013/5/2 Jona Christopher Sahnwaldt <[email protected]>
>
>> On 2 May 2013 20:58, Julien Plu <[email protected]>
>> wrote:
>> > Ok, so if I well understood what you mean is writing this inside the
>> > dump/pom.xml :
>> >
>> > <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
>> > <jvmArg>-Dhttp.proxyPort=80</jvmArg>
>> > <jvmArg>-Dhttp.proxyUser=foo</jvmArg>
>> > <jvmArg>-Dhttp.proxyPassword=bar</jvmArg>
>> >
>> > And retrieve the value of http.proxyUser and http.proxyPassword with
>> this :
>> >
>> > String user = System.getProperty("http.proxyUser", "");
>> > String password = System.getProperty("http.proxyPassword", "");
>> >
>> > For using them inside the ProxyAuthenticator class. If it's that so ok
>> I'm
>> > totally agree with, and I will modify my code for taking this way :-)
>>
>> Exactly. :-)
>>
>> As I said, you should look at the code at
>>
>> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
>> There are some security issues. Basically, you can just copy that
>> code, or port it to Scala if you want.
>>
>> >
>> > Best.
>> >
>> > Julien.
>> >
>> >
>> > 2013/5/2 Jona Christopher Sahnwaldt <[email protected]>
>> >>
>> >> On 2 May 2013 17:34, Julien Plu <[email protected]>
>> >> wrote:
>> >> >>Use the system properties http.proxyUser and http.proxyPassword.
>> >> >>Usually, system properties are problematic, but in this case, they're
>> >> >>ok: 1. we use system properties for the proxy settings anyway, 2. we
>> >> >>don't have to change the download config, and 3. we can more easily
>> >> >>reuse the solution elsewhere.
>> >> >
>> >> > I think I badly explained myself. You have to give your proxy logs,
>> so
>> >> > you
>> >> > have to write them somewhere, and into the code is the worst solution
>> >> > ever,
>> >> > and if you pass them through the jvm args like "-Dhttp.proxyUser=xxx
>> >> > -Dhttp.proxyPassword=xxx" it doesn't works.
>> >>
>> >> I know that simply setting these system properties doesn't help. You
>> >> need the Authenticator.setAuthenticator(...) code that we're
>> >> discussing here. The question is: how does that code get the username
>> >> and password?
>> >>
>> >> One possibility is what you did - add config parameters, read them,
>> >> pass them to the code. That works, but wherever you want to enable
>> >> proxy authentication, you have to change the code that handles the
>> >> configuration.
>> >>
>> >> Another possibility is: the code reads the system properties
>> >> http.proxyUser and http.proxyPassword:
>> >>
>> >> String user = System.getProperty("http.proxyUser", "");
>> >> String password = System.getProperty("http.proxyPassword", "");
>> >>
>> >> In Java, you can set and read whichever system property you want.
>> >> (Well, it depends on your security settings, but that doesn't matter
>> >> in our case.) These two properties are ignored by the Java classes
>> >> that handle proxy requests, that's correct. But that doesn't mean we
>> >> can't use them for our own purposes.
>> >>
>> >> For more details, please see the code at
>> >>
>> >>
>> >>
>> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
>> >>
>> >> I think that's the best and safest implementation of setting the
>> >> authenticator. There's more to this issue than it seems. If I
>> >> understand correctly, the Authenticator is used for *all* HTTP
>> >> requests, not just for proxy requests.
>> >>
>> >> > I tested it. So a good solution
>> >> > is to write them into a property file.
>> >>
>> >> The problem with this is that we have to change the configuration of
>> >> every piece of code in the framework that uses HTTP. When the code
>> >> that sets the authenticator reads the system properties, that's not
>> >> necessary.
>> >>
>> >> System properties are often not a good choice for configuration,
>> >> mostly because they affect the whole system and there is no way for
>> >> different modules within one JVM to use different settings. But in our
>> >> case, the system is rather small - there are no sub-modules - and it's
>> >> very unlikely that we want different proxy settings.
>> >>
>> >> Additionally, the user has to configure the proxy host and port in the
>> >> pom.xml anyway. I think it's a lot more intuitive to have
>> >>
>> >> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
>> >> <jvmArg>-Dhttp.proxyPort=80</jvmArg>
>> >> <jvmArg>-Dhttp.proxyUser=foo</jvmArg>
>> >> <jvmArg>-Dhttp.proxyPassword=bar</jvmArg>
>> >>
>> >> in pom.xml than to have
>> >>
>> >> <jvmArg>-Dhttp.proxyHost=proxy.server.com</jvmArg>
>> >> <jvmArg>-Dhttp.proxyPort=80</jvmArg>
>> >>
>> >> in pom.xml and
>> >>
>> >> proxyUser=foo
>> >> proxyPassword=bar
>> >>
>> >> in some other configuration file.
>> >>
>> >> As for the proxy settings for the extraction: I was wrong, you
>> >> probably do need them there too. We don't really make http calls, but
>> >> some code deep down in Java's XML handling probably does. It's silly
>> >> and we should turn it off, but that requires a bit of coding for which
>> >> we didn't have time yet.
>> >>
>> >> http://sourceforge.net/mailarchive/message.php?msg_id=30590876
>> >>
>> >> JC
>> >>
>> >> >
>> >> > Next, create a "util" class is, indeed, a good way to reuse it
>> easily.
>> >> >
>> >> > Best.
>> >> >
>> >> > Julien.
>> >> >
>> >> >
>> >> > 2013/5/2 Jona Christopher Sahnwaldt <[email protected]>
>> >> >>
>> >> >> P.S.: ProxyAuthentication is a good class name as well. :-)
>> >> >>
>> >> >> On 2 May 2013 17:04, Jona Christopher Sahnwaldt <[email protected]>
>> >> >> wrote:
>> >> >> > On 2 May 2013 12:55, Julien Plu <
>> [email protected]>
>> >> >> > wrote:
>> >> >> >> Ok, now that works like a charm. I added just two properties
>> inside
>> >> >> >> "download.minimal.properties" file which are : "user" and
>> "password"
>> >> >> >> and the
>> >> >> >> authentication is made only is these parameters are set. And I
>> >> >> >> created
>> >> >> >> a
>> >> >> >> class "ProxyAuthentication.scala".
>> >> >> >>
>> >> >> >
>> >> >> > A few recommendations:
>> >> >> >
>> >> >> > - Use the system properties http.proxyUser and http.proxyPassword.
>> >> >> > Usually, system properties are problematic, but in this case,
>> they're
>> >> >> > ok: 1. we use system properties for the proxy settings anyway, 2.
>> we
>> >> >> > don't have to change the download config, and 3. we can more
>> easily
>> >> >> > reuse the solution elsewhere. See below.
>> >> >> >
>> >> >> > - I just googled a bit and think we should make the
>> implementation a
>> >> >> > bit safer:
>> >> >> >
>> >> >> >
>> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java/16340273#16340273
>> >> >> >
>> >> >> > - Put the code in a new util class ProxyUtils.scala in the core
>> >> >> > module
>> >> >> > in the util package
>> >> >> >
>> >> >> >
>> >> >> >
>> https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction/util
>> >> >> > , or you can just use ProxyUtils.java and create a new util
>> package
>> >> >> > in
>> >> >> >
>> >> >> >
>> >> >> >
>> https://github.com/dbpedia/extraction-framework/tree/master/core/src/main/java/org/dbpedia/extraction
>> >> >> > , then you won't have to port the code to Scala.
>> >> >> >
>> >> >> > We make HTTP calls in a few other places, so we will have to call
>> the
>> >> >> > method from there as well, so we should have an implementation in
>> the
>> >> >> > core module.
>> >> >> >
>> >> >> >> I have to do the same thing for "Extraction.scala" class or this
>> >> >> >> class
>> >> >> >> doesn't need an internet connexion ?
>> >> >> >
>> >> >> > I'm pretty sure it doesn't. I don't think we make any requests
>> during
>> >> >> > extraction.
>> >> >> >
>> >> >> > But there are other places. You can look in the pom.xml files,
>> they
>> >> >> > should contain Scala launchers for all main classes that we have.
>> For
>> >> >> > example, all the launchers in core/pom.xml download some stuff
>> over
>> >> >> > HTTP, so if you want you can add a call to your new method in
>> each of
>> >> >> > their main() methods. When your code uses the system properties,
>> you
>> >> >> > just have to add one line:
>> ProxyUtils.configureProxyAuthenticator();
>> >> >> >
>> >> >> > Cheers,
>> >> >> > JC
>> >> >> >
>> >> >> >
>> >> >> >
>> >> >> >>
>> >> >> >>
>> >> >> >> 2013/4/30 Julien Plu <[email protected]>
>> >> >> >>>
>> >> >> >>> That's a really good idea ! I will do it thursday because now I
>> >> >> >>> finished
>> >> >> >>> my working day and tomorrow is a national day off in France.
>> >> >> >>>
>> >> >> >>> I have also try to solve some bugs in the extraction with
>> >> >> >>> separators.
>> >> >> >>>
>> >> >> >>> Best.
>> >> >> >>>
>> >> >> >>> Julien.
>> >> >> >>>
>> >> >> >>>
>> >> >> >>> 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> >> >> >>>>
>> https://github.com/dbpedia/extraction-framework/blob/master/dump/src/main/scala/org/dbpedia/extraction/dump/download/Download.scala
>> >> >> >>>>
>> >> >> >>>> It would be cool if you send a pull request when you are done!
>> As
>> >> >> >>>> a
>> >> >> >>>> first step, you can simply copy & paste the code from SO and
>> >> >> >>>> hard-code
>> >> >> >>>> your settings to test if that solves your problem at all, but
>> it
>> >> >> >>>> would
>> >> >> >>>> be nice if you could make the code a little smarter so others
>> can
>> >> >> >>>> use
>> >> >> >>>> it: call System.getProperty("http.proxyUser") and "...Password"
>> >> >> >>>> and
>> >> >> >>>> only install the authenticator if both are set. Good luck!
>> >> >> >>>>
>> >> >> >>>> On 30 April 2013 19:47, Julien Plu
>> >> >> >>>> <[email protected]>
>> >> >> >>>> wrote:
>> >> >> >>>> > Ok so apparently it's normal that it's doesn't works with
>> >> >> >>>> > command
>> >> >> >>>> > line,
>> >> >> >>>> > I
>> >> >> >>>> > have to modify the main. So which file contains the main
>> >> >> >>>> > function ?
>> >> >> >>>> >
>> >> >> >>>> > Best.
>> >> >> >>>> >
>> >> >> >>>> > Julien.
>> >> >> >>>> >
>> >> >> >>>> >
>> >> >> >>>> > 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
>> >> >> >>>> >>
>> >> >> >>>> >> Maybe you have to tweak the code a bit:
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> >> >> >>>> >>
>> http://stackoverflow.com/questions/1626549/authenticated-http-proxy-with-java
>> >> >> >>>> >>
>> >> >> >>>> >> On 30 April 2013 19:28, Julien Plu
>> >> >> >>>> >> <[email protected]>
>> >> >> >>>> >> wrote:
>> >> >> >>>> >> > Doesn't works :-(
>> >> >> >>>> >> >
>> >> >> >>>> >> > Best
>> >> >> >>>> >> >
>> >> >> >>>> >> > Julien.
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> > 2013/4/30 Jona Christopher Sahnwaldt <[email protected]>
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> Hi Julien,
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> try adding the settings for https as well, e.g.
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> <jvmArg>-Dhttps.proxyHost=myproxyhost</jvmArg>
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> Might help, I'm not sure.
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> JC
>> >> >> >>>> >> >>
>> >> >> >>>> >> >> On 30 April 2013 16:57, Julien Plu
>> >> >> >>>> >> >> <[email protected]>
>> >> >> >>>> >> >> wrote:
>> >> >> >>>> >> >> > Hi,
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> > I try to use the extraction framework on my Windows
>> >> >> >>>> >> >> > machine,
>> >> >> >>>> >> >> > but
>> >> >> >>>> >> >> > I
>> >> >> >>>> >> >> > fall
>> >> >> >>>> >> >> > on a
>> >> >> >>>> >> >> > 407 http proxy authentication error during the dumps
>> >> >> >>>> >> >> > download. I
>> >> >> >>>> >> >> > put
>> >> >> >>>> >> >> > these
>> >> >> >>>> >> >> > lines inside the dump/pom.xml :
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyHost=myproxyhost</jvmArg>
>> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyPort=port</jvmArg>
>> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyUser=myuser</jvmArg>
>> >> >> >>>> >> >> > <jvmArg>-Dhttp.proxyPassword=mypass</jvmArg>
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> > Even this :
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> > <jvmArg>-Djava.net.useSystemProxies=true</jvmArg>
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> > Doesn't work.
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> > There is something more to use ?
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> > Best.
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> > Julien.
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> >
>> ------------------------------------------------------------------------------
>> >> >> >>>> >> >> > Introducing AppDynamics Lite, a free troubleshooting
>> tool
>> >> >> >>>> >> >> > for
>> >> >> >>>> >> >> > Java/.NET
>> >> >> >>>> >> >> > Get 100% visibility into your production application -
>> at
>> >> >> >>>> >> >> > no
>> >> >> >>>> >> >> > cost.
>> >> >> >>>> >> >> > Code-level diagnostics for performance bottlenecks with
>> >> >> >>>> >> >> > <2%
>> >> >> >>>> >> >> > overhead
>> >> >> >>>> >> >> > Download for free and get started troubleshooting in
>> >> >> >>>> >> >> > minutes.
>> >> >> >>>> >> >> > http://p.sf.net/sfu/appdyn_d2d_ap1
>> >> >> >>>> >> >> > _______________________________________________
>> >> >> >>>> >> >> > Dbpedia-discussion mailing list
>> >> >> >>>> >> >> > [email protected]
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >> >
>> https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
>> >> >> >>>> >> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >> >
>> >> >> >>>> >
>> >> >> >>>> >
>> >> >> >>>
>> >> >> >>>
>> >> >> >>
>> >> >
>> >> >
>> >
>> >
>>
>
>
------------------------------------------------------------------------------
Get 100% visibility into Java/.NET code with AppDynamics Lite
It's a free troubleshooting tool designed for production
Get down to code-level detail for bottlenecks, with <2% overhead.
Download for free and get started troubleshooting in minutes.
http://p.sf.net/sfu/appdyn_d2d_ap2
_______________________________________________
Dbpedia-discussion mailing list
[email protected]
https://lists.sourceforge.net/lists/listinfo/dbpedia-discussion
