GitHub user kamir opened a pull request:
https://github.com/apache/incubator-toree/pull/118
Allow also URLs with additional parameters
Not all URLs end with JAR or ZIP, e.g., if we load content from an HTTPFS
Service.
Examples for testing: (a,c,d work as expeced) (b => more logic needed to
create a JAR name from this URL)
import java.io.{File, PrintStream}
import java.net.URL
import java.nio.file.{Files, Paths}
def getFileFromLocation(location: String): String = {
val url = new URL(location)
val file = url.getFile.split("/")
if (file.length > 0) {
if ( file.last.contains('?') ) {
file.last.split('?')(0)
}
else {
file.last
}
} else {
""
}
}
val fnA =
"http://cdsw-mk4-1.gce.cloudera.com:8888/filebrowser/download=/user/systest/TOOLBOX/SparkShellUtilities.jar?user.name=123";
val a = getFileFromLocation(fnA)
a
val fnB =
"https://oss.sonatype.org/service/local/artifact/maven/content?r=snapshots&g=org.gephi&a=gephi-toolkit&v=0.9.2-SNAPSHOT&c=all";
val b = getFileFromLocation(fnB)
b
val fnC =
"http://cdsw-mk4-1.gce.cloudera.com:8888/filebrowser/download=/user/systest/TOOLBOX/SparkShellUtilities.jar";
val c = getFileFromLocation(fnC)
c
val fnD =
"http://cdsw-mk4-1.gce.cloudera.com:8888/filebrowser/download=/user/systest/TOOLBOX/SparkShellUtilities.jar?user.name=123&doas=123456";
val d = getFileFromLocation(fnD)
d
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/kamir/incubator-toree patch-1
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/incubator-toree/pull/118.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #118
----
commit d99cdaa1992c15d8e7a43c3446dd17b901d2fa70
Author: Mirko Kämpf <[email protected]>
Date: 2017-03-23T11:07:04Z
Allow also URLs with additional parameters
Not all URLs end with JAR or ZIP, e.g., if we load content from an HTTPFS
Service.
Examples for testing: (a,c,d work as expeced) (b => more logic needed to
create a JAR name from this URL)
import java.io.{File, PrintStream}
import java.net.URL
import java.nio.file.{Files, Paths}
def getFileFromLocation(location: String): String = {
val url = new URL(location)
val file = url.getFile.split("/")
if (file.length > 0) {
if ( file.last.contains('?') ) {
file.last.split('?')(0)
}
else {
file.last
}
} else {
""
}
}
val fnA =
"http://cdsw-mk4-1.gce.cloudera.com:8888/filebrowser/download=/user/systest/TOOLBOX/SparkShellUtilities.jar?user.name=123";
val a = getFileFromLocation(fnA)
a
val fnB =
"https://oss.sonatype.org/service/local/artifact/maven/content?r=snapshots&g=org.gephi&a=gephi-toolkit&v=0.9.2-SNAPSHOT&c=all";
val b = getFileFromLocation(fnB)
b
val fnC =
"http://cdsw-mk4-1.gce.cloudera.com:8888/filebrowser/download=/user/systest/TOOLBOX/SparkShellUtilities.jar";
val c = getFileFromLocation(fnC)
c
val fnD =
"http://cdsw-mk4-1.gce.cloudera.com:8888/filebrowser/download=/user/systest/TOOLBOX/SparkShellUtilities.jar?user.name=123&doas=123456";
val d = getFileFromLocation(fnD)
d
----
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at [email protected] or file a JIRA ticket
with INFRA.
---
