[
https://issues.apache.org/jira/browse/SPARK-10863?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005019#comment-15005019
]
Oscar D. Lara Yejas edited comment on SPARK-10863 at 11/14/15 1:19 AM:
-----------------------------------------------------------------------
[~felixcheung] I think a solution to all three issues would be to implement
wrapper classes for complex types. For example, for StructType, we could have
something like the small prototype I implemented below (still very raw, but
just to give you an idea). I'd also need to implement class Row accordingly to
handle the values.
I could do something similar for MapType, and I believe a list/vector should
suffice for ArrayType.
Thoughts?
{code:title=Struct.R|borderStyle=solid}
# You can actually just copy and paste the code below on R to run it
setClass("StructField",
representation(
name = "character",
type = "character"
))
# A Struct is a set of StructField objects, modeled as an environment
setClass("Struct",
representation(
struct = "environment"
))
# Initialize a Struct from a list of StructField objects
setMethod("initialize", signature = "Struct", definition=
function(.Object, fields) {
lapply(fields, function(field) {
.Object@struct[[field@name]] <- field
})
return(.Object)
})
# Overwrite [[ operator to access the environment directly
setGeneric("[[")
setMethod("[[", signature="Struct", definition=
function(x, i) {
return(x@struct[[i]])
})
# Overwrite [[<- operator to access the environment directly
setGeneric("[[<-")
setMethod("[[<-", signature="Struct", definition=
function(x, i, value) {
if (class(value) == "StructField") {
x@struct[[i]] <- value
}
return(x)
})
field1 <- new("StructField", name="x", type="numeric")
field2 <- new("StructField", name="y", type="character")
s <- new("Struct", fields=list(field1, field2))
s[["x"]]
s[["z"]] <- new("StructField", name="z", type="logical")
{code}
was (Author: olarayej):
[~felixcheung] I think a solution to all three issues would be to implement
wrapper classes for complex types. For example, for StructType, we could have
something like the small prototype I implemented below (still very raw, but
just to give you an idea). I'd also need to implement class Row accordingly to
handle the values.
I could do something similar for MapType, and I believe a list/vector should
suffice for ArrayType.
Thoughts?
# You can actually just copy and paste the code below on R to run it
setClass("StructField",
representation(
name = "character",
type = "character"
))
# A Struct is a set of StructField objects, modeled as an environment
setClass("Struct",
representation(
struct = "environment"
))
# Initialize a Struct from a list of StructField objects
setMethod("initialize", signature = "Struct", definition=
function(.Object, fields) {
lapply(fields, function(field) {
.Object@struct[[field@name]] <- field
})
return(.Object)
})
# Overwrite [[ operator to access the environment directly
setGeneric("[[")
setMethod("[[", signature="Struct", definition=
function(x, i) {
return(x@struct[[i]])
})
# Overwrite [[<- operator to access the environment directly
setGeneric("[[<-")
setMethod("[[<-", signature="Struct", definition=
function(x, i, value) {
if (class(value) == "StructField") {
x@struct[[i]] <- value
}
return(x)
})
field1 <- new("StructField", name="x", type="numeric")
field2 <- new("StructField", name="y", type="character")
s <- new("Struct", fields=list(field1, field2))
s[["x"]]
s[["z"]] <- new("StructField", name="z", type="logical")
> Method coltypes() to return the R column types of a DataFrame
> -------------------------------------------------------------
>
> Key: SPARK-10863
> URL: https://issues.apache.org/jira/browse/SPARK-10863
> Project: Spark
> Issue Type: Sub-task
> Components: SparkR
> Affects Versions: 1.5.0
> Reporter: Oscar D. Lara Yejas
> Assignee: Oscar D. Lara Yejas
> Fix For: 1.6.0
>
>
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
