Re: How to create DataFrame from a binary file?

2015-08-10 Thread Ted Yu
gt;>>>> >>>>> I can see that PortableDataStream has method called toArray which can >>>>> convert into byte array I was thinking if I have JavaRDD can I >>>>> call >>>>> the following and get DataFrame >>>>> &g

Re: How to create DataFrame from a binary file?

2015-08-10 Thread Umesh Kacha
as thinking if I have JavaRDD can I >>>> call >>>> the following and get DataFrame >>>> >>>> DataFrame binDataFrame = >>>> sqlContext.createDataFrame(javaBinRdd,Byte.class); >>>> >>>> Please guide I am new to Spark.

Re: How to create DataFrame from a binary file?

2015-08-09 Thread bo yang
as thinking if I can convert my custom format into >>> DataFrame >>> using binary operations then I dont need to create my own custom Hadoop >>> format am I on right track? Will reading binary data into DataFrame >>> scale? >>> >>> >>> >

Re: How to create DataFrame from a binary file?

2015-08-09 Thread Umesh Kacha
n custom format which is >> binary >> format and I was thinking if I can convert my custom format into DataFrame >> using binary operations then I dont need to create my own custom Hadoop >> format am I on right track? Will reading binary data into DataFrame scale? >>

Re: How to create DataFrame from a binary file?

2015-08-09 Thread bo yang
to DataFrame > using binary operations then I dont need to create my own custom Hadoop > format am I on right track? Will reading binary data into DataFrame scale? > > > > -- > View this message in context: > http://apache-spark-user-list.1001560.n3.nabble.com/How-to-create-D

How to create DataFrame from a binary file?

2015-08-08 Thread unk1102
ormat into DataFrame using binary operations then I dont need to create my own custom Hadoop format am I on right track? Will reading binary data into DataFrame scale? -- View this message in context: http://apache-spark-user-list.1001560.n3.nabble.com/How-to-create-DataFrame-from-a-binary-file-tp24179