No, it is just being truncated for display as the ... implies. Pass truncate=false to the show command.
On Sun, Aug 28, 2016, 15:24 Kevin Tran <kevin...@gmail.com> wrote: > Hi, > I wrote to parquet file as following: > > +--------------------+ > | word| > +--------------------+ > |THIS IS MY CHARACTERS ...| > |// ANOTHER LINE OF CHAC...| > +--------------------+ > > These lines are not full text and it is being trimmed down. > Does anyone know how many chacters StringType could handle ? > > In the Spark code: > org.apache.spark.sql.types.StringType > /** > * The default size of a value of the StringType is 4096 bytes. > */ > override def defaultSize: Int = 4096 > > > Thanks, > Kevin. >
