[
https://issues.apache.org/jira/browse/SPARK-20425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-20425.
-----------------------------
Resolution: Fixed
Fix Version/s: 2.3.0
> Support an extended display mode to print a column data per line
> ----------------------------------------------------------------
>
> Key: SPARK-20425
> URL: https://issues.apache.org/jira/browse/SPARK-20425
> Project: Spark
> Issue Type: Improvement
> Components: SQL
> Affects Versions: 2.1.0
> Reporter: Takeshi Yamamuro
> Assignee: Takeshi Yamamuro
> Priority: Minor
> Fix For: 2.3.0
>
>
> In the master, when printing Dataset with many columns, the readability is
> too low like;
> {code}
> scala> val df = spark.range(100).selectExpr((0 until 100).map(i => s"rand()
> AS c$i"): _*)
> scala> df.show(3, 0)
> +------------------+------------------+------------------+-------------------+------------------+------------------+-------------------+------------------+------------------+------------------+------------------+-------------------+------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------------------+-------------------+--------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+--------------------+--------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+-------------------+------------------+-------------------+------------------+------------------+-----------------+-------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+------------------+-------------------+-------------------+------------------+------------------+------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+
> |c0 |c1 |c2 |c3
> |c4 |c5 |c6 |c7
> |c8 |c9 |c10 |c11
> |c12 |c13 |c14 |c15
> |c16 |c17 |c18 |c19
> |c20 |c21 |c22 |c23
> |c24 |c25 |c26 |c27
> |c28 |c29 |c30 |c31
> |c32 |c33 |c34 |c35
> |c36 |c37 |c38 |c39
> |c40 |c41 |c42 |c43
> |c44 |c45 |c46 |c47
> |c48 |c49 |c50 |c51
> |c52 |c53 |c54 |c55
> |c56 |c57 |c58 |c59
> |c60 |c61 |c62 |c63
> |c64 |c65 |c66 |c67
> |c68 |c69 |c70 |c71
> |c72 |c73 |c74 |c75
> |c76 |c77 |c78 |c79
> |c80 |c81 |c82
> |c83 |c84 |c85 |c86
> |c87 |c88 |c89 |c90
> |c91 |c92 |c93 |c94
> |c95 |c96 |c97 |c98
> |c99 |
> +------------------+------------------+------------------+-------------------+------------------+------------------+-------------------+------------------+------------------+------------------+------------------+-------------------+------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------------------+-------------------+--------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+--------------------+--------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+-------------------+------------------+-------------------+------------------+------------------+-----------------+-------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+------------------+-------------------+-------------------+------------------+------------------+------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+
> |0.6306087152476858|0.9174349686288383|0.5511324165035159|0.3320844128641819
> |0.7738486877101489|0.2154915886962553|0.4754997600674299 |0.922780639280355
> |0.7136894772661909|0.2277580838165979|0.5926874459847249|0.40311408392226633|0.467830264333843
> |0.8330466896984213|0.1893258482389527|0.6320849515511165
> |0.7530911056912044 |0.06700254871955424|0.370528597355559
> |0.2755437445193154|0.23704391110980128|0.8067400174905822|0.13597793616251852|0.1708888820162453|0.01672725007605702|0.983118121881555
>
> |0.25040195628629924|0.060537253723083384|0.20000530582637488|0.3400572407133511|0.9375689433322597
>
> |0.057039316954370256|0.8053269714347623|0.5247817572228813|0.28419308820527944|0.9798908885194533
>
> |0.31805988175678146|0.7034448027077574|0.5400575751346084|0.25336322371116216|0.9361634546853429|0.6118681368289798|0.6295081549153907
>
> |0.13417468943957422|0.41617137072255794|0.7267230869252035|0.023792726137561115|0.5776157058356362
> |0.04884204913195467|0.26728716103441275|0.646680370807925
> |0.9782712690657244
> |0.16434031314818154|0.20985522381321275|0.24739842475440077
> |0.26335189682977334|0.19604841662422068|0.10742950487300651|0.20283136488091502|0.3100312319723688|0.886959006630645
>
> |0.25157102269776244|0.34428775168410786|0.3500506818575777|0.3781142441912052
>
> |0.8560316444386715|0.4737104888956839|0.735903101602148|0.02236617130529006|0.8769074095835873
> |0.2001426662503153|0.5534032319238532
> |0.7289496620397098|0.41955191309992157|0.9337700133660436
> |0.34059094378451005|0.6419144759403556|0.08167496930341167|0.9947099478497635|0.48010888605366586|0.22314796858167918|0.17786598882331306|0.7351521162297135
> |0.5422057170020095 |0.9521927872726792 |0.7459825486368227
> |0.40907708791990627|0.8903819313311575|0.7251413746923618
> |0.2977174938745204
> |0.9515209660203555|0.9375968604766713|0.5087851740042524|0.4255237544908751
> |0.8023768698664653|0.48003189618006703|0.1775841829745185|0.09050775629268382|0.6743909291138167
> |0.2498415755876865 |
> |0.6866473844170801|0.4774360641212433|0.631696201340726
> |0.33979113021468343|0.5663049010847052|0.7280190472258865|0.41370958502324806|0.9977433873622218|0.7671957338989901|0.2788708556233931|0.3355106391656496|0.88478952319287
>
> |0.0333974166999893|0.6061744715862606|0.9617779139652359|0.22484954822341863|0.12770906021550898|0.5577789629508672
> |0.2877649024640704|0.5566577406549361|0.9334933255278052
> |0.9166720585157266|0.9689249324600591 |0.6367502457478598|0.7993572745928459
> |0.23213222324218108|0.11928284054154137|0.6173493362456599
> |0.0505122058694798
> |0.9050228629552983|0.17112767911121707|0.47395598348370005
> |0.5820498657823081|0.6241124650645072|0.18587258258036776|0.14987593554122225|0.3079446253653946
> |0.9414228822867968|0.8362276265462365|0.9155655305576353
> |0.5121559807153562|0.8963362656525707|0.22765970274318037|0.8177039187132797
> |0.8190326635933787 |0.5256005177032199|0.8167598457269669
> |0.030936807130934496|0.6733006585281015 |0.4208049626816347
> |0.24603085738518538|0.22719198954208153|0.1622280557565281
> |0.22217325159218038|0.014684419513742553|0.08987111517447499|0.2157764759142622
> |0.8223414104088321 |0.4868624404491777
> |0.4016191733088167|0.6169281906889263|0.15603611040433385|0.18289285085714913|0.9538408988218972|0.15037154865295121|0.5364516961987454|0.8077254873163031|0.712600478545675|0.7277477241003857
>
> |0.19822912960348305|0.8305051199208777|0.18631911396566114|0.8909532487898342|0.3470409226992506
> |0.35306974180587636|0.9107058868891469
> |0.3321327206004986|0.48952332459050607|0.3630403307479373|0.5400046826340376
> |0.5387377194310529
> |0.42860539421837585|0.23214101630985995|0.21438968839794847|0.15370603160082352|0.04355605642700022|0.6096006707067466
>
> |0.6933354157094292|0.06302172470859002|0.03174631856164001|0.664243581650643
> |0.7833239547446621|0.696884598352864
> |0.34626385933237736|0.9263495598791336|0.404818892816584
> |0.2085585394755507|0.6150004897990109
> |0.05391193524302473|0.28188484028329097|
> +------------------+------------------+------------------+-------------------+------------------+------------------+-------------------+------------------+------------------+------------------+------------------+-------------------+------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+--------------------+-------------------+------------------+-------------------+--------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+------------------+------------------+-------------------+-------------------+-------------------+------------------+--------------------+--------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+--------------------+-------------------+-------------------+-------------------+-------------------+------------------+------------------+-------------------+-------------------+------------------+-------------------+------------------+------------------+-----------------+-------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+-------------------+------------------+-------------------+-------------------+------------------+------------------+------------------+-------------------+------------------+-------------------+------------------+-------------------+-------------------+-------------------+
> only showing top 2 rows
> {code}
> psql, CLI for PostgreSQL supports an extended display mode for this case like:
> http://stackoverflow.com/questions/9604723/alternate-output-format-for-psql
> {code}
> -RECORD 0-------------------
> c0 | 0.6306087152476858
> c1 | 0.9174349686288383
> c2 | 0.5511324165035159
> ...
> c98 | 0.05391193524302473
> c99 | 0.28188484028329097
> -RECORD 1-------------------
> c0 | 0.6866473844170801
> c1 | 0.4774360641212433
> c2 | 0.631696201340726
> ...
> c98 | 0.05391193524302473
> c99 | 0.28188484028329097
> only showing top 2 rows
> {code}
> I feel it is useful to support this mode in spark-shell.
> Thought?
--
This message was sent by Atlassian JIRA
(v6.3.15#6346)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
