[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user maropu commented on the issue: https://github.com/apache/spark/pull/13300 Aha, I see. Anyway, we need to keep discussion not here but the JIRA! (because this is the closed..) --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13300 Oh, I remember the answer from my previous similar question, which was that we should not add some APIs just for consistency. I have some references about the requests for this feature

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user maropu commented on the issue: https://github.com/apache/spark/pull/13300 Yea, I also think `json` and `csv` stuffs should be consistent and they'd be better to have the same code structure and behaviour as @HyukjinKwon said. Since we do not have

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/13300 @HyukjinKwon Thanks! After your #16680 is merged, submit a PR with the code you show above. then. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/13300 Actually, this feature might not be urgent as said above but IMO I like this feature to be honest. I guess the reason it was hold is that IMHO it does not look a clean fix. I recently

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user maropu commented on the issue: https://github.com/apache/spark/pull/13300 This pr seems stale and inactive. I know this kind of API changes has lower priorities now. So, how about closing this pr for now and setting `LATER` in the corresponding JIRA? Thought? cc: @rxin

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/13300 @maropu Thanks for the comments! It seems like adding such new datasource API in DataFrameReader is not in the priority now. That is why it has been in relatively idle state now. What you are

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user maropu commented on the issue: https://github.com/apache/spark/pull/13300 I checked the feasibility to implement `from_json` in `sql.functions`; If we move csv parser code (`CSVReader`, `CSVOptions`, ...) from `o.a.s.sql.execution.datasources.csv` to

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user maropu commented on the issue: https://github.com/apache/spark/pull/13300 What's the status of this pr? It seems to be more natural that we implement `from_csv` in a similar way of `from_json` in

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user pjfanning commented on the issue: https://github.com/apache/spark/pull/13300 Would it be feasible to get this merged for Spark 2.1.0? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/13300 @rxin Do you think we can revisit this feature and have it in 2.1? Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user rxin commented on the issue: https://github.com/apache/spark/pull/13300 @pjfanning we are now focusing on bug fixes and stability fixes rather than adding new features. --- If your project is set up for it, you can reply to this email and have your reply appear on

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user pjfanning commented on the issue: https://github.com/apache/spark/pull/13300 @HyukjinKwon @rxin @falaki Would it be feasible to get this merged for Spark 2.0 release? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user xwu0226 commented on the issue: https://github.com/apache/spark/pull/13300 @yhuai @rxin Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes

[GitHub] spark issue #13300: [SPARK-15463][SQL] support creating dataframe out of Dat...

Github user pjfanning commented on the issue: https://github.com/apache/spark/pull/13300 @xwu0226 the unit tests you added seem sufficient --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have