Re: FYI: The evolution on `CHAR` type behavior

2020-03-15 Thread Reynold Xin
Are we sure "not padding" is "incorrect"? I don't know whether ANSI SQL actually requires padding, but plenty of databases don't actually pad. https://docs.snowflake.net/manuals/sql-reference/data-types-text.html (

[DISCUSS] Resolve ambiguous parser rule between two "create table"s

2020-03-15 Thread Jungtaek Lim
Hi devs, I'd like to initiate discussion and hear the voices for resolving ambiguous parser rule between two "create table"s being brought by SPARK-30098 [1]. Previously, "create table" parser rules were clearly distinguished via "USING provider", which was very intuitive and deterministic. Say,

Re: FYI: The evolution on `CHAR` type behavior

2020-03-15 Thread Dongjoon Hyun
Hi, Reynold. Please see the following for the context. https://issues.apache.org/jira/browse/SPARK-31136 "Revert SPARK-30098 Use default datasource as provider for CREATE TABLE syntax" I raised the above issue according to the new rubric, and the banning was the proposed alternative to reduce

Spark 2.4.x and 3.x datasourcev2 api documentation & references

2020-03-15 Thread MadDoxX
Hi, i have been following closely the new proposed api, and seems until now some lack of documentation... specially for people trying to implement them in JAVA. Anyway to give out some reference as starting point for java developpers as tests classes ? (anyway new api looks promising) !