[ https://issues.apache.org/jira/browse/SPARK-3928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14533277#comment-14533277 ]
Nicholas Chammas commented on SPARK-3928: ----------------------------------------- {quote} Comma separated lists: were supported, will not be supported anymore. Use the varargs method to pass more than one file. This is because {{,}} is a valid character in a filename and so the old implementation was broken for some people. {quote} Isn't this inconsistent with how {{textFile()}} works? {{textFile()}} allows you to pass a single, comma-delimited string of file paths. If we want to support files with commas in their name -- which sounds like a corner case -- shouldn't we instead offer some kind of escaping mechanism for commas? It would be more work for those who have commas in their files names, but that seems like a fair tradeoff. If you do weird things, then you should expect to do more work. The advantage for the rest of us is that we get a consistent way of globbing files across {{textFile()}} and {{parquetFile()}}. > Support wildcard matches on Parquet files > ----------------------------------------- > > Key: SPARK-3928 > URL: https://issues.apache.org/jira/browse/SPARK-3928 > Project: Spark > Issue Type: Improvement > Components: Spark Core, SQL > Reporter: Nicholas Chammas > Assignee: Cheng Lian > Priority: Minor > Fix For: 1.3.0 > > > {{SparkContext.textFile()}} supports patterns like {{part-*}} and > {{2014-\?\?-\?\?}}. > It would be nice if {{SparkContext.parquetFile()}} did the same. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org