Github user gengliangwang commented on a diff in the pull request: https://github.com/apache/spark/pull/20535#discussion_r180714763 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/DataSourceOptions.java --- @@ -17,16 +17,61 @@ package org.apache.spark.sql.sources.v2; +import java.io.IOException; import java.util.HashMap; import java.util.Locale; import java.util.Map; import java.util.Optional; +import java.util.stream.Stream; + +import com.fasterxml.jackson.databind.ObjectMapper; import org.apache.spark.annotation.InterfaceStability; /** * An immutable string-to-string map in which keys are case-insensitive. This is used to represent * data source options. + * + * Each data source implementation can define its own options and teach its users how to set them. + * Spark doesn't have any restrictions about what options a data source should or should not have. + * Instead Spark defines some standard options that data sources can optionally adopt. It's possible + * that some options are very common and many data sources use them. However different data + * sources may define the common options(key and meaning) differently, which is quite confusing to + * end users. + * + * The standard options defined by Spark: + * <table summary="standard data source options"> + * <tr> + * <th><b>Option key</b></th> + * <th><b>Option value</b></th> + * </tr> + * <tr> + * <td>path</td> + * <td>A path string of the data files/directories, like + * <code>path1</code>, <code>/absolute/file2</code>, <code>path3/*</code>. The path can + * either be relative or absolute, points to either file or directory, and can contain + * wildcards. This option is commonly used by file-based data sources.</td> + * </tr> + * <tr> + * <td>paths</td> + * <td>A JSON array style paths string of the data files/directories, like + * <code>["path1", "/absolute/file2"]</code>. The format of each path is same as the + * <code>path</code> option, plus it should follow JSON string literal format, e.g. quotes + * should be escaped, <code>pa\"th</code> means pa"th. --- End diff -- pa\"th?
--- --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org