paul-rogers commented on a change in pull request #1798: DRILL-7279: Enable
provided schema for text files without headers
URL: https://github.com/apache/drill/pull/1798#discussion_r291007270
##########
File path:
exec/vector/src/main/java/org/apache/drill/exec/record/metadata/TupleMetadata.java
##########
@@ -46,6 +46,10 @@
public interface TupleMetadata extends Propertied, Iterable<ColumnMetadata> {
public static final String IS_STRICT_SCHEMA_PROP = DRILL_PROP_PREFIX +
"strict";
+ public static final String HAS_HEADERS_PROP = DRILL_PROP_PREFIX + "headers";
+ public static final String SKIP_FIRST_LINE_PROP = DRILL_PROP_PREFIX +
"skipFirstLine";
+ public static final String DELIMITER_PROP = DRILL_PROP_PREFIX + "delimiter";
+ public static final String COMMENT_CHAR_PROP = DRILL_PROP_PREFIX +
"commentChar";
Review comment:
Added unit tests. This revealed that quote delimiters **never worked** in
the V2 text reader. That is, the following never worked:
```
quote="@"
escape="~"
```
Data:
```
@foo~@bar@
```
Reworked the parsing code for quoted strings to fix this issue, the output
is now `foo@bar`. Also fixed:
```
@foo~~bar@
```
An escape without a quote should be a normal char. The output is now
`foo~~bar`.
Verified that V3 works for the quote and delimiter as the same character
(did not check V2):
```
quote="@"
escape="@"
```
Data:
```
@foo@@bar@
```
Produces output `foo@bar`.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
