clintropolis commented on a change in pull request #11360:
URL: https://github.com/apache/druid/pull/11360#discussion_r656090144
##########
File path: docs/ingestion/data-formats.md
##########
@@ -223,6 +223,41 @@ The Parquet `inputFormat` has the following components:
|flattenSpec| JSON Object |Define a [`flattenSpec`](#flattenspec) to extract
nested values from a Parquet file. Note that only 'path' expression are
supported ('jq' is unavailable).| no (default will auto-discover 'root' level
properties) |
| binaryAsString | Boolean | Specifies if the bytes parquet column which is
not logically marked as a string or enum type should be treated as a UTF-8
encoded string. | no (default = false) |
+### Thrift Stream
Review comment:
hmm, I'm not sure we have any other 'contrib' extensions described in
this section, it might be best if this lives in
https://github.com/apache/druid/blob/master/docs/development/extensions-contrib/thrift.md
for now. On the other hand, thrift i think is the only data format that isn't
a core extension (maybe in the future we should just consider adding
integration tests and making it a core extension?), so maybe it is ok to be
here. @techdocsmith do you have any thoughts?
Also, looking closer at the code, I guess this might also work with batch
ingestion too since the deserializer detects the format based on the bytes
given to it, though I haven't personally used this extension or tested this
scenario. I'll see if I can find some time to pull your branch and test it out
##########
File path: extensions-contrib/thrift-extensions/pom.xml
##########
@@ -141,6 +141,36 @@
<artifactId>hamcrest-core</artifactId>
<scope>test</scope>
</dependency>
+ <dependency>
+ <groupId>com.google.code.findbugs</groupId>
+ <artifactId>jsr305</artifactId>
+ <version>2.0.1</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>commons-io</groupId>
+ <artifactId>commons-io</artifactId>
+ <version>2.9.0</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>joda-time</groupId>
+ <artifactId>joda-time</artifactId>
+ <version>2.10.5</version>
+ <scope>provided</scope>
+ </dependency>
+ <dependency>
+ <groupId>com.fasterxml.jackson.core</groupId>
+ <artifactId>jackson-core</artifactId>
+ <version>2.10.2</version>
Review comment:
i think versions on a lot of these should be already defined in the top
level pom (the dependency checker in travis sometimes suggests more than is
necessary to fix the issue)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
