fx19880617 commented on a change in pull request #4874: Pinot ingestion job -
Standalone
URL: https://github.com/apache/incubator-pinot/pull/4874#discussion_r353544048
##########
File path:
pinot-batch-ingestion/pinot-standalone/src/main/java/org/apache/pinot/ingestion/standalone/SegmentGenerationJobRunner.java
##########
@@ -227,41 +236,83 @@ public void run()
}
}
- private Schema getSchema()
- throws Exception {
- URI schemaURI = new URI(_spec.getTableSpec().getSchemaURI());
+ private Schema getSchema() {
+ URI schemaURI;
+ try {
+ schemaURI = new URI(_spec.getTableSpec().getSchemaURI());
+ } catch (URISyntaxException e) {
+ throw new RuntimeException("Schema URI is not valid - " +
_spec.getTableSpec().getSchemaURI(), e);
+ }
String scheme = schemaURI.getScheme();
String schemaJson;
if (PinotFSFactory.isSchemeSupported(scheme)) {
// Try to use PinotFS to read schema URI
PinotFS pinotFS = PinotFSFactory.create(scheme);
- InputStream schemaStream = pinotFS.open(schemaURI);
- schemaJson = IOUtils.toString(schemaStream, StandardCharsets.UTF_8);
+ InputStream schemaStream;
+ try {
+ schemaStream = pinotFS.open(schemaURI);
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to open schema file stream on Pinot
fs - " + schemaURI, e);
+ }
+ try {
+ schemaJson = IOUtils.toString(schemaStream, StandardCharsets.UTF_8);
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to read from schema file data
stream on Pinot fs - " + schemaURI, e);
+ }
} else {
// Try to directly read from URI.
- schemaJson = IOUtils.toString(schemaURI, StandardCharsets.UTF_8);
+ try {
+ schemaJson = IOUtils.toString(schemaURI, StandardCharsets.UTF_8);
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to read from Schema URI - " +
schemaURI, e);
+ }
+ }
+ try {
+ return Schema.fromString(schemaJson);
+ } catch (IOException e) {
+ throw new RuntimeException("Failed to decode Pinot schema from json
string - " + schemaJson, e);
Review comment:
I feel it's ok since this the last failure log for schema decoder, and users
should be able to copy this log and try to validate that.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
