[
https://issues.apache.org/jira/browse/GOBBLIN-1462?focusedWorklogId=608859&page=com.atlassian.jira.plugin.system.issuetabpanels:worklog-tabpanel#worklog-608859
]
ASF GitHub Bot logged work on GOBBLIN-1462:
-------------------------------------------
Author: ASF GitHub Bot
Created on: 09/Jun/21 01:11
Start Date: 09/Jun/21 01:11
Worklog Time Spent: 10m
Work Description: sv2000 commented on a change in pull request #3302:
URL: https://github.com/apache/gobblin/pull/3302#discussion_r647895630
##########
File path:
gobblin-runtime/src/main/java/org/apache/gobblin/runtime/api/FsSpecConsumer.java
##########
@@ -112,8 +113,11 @@ public FsSpecConsumer(@Nullable FileSystem fs, Config
config) {
JobSpec.Builder jobSpecBuilder = new
JobSpec.Builder(avroJobSpec.getUri());
Properties props = new Properties();
props.putAll(avroJobSpec.getProperties());
-
jobSpecBuilder.withJobCatalogURI(avroJobSpec.getUri()).withVersion(avroJobSpec.getVersion())
-
.withDescription(avroJobSpec.getDescription()).withConfigAsProperties(props);
+ jobSpecBuilder.withJobCatalogURI(avroJobSpec.getUri())
+ .withVersion(avroJobSpec.getVersion())
+ .withDescription(avroJobSpec.getDescription())
+ .withConfigAsProperties(props)
Review comment:
Yes, we will still need this line. The only change is we are now
explicit about how the config member variable of JobSpec is set. Previously, it
defaults to using ConfigUtils#propertiesToTypedConfig() which throws an
exception when it encounters prefixed keys.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
Issue Time Tracking
-------------------
Worklog Id: (was: 608859)
Time Spent: 0.5h (was: 20m)
> Ensure FsSpecConsumer handles config keys which are prefixes of other keys
> --------------------------------------------------------------------------
>
> Key: GOBBLIN-1462
> URL: https://issues.apache.org/jira/browse/GOBBLIN-1462
> Project: Apache Gobblin
> Issue Type: Bug
> Components: gobblin-cluster
> Affects Versions: 0.15.0
> Reporter: Sudarshan Vasudevan
> Assignee: Sudarshan Vasudevan
> Priority: Major
> Fix For: 0.16.0
>
> Time Spent: 0.5h
> Remaining Estimate: 0h
>
> Currently, the FsSpecConsumer fails when building a JobSpec if the
> configuration contains keys which are prefixes of other keys, which is not
> allowed in typesafe config. In practice, this constraint may be violated
> particularly when loading system properties which are outside application's
> control. To ensure these properties are safely handled, the FsSpecConsumer
> should use the proertiesToConfig() method in PropertiesUtils which adds a
> suffix to the root properties.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
