mcvsubbu commented on a change in pull request #4942: Refactor SegmentFetcher
to handle retry for all URIs
URL: https://github.com/apache/incubator-pinot/pull/4942#discussion_r360596149
##########
File path:
pinot-common/src/main/java/org/apache/pinot/common/utils/fetcher/SegmentFetcher.java
##########
@@ -16,34 +16,23 @@
* specific language governing permissions and limitations
* under the License.
*/
-package org.apache.pinot.common.segment.fetcher;
+package org.apache.pinot.common.utils.fetcher;
import java.io.File;
-import java.util.Set;
+import java.net.URI;
import org.apache.commons.configuration.Configuration;
public interface SegmentFetcher {
/**
- * Initializes configurations
- * @param configs
+ * Initializes the segment fetcher.
*/
- void init(Configuration configs);
+ void init(Configuration config);
/**
- * Fetches segment from a uri location to the local filesystem. Can come
from a remote fs or a local fs.
- * @param uri current segment location
- * @param tempFile location segment will be stored locally
- * @throws Exception
+ * Fetches a segment from URI location to local.
*/
- void fetchSegmentToLocal(String uri, File tempFile)
+ void fetchSegmentToLocal(URI uri, File dest)
throws Exception;
-
- /**
- * Returns a list of config keys whose value should not be logged.
- *
- * @return List of protected config keys
- */
- Set<String> getProtectedConfigKeys();
Review comment:
I believe we have exposed and documented this interface. This is an
incompatible change. Can't say off-hand what the effects are. Are we sure that
segment fetchers will not have their own protected config keys? E.g. a
HdfsSegmentFetcher could have auth mechanisms and secrets that we don't want
logged.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
