Jackie-Jiang commented on a change in pull request #4942: Refactor
SegmentFetcher to handle retry for all URIs
URL: https://github.com/apache/incubator-pinot/pull/4942#discussion_r360600702
##########
File path:
pinot-common/src/main/java/org/apache/pinot/common/utils/fetcher/SegmentFetcher.java
##########
@@ -16,34 +16,23 @@
* specific language governing permissions and limitations
* under the License.
*/
-package org.apache.pinot.common.segment.fetcher;
+package org.apache.pinot.common.utils.fetcher;
import java.io.File;
-import java.util.Set;
+import java.net.URI;
import org.apache.commons.configuration.Configuration;
public interface SegmentFetcher {
/**
- * Initializes configurations
- * @param configs
+ * Initializes the segment fetcher.
*/
- void init(Configuration configs);
+ void init(Configuration config);
/**
- * Fetches segment from a uri location to the local filesystem. Can come
from a remote fs or a local fs.
- * @param uri current segment location
- * @param tempFile location segment will be stored locally
- * @throws Exception
+ * Fetches a segment from URI location to local.
*/
- void fetchSegmentToLocal(String uri, File tempFile)
+ void fetchSegmentToLocal(URI uri, File dest)
throws Exception;
-
- /**
- * Returns a list of config keys whose value should not be logged.
- *
- * @return List of protected config keys
- */
- Set<String> getProtectedConfigKeys();
Review comment:
I checked the documentation, and seems this interface is not exposed. This
interface should be internal only as a helper, which is backed by the PinotFS.
There should be no HdfsSegmentFetcher, but only HadoopPinotFS, and the current
PinotFSSegmentFetcher should work.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
