[
https://issues.apache.org/jira/browse/NUTCH-3249?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18142981#comment-18142981
]
ASF GitHub Bot commented on NUTCH-3249:
---------------------------------------
lewismc commented on code in PR #994:
URL: https://github.com/apache/nutch/pull/994#discussion_r4538360900
##########
src/java/org/apache/nutch/tools/CommonCrawlDataDumper.java:
##########
@@ -334,7 +334,8 @@ public void dump(File outputDir, File segmentRootDir, File
linkdb, boolean gzip,
reverseKey, url, timestamp, extensionName, !gzip);
outputRelativePath = outputFullPath
.substring(0, outputFullPath.lastIndexOf(File.separator) -
1);
- filename = content.getMetadata().get(Metadata.DATE) + "."
Review Comment:
Excellent catch. We can look at augmenting test coverage in NUTCH-3251.
Thank you.
> CommonCrawlDataDumper should use safe names for archive entries built from
> crawled content
> ------------------------------------------------------------------------------------------
>
> Key: NUTCH-3249
> URL: https://issues.apache.org/jira/browse/NUTCH-3249
> Project: Nutch
> Issue Type: Bug
> Components: commoncrawl
> Affects Versions: 1.23
> Reporter: Lewis John McGibbney
> Priority: Major
> Fix For: 1.24
>
>
> CommonCrawlDataDumper builds archive entry names from values taken out of
> crawled content. Those values are not limited to a safe file name, so an
> entry name can point outside the output directory.
> Whether unpacking writes outside that directory depends on the program that
> unpacks the archive. The defect is that Nutch writes the unsafe name.
> Limit any crawled value used in an entry name to a single safe path segment.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)