dev  

Messages by Thread

• [I] [Bug report] URLFilterBolt seems to emit unanchored [stormcrawler] via GitHub
  • Re: [I] [Bug report] URLFilterBolt seems to emit unanchored tuples [stormcrawler] via GitHub
  • Re: [I] [Bug report] URLFilterBolt seems to emit unanchored tuples [stormcrawler] via GitHub
  • Re: [I] [Bug report] URLFilterBolt seems to emit unanchored tuples [stormcrawler] via GitHub
• Release 3.7.0? Julien Nioche
  • Re: Release 3.7.0? Richard Zowalla
• [GH] (stormcrawler/2013): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/2013): Workflow run "Java CI with Maven" failed! GitBox
• [PR] Mark HTTPClient as deprecated [stormcrawler] via GitHub
  • Re: [PR] Mark HTTPClient as deprecated [stormcrawler] via GitHub
• [PR] Mark Selenium module as deprecated [stormcrawler] via GitHub
  • Re: [PR] Mark Selenium module as deprecated [stormcrawler] via GitHub
• [PR] Mark CloudSearch as deprecated [stormcrawler] via GitHub
  • Re: [PR] Mark CloudSearch as deprecated [stormcrawler] via GitHub
• [PR] Upgrade to Apache Storm 3.0.0 and Java 25 [stormcrawler] via GitHub
• [DISCUSS] Storm 3 meets StormCrawler Richard Zowalla
  • Re: [DISCUSS] Storm 3 meets StormCrawler Julien Nioche
  • Re: [DISCUSS] Storm 3 meets StormCrawler Richard Zowalla
• [I] Deprecate AWS CloudSearch [stormcrawler] via GitHub
  • Re: [I] Deprecate AWS CloudSearch [stormcrawler] via GitHub
• [I] Upgrade to Apache Storm 3.0.0 [stormcrawler] via GitHub
• [I] Deprecate Selenium module [stormcrawler] via GitHub
  • Re: [I] Deprecate Selenium module [stormcrawler] via GitHub
• [I] Deprecate HTTPClient protocol implementation [stormcrawler] via GitHub
  • Re: [I] Deprecate HTTPClient protocol implementation [stormcrawler] via GitHub
• [PR] Allow indexer bolts to use a metadata value as the document ID [stormcrawler] via GitHub
  • Re: [PR] Allow indexer bolts to use a metadata value as the document ID [stormcrawler] via GitHub
  • Re: [PR] Allow indexer bolts to use a metadata value as the document ID [stormcrawler] via GitHub
  • Re: [PR] Allow indexer bolts to use a metadata value as the document ID [stormcrawler] via GitHub
• [PR] Resolve relative outlinks against <base href> in JSoupParserBolt [stormcrawler] via GitHub
  • Re: [PR] Resolve relative outlinks against <base href> in JSoupParserBolt [stormcrawler] via GitHub
• Re: [I] Remove needsDOM() in ParseFilter [stormcrawler] via GitHub
  • Re: [I] Remove needsDOM() in ParseFilter [stormcrawler] via GitHub
• Re: [I] SubDocumentsFilterTest should not use a sitemap file [stormcrawler] via GitHub
• [PR] build(deps): bump actions/checkout from 7.0.0 to 7.0.1 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump actions/checkout from 7.0.0 to 7.0.1 [stormcrawler] via GitHub
• [PR] build(deps): bump software.amazon.awssdk:bom from 2.47.5 to 2.48.3 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump software.amazon.awssdk:bom from 2.47.5 to 2.48.3 [stormcrawler] via GitHub
• [PR] build(deps): bump langchain4j.version from 1.17.2 to 1.18.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump langchain4j.version from 1.17.2 to 1.18.0 [stormcrawler] via GitHub
• [PR] #108 Normalize misspelled HTTP headers [stormcrawler] via GitHub
  • Re: [PR] #108 Normalize misspelled HTTP headers [stormcrawler] via GitHub
  • Re: [PR] #108 Normalize misspelled HTTP headers [stormcrawler] via GitHub
• [PR] Fix #304: Use HTML test document instead of sitemap XML in SubDocumentsFilterTest [stormcrawler] via GitHub
  • Re: [PR] Fix #304: Use HTML test document instead of sitemap XML in SubDocumentsFilterTest [stormcrawler] via GitHub
  • Re: [PR] Fix #304: Use HTML test document instead of sitemap XML in SubDocumentsFilterTest [stormcrawler] via GitHub
  • Re: [PR] Fix #304: Use HTML test document instead of sitemap XML in SubDocumentsFilterTest [stormcrawler] via GitHub
  • Re: [PR] Fix #304: Use HTML test document instead of sitemap XML in SubDocumentsFilterTest [stormcrawler] via GitHub
• [PR] bugfix - max content not applied for robots.txt when redirecting [stormcrawler] via GitHub
  • Re: [PR] bugfix - max content not applied for robots.txt when redirecting [stormcrawler] via GitHub
• [PR] build(deps): bump actions/setup-java from 5.5.0 to 5.6.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump actions/setup-java from 5.5.0 to 5.6.0 [stormcrawler] via GitHub
• Feedback on Website Refresh Richard Zowalla
  • Re: Feedback on Website Refresh Richard Zowalla
  • Re: Feedback on Website Refresh Richard Zowalla
  • Re: Feedback on Website Refresh Dávid Szigecsán
  • Re: Feedback on Website Refresh Richard Zowalla
• [PR] #2001 Document adaptive scheduler metadata [stormcrawler] via GitHub
  • Re: [PR] #2001 Document adaptive scheduler metadata [stormcrawler] via GitHub
  • Re: [PR] #2001 Document adaptive scheduler metadata [stormcrawler] via GitHub
  • Re: [PR] #2001 Document adaptive scheduler metadata [stormcrawler] via GitHub
  • Re: [PR] #2001 Document adaptive scheduler metadata [stormcrawler] via GitHub
  • Re: [PR] #2001 Document adaptive scheduler metadata [stormcrawler] via GitHub
• [I] [Improvement] Document metadata to be persisted for adaptive scheduling [stormcrawler] via GitHub
  • Re: [I] [Improvement] Document metadata to be persisted for adaptive scheduling [stormcrawler] via GitHub
• [PR] Enforce robots.txt Crawl-delay in URLFrontier queues [stormcrawler] via GitHub
  • Re: [PR] Enforce robots.txt Crawl-delay in URLFrontier queues [stormcrawler] via GitHub
  • Re: [PR] Enforce robots.txt Crawl-delay in URLFrontier queues [stormcrawler] via GitHub
  • Re: [PR] Enforce robots.txt Crawl-delay in URLFrontier queues [stormcrawler] via GitHub
  • Re: [PR] Enforce robots.txt Crawl-delay in URLFrontier queues [stormcrawler] via GitHub
• [I] [Improvement] WARC writer: WARC-Protocol header to follow WARC field proposals [stormcrawler] via GitHub
• [PR] [WARC] Format WARC-IP-Address header values following the WARC community recommendations [stormcrawler] via GitHub
  • Re: [PR] [WARC] Format WARC-IP-Address header values following the WARC community recommendations [stormcrawler] via GitHub
  • Re: [PR] [WARC] Format WARC-IP-Address header values following the WARC community recommendations [stormcrawler] via GitHub
  • Re: [PR] [WARC] Format WARC-IP-Address header values following the WARC community recommendations [stormcrawler] via GitHub
  • Re: [PR] [WARC] Format WARC-IP-Address header values following the WARC community recommendations [stormcrawler] via GitHub
• [DISCUSS] Deprecating the Selenium module and the HTTPClient protocol implementation Richard Zowalla
• [GH] (stormcrawler/verbatimHeadersPW): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/verbatimHeadersPW): Workflow run "Java CI with Maven" failed! GitBox
• [PR] Verbatim headers for Playwright [stormcrawler] via GitHub
  • Re: [PR] Verbatim headers for Playwright [stormcrawler] via GitHub
• [D] Deprecate HTTPClient protocol implementation [stormcrawler] via GitHub
  • Re: [D] Deprecate HTTPClient protocol implementation [stormcrawler] via GitHub
  • Re: [D] Deprecate HTTPClient protocol implementation [stormcrawler] via GitHub
• [D] Deprecate Selenium module? [stormcrawler] via GitHub
  • Re: [D] Deprecate Selenium module? [stormcrawler] via GitHub
  • Re: [D] Deprecate Selenium module? [stormcrawler] via GitHub
• Re: [I] Fetcher: optionally slow down fetching from hosts with repeated exceptions [stormcrawler] via GitHub
• [PR] Workaround to activate LogViewer in docker compose [stormcrawler] via GitHub
  • Re: [PR] Workaround to activate LogViewer in docker compose [stormcrawler] via GitHub
• [PR] build(deps): bump software.amazon.awssdk:bom from 2.46.21 to 2.47.5 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump software.amazon.awssdk:bom from 2.46.21 to 2.47.5 [stormcrawler] via GitHub
• [PR] build(deps): bump selenium.version from 4.45.0 to 4.46.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump selenium.version from 4.45.0 to 4.46.0 [stormcrawler] via GitHub
• [PR] build(deps): bump junit.version from 6.1.1 to 6.1.2 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump junit.version from 6.1.1 to 6.1.2 [stormcrawler] via GitHub
• [PR] build(deps): bump langchain4j.version from 1.17.1 to 1.17.2 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump langchain4j.version from 1.17.1 to 1.17.2 [stormcrawler] via GitHub
• [GH] (stormcrawler/pluggableRobotRuleParser): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/pluggableRobotRuleParser): Workflow run "Java CI with Maven" failed! GitBox
• [PR] Pluggable robot rule parser [stormcrawler] via GitHub
  • Re: [PR] Pluggable robot rule parser [stormcrawler] via GitHub
  • Re: [PR] Pluggable robot rule parser [stormcrawler] via GitHub
  • Re: [PR] Pluggable robot rule parser [stormcrawler] via GitHub
  • Re: [PR] Pluggable robot rule parser [stormcrawler] via GitHub
  • Re: [PR] Pluggable robot rule parser [stormcrawler] via GitHub
  • Re: [PR] Pluggable robot rule parser [stormcrawler] via GitHub
• [PR] Don't trigger mvn workflow for changes to the docs [stormcrawler] via GitHub
  • Re: [PR] Don't trigger mvn workflow for changes to the docs [stormcrawler] via GitHub
• [PR] Adding Green Web Foundation to Powered By [stormcrawler] via GitHub
  • Re: [PR] Adding Green Web Foundation to Powered By [stormcrawler] via GitHub
• [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Adaptive back-off for rate-limited hosts via the queue stream [stormcrawler] via GitHub
• [GH] (stormcrawler/feat/1106-adaptive-host-backoff): Workflow run "Java CI with Maven" failed! GitBox
• [PR] build(deps): bump actions/setup-java from 5.4.0 to 5.5.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump actions/setup-java from 5.4.0 to 5.5.0 [stormcrawler] via GitHub
• [PR] Feature/store request url [stormcrawler] via GitHub
• [PR] Port QueueBolt (queue stream) to the opensearch-java module [stormcrawler] via GitHub
  • Re: [PR] Port QueueBolt (queue stream) to the opensearch-java module [stormcrawler] via GitHub
  • Re: [PR] Port QueueBolt (queue stream) to the opensearch-java module [stormcrawler] via GitHub
• [I] Propagate robots.txt crawl-delay to URLFrontier via setDelay [stormcrawler] via GitHub
• [GH] (stormcrawler/dependabot/maven/langchain4j.version-1.17.1): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/dependabot/maven/com.microsoft.playwright-playwright-1.61.0): Workflow run "Java CI with Maven" failed! GitBox
• [PR] build(deps): bump opensearch.version from 2.19.5 to 2.19.6 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump opensearch.version from 2.19.5 to 2.19.6 [stormcrawler] via GitHub
• [PR] build(deps): bump software.amazon.awssdk:bom from 2.46.17 to 2.46.21 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump software.amazon.awssdk:bom from 2.46.17 to 2.46.21 [stormcrawler] via GitHub
• [PR] build(deps): bump com.microsoft.playwright:playwright from 1.60.0 to 1.61.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump com.microsoft.playwright:playwright from 1.60.0 to 1.61.0 [stormcrawler] via GitHub
• [PR] build(deps): bump langchain4j.version from 1.17.0 to 1.17.1 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump langchain4j.version from 1.17.0 to 1.17.1 [stormcrawler] via GitHub
• Re: [PR] Queue stream [stormcrawler] via GitHub
  • Re: [PR] Queue stream [stormcrawler] via GitHub
• [GH] (stormcrawler/990): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/990): Workflow run "Java CI with Maven" failed! GitBox
• Re: [I] Send host info on a specific stream [stormcrawler] via GitHub
  • Re: [I] Send host info on a specific stream [stormcrawler] via GitHub
• [PR] Emit Retry-After host back-off and block the URLFrontier queue [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
  • Re: [PR] Block the URLFrontier queue on Retry-After via the queue stream [stormcrawler] via GitHub
• [PR] build(deps): bump actions/cache from 6.0.0 to 6.1.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump actions/cache from 6.0.0 to 6.1.0 [stormcrawler] via GitHub
• [PR] build(deps): bump org.apache:apache from 38 to 39 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump org.apache:apache from 38 to 39 [stormcrawler] via GitHub
• [PR] build(deps): bump langchain4j.version from 1.16.3 to 1.17.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump langchain4j.version from 1.16.3 to 1.17.0 [stormcrawler] via GitHub
• [PR] build(deps): bump software.amazon.awssdk:bom from 2.46.15 to 2.46.17 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump software.amazon.awssdk:bom from 2.46.15 to 2.46.17 [stormcrawler] via GitHub
• [PR] build(deps): bump junit.version from 6.1.0 to 6.1.1 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump junit.version from 6.1.0 to 6.1.1 [stormcrawler] via GitHub
• [PR] build(deps): bump actions/setup-java from 5.3.0 to 5.4.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump actions/setup-java from 5.3.0 to 5.4.0 [stormcrawler] via GitHub
• [PR] build(deps): bump actions/cache from 5.0.5 to 6.0.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump actions/cache from 5.0.5 to 6.0.0 [stormcrawler] via GitHub
• [PR] use placeholder instead of older version of archetypes in README [stormcrawler] via GitHub
  • Re: [PR] use placeholder instead of older version of archetypes in README [stormcrawler] via GitHub
• Re: [I] JSoup parser to handle text/plain [stormcrawler] via GitHub
• [PR] build(deps): bump software.amazon.awssdk:bom from 2.46.10 to 2.46.15 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump software.amazon.awssdk:bom from 2.46.10 to 2.46.15 [stormcrawler] via GitHub
• [PR] build(deps): bump selenium.version from 4.44.0 to 4.45.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump selenium.version from 4.44.0 to 4.45.0 [stormcrawler] via GitHub
• [PR] build(deps): bump langchain4j.version from 1.16.2 to 1.16.3 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump langchain4j.version from 1.16.2 to 1.16.3 [stormcrawler] via GitHub
• [PR] build(deps): bump actions/checkout from 6.0.3 to 7.0.0 [stormcrawler] via GitHub
  • Re: [PR] build(deps): bump actions/checkout from 6.0.3 to 7.0.0 [stormcrawler] via GitHub
• [PR] Bump actions/setup-java from 5.2.0 to 5.3.0 [stormcrawler] via GitHub
  • Re: [PR] Bump actions/setup-java from 5.2.0 to 5.3.0 [stormcrawler] via GitHub
• [GH] (stormcrawler/fix/1955-filespout-distributed-mode): Workflow run "Java CI with Maven" is working again! GitBox
• [PR] #1955 - Make FileSpout work in distributed mode [stormcrawler] via GitHub
  • Re: [PR] #1955 - Make FileSpout work in distributed mode [stormcrawler] via GitHub
  • Re: [PR] #1955 - Make FileSpout work in distributed mode [stormcrawler] via GitHub
  • Re: [PR] #1955 - Make FileSpout work in distributed mode [stormcrawler] via GitHub
• Re: [I] support retry-after in FetcherBolt [stormcrawler] via GitHub
  • Re: [I] support retry-after in FetcherBolt [stormcrawler] via GitHub
• Re: [I] Protocol-okhttp: implement IP filter [stormcrawler] via GitHub
• [I] [Bug report] FileSpout does not work in distributed mode [stormcrawler] via GitHub
  • Re: [I] [Bug report] FileSpout does not work in distributed mode [stormcrawler] via GitHub
• [GH] (stormcrawler/feature/466): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/FileSpoutCompressedFiles): Workflow run "Java CI with Maven" is working again! GitBox
  • [GH] (stormcrawler/FileSpoutCompressedFiles): Workflow run "Java CI with Maven" is working again! GitBox
• [GH] (stormcrawler/fix/multiproxy-lookup-map): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/fix/multiproxy-lookup-map): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/fix/multiproxy-lookup-map): Workflow run "Java CI with Maven" failed! GitBox
  • [GH] (stormcrawler/fix/multiproxy-lookup-map): Workflow run "Java CI with Maven" failed! GitBox
• [PR] #1951 refactor: replace linear proxy scan with lookup map in MultiProxyManager [stormcrawler] via GitHub

• Earlier messages