[
https://issues.apache.org/jira/browse/NUTCH-2856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17889292#comment-17889292
]
ASF GitHub Bot commented on NUTCH-2856:
---------------------------------------
lewismc commented on code in PR #826:
URL: https://github.com/apache/nutch/pull/826#discussion_r1799894039
##########
conf/log4j2.xml:
##########
@@ -25,7 +25,8 @@
<Appenders>
<RollingFile name="RollingFile"
fileName="${hadoop.log.dir}/${hadoop.log.file}"
filePattern="${hadoop.log.dir}/$${date:yyyy-MM}/nutch-%d{yyyy-MM-dd}.log.gz">
- <PatternLayout pattern="%d %p %c{1.} [%t] %m%n" />
+ <!--<PatternLayout pattern="%d %p %c{1.} [%t] %m%n" />-->
+ <PatternLayout pattern="%d %p %c [%t] %m%n" />
Review Comment:
OK
##########
src/plugin/protocol-smb/ivy.xml:
##########
@@ -0,0 +1,47 @@
+<?xml version="1.0" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<ivy-module xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd";
+ xmlns:ns0="http://ant.apache.org/ivy/maven"; version="2.0">
+ <info organisation="org.apache.nutch" module="${ant.project.name}">
+ <license name="Apache 2.0"/>
+ <ivyauthor name="Apache Nutch Team" url="http://nutch.apache.org"/>
+ <description>
+ Apache Nutch
+ </description>
+ </info>
+
+ <configurations>
+ <include file="../../..//ivy/ivy-configurations.xml"/>
+ </configurations>
+
+ <publications>
+ <!--get the artifact from our module name-->
+ <artifact conf="master"/>
+ </publications>
+
+ <dependencies>
+ <dependency org="com.hierynomus" name="smbj" rev="0.13.0"/>
+ <dependency org="net.engio" name="mbassador" rev="1.3.0"/>
Review Comment:
Absolutely, I understand.
_Some_ plugins contain a file called _howto_upgrade_${plugin}.md_ which
contains guidance on this. We could/should extract all of these files to a
centralized location.
For an example you can see
[howto_upgrade_solr.md](https://github.com/apache/nutch/blob/master/src/plugin/indexer-solr/howto_upgrade_solr.md).
##########
src/plugin/protocol-smb/plugin.xml:
##########
@@ -0,0 +1,53 @@
+<?xml version="1.0" encoding="UTF-8"?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<plugin
+ id="protocol-smb"
+ name="SMB Protocol based on https://github.com/hierynomus/smbj";
+ version="1.0.0"
+ provider-name="Hiran Chaudhuri">
+
+ <runtime>
+ <library name="asn-one-0.6.0.jar"/>
Review Comment:
If you follow the guidance I gave above regarding adapting the steps in
[howto_upgrade_solr.md](https://github.com/apache/nutch/blob/master/src/plugin/indexer-solr/howto_upgrade_solr.md)
then this stage will be done for you.
##########
src/plugin/build.xml:
##########
@@ -78,6 +78,7 @@
<ant dir="protocol-httpclient" target="deploy"/>
<ant dir="protocol-interactiveselenium" target="deploy" />
<ant dir="protocol-okhttp" target="deploy"/>
+ <ant dir="protocol-smb" target="deploy"/>
Review Comment:
If you look in the same file further down, you will find the
[test](https://github.com/apache/nutch/blob/master/src/plugin/build.xml#L115)
and
[clean](https://github.com/apache/nutch/blob/master/src/plugin/build.xml#L178)
targets.
##########
.gitignore:
##########
@@ -26,3 +26,7 @@ lib/spotbugs-*
ivy/dependency-check-ant/*
.gradle*
ivy/apache-rat-*
+.vscode
+crawl
Review Comment:
Ar you referring to the [Nutch
tutorial](https://cwiki.apache.org/confluence/display/NUTCH/NutchTutorial)?
The tutorial is not aimed towards _developers_ per say so it is not
concerned with accidentally committing these directories along with a PR.
I understand why you have added them in your local branch but they should
not be included in the PR. Each developer may locate the same directories in
different places...
##########
runNutch.sh:
##########
@@ -0,0 +1,32 @@
+#/bin/bash
Review Comment:
OK
##########
src/plugin/protocol-smb/ivy.xml:
##########
@@ -0,0 +1,47 @@
+<?xml version="1.0" ?>
+<!--
+ Licensed to the Apache Software Foundation (ASF) under one or more
+ contributor license agreements. See the NOTICE file distributed with
+ this work for additional information regarding copyright ownership.
+ The ASF licenses this file to You under the Apache License, Version 2.0
+ (the "License"); you may not use this file except in compliance with
+ the License. You may obtain a copy of the License at
+
+ http://www.apache.org/licenses/LICENSE-2.0
+
+ Unless required by applicable law or agreed to in writing, software
+ distributed under the License is distributed on an "AS IS" BASIS,
+ WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ See the License for the specific language governing permissions and
+ limitations under the License.
+-->
+
+<ivy-module xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";
+ xsi:noNamespaceSchemaLocation="http://ant.apache.org/ivy/schemas/ivy.xsd";
+ xmlns:ns0="http://ant.apache.org/ivy/maven"; version="2.0">
+ <info organisation="org.apache.nutch" module="${ant.project.name}">
+ <license name="Apache 2.0"/>
+ <ivyauthor name="Apache Nutch Team" url="http://nutch.apache.org"/>
+ <description>
+ Apache Nutch
+ </description>
+ </info>
+
+ <configurations>
+ <include file="../../..//ivy/ivy-configurations.xml"/>
+ </configurations>
+
+ <publications>
+ <!--get the artifact from our module name-->
+ <artifact conf="master"/>
+ </publications>
+
+ <dependencies>
+ <dependency org="com.hierynomus" name="smbj" rev="0.13.0"/>
+ <dependency org="net.engio" name="mbassador" rev="1.3.0"/>
+ <dependency org="org.bouncycastle" name="bcprov-jdk18on" rev="1.75"/>
+ <dependency org="com.hierynomus" name="asn-one" rev="0.6.0"/>
+ <dependency org="commons-io" name="commons-io" rev="2.17.0"/>
Review Comment:
Yes please ignore this comment apologies. That being said with `commons-io`
already defined in `ivy/ivy.xml` we can remove it from the plugin or at-least
synchronize the version with the version set in `ivy/ivy.xml`.
> Implement a protocol-smb plugin based on hierynomus/smbj
> --------------------------------------------------------
>
> Key: NUTCH-2856
> URL: https://issues.apache.org/jira/browse/NUTCH-2856
> Project: Nutch
> Issue Type: New Feature
> Components: external, plugin, protocol
> Reporter: Hiran Chaudhuri
> Assignee: Hiran Chaudhuri
> Priority: Major
> Fix For: 1.21
>
>
> The plugin protocol-smb advertized on
> [https://cwiki.apache.org/confluence/display/NUTCH/PluginCentral] actually
> refers to the JCIFS library. According to this library's homepage
> [https://www.jcifs.org/]:
> _If you're looking for the latest and greatest open source Java SMB library,
> this is not it. JCIFS has been in maintenance-mode-only for several years and
> although what it does support works fine (SMB1, NTLMv2, midlc, MSRPC and
> various utility classes), jCIFS does not support the newer SMB2/3 variants of
> the SMB protocol which is slowly becoming required (Windows 10 requires
> SMB2/3). JCIFS only supports SMB1 but Microsoft has deprecated SMB1 in their
> products. *So if SMB1 is disabled on your network, JCIFS' file related
> operations will NOT work.*_
> Looking at
> [https://en.wikipedia.org/wiki/Server_Message_Block#SMB_/_CIFS_/_SMB1:|https://en.wikipedia.org/wiki/Server_Message_Block#SMB_/_CIFS_/_SMB1]
> _Microsoft added SMB1 to the Windows Server 2012 R2 deprecation list in June
> 2013. Windows Server 2016 and some versions of Windows 10 Fall Creators
> Update do not have SMB1 installed by default._
> As a conclusion, the chances that SMB1 protocol is installed and/or
> configured are getting vastly smaller. Therefore some migration towards
> SMB2/3 is required. Luckily the JCIFS homepage lists alternatives:
> * [jcifs-codelibs|https://github.com/codelibs/jcifs]
> * [jcifs-ng|https://github.com/AgNO3/jcifs-ng]
> * [smbj|https://github.com/hierynomus/smbj]
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
