[jira] [Created] (NUTCH-1577) Add target for creating eclipse project
Tejas Patil created NUTCH-1577: -- Summary: Add target for creating eclipse project Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 2.1, 1.6 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Fix For: 1.7, 2.2 Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated NUTCH-1577: --- Attachment: NUTCH-1577.trunk.patch Here is a patch for trunk. How to use it: * on a SVN checkout of trunk, apply the patch * run ant eclipse * In eclipse: File - Import - Existing projects into workspace. Give the patch of the trunk directory. Initially it would show some errors (red dots) but those will go away after it auto-compiles the newly imported project. Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671305#comment-13671305 ] Tejas Patil edited comment on NUTCH-1577 at 5/31/13 10:19 AM: -- Here is a patch for trunk. How to use it: * on a SVN checkout of trunk, apply the patch * run ant eclipse * In eclipse: File - Import - Existing projects into workspace. Give the path of the trunk directory. Initially it would show some errors (red dots) but those will go away after it auto-compiles the newly imported project. was (Author: tejasp): Here is a patch for trunk. How to use it: * on a SVN checkout of trunk, apply the patch * run ant eclipse * In eclipse: File - Import - Existing projects into workspace. Give the patch of the trunk directory. Initially it would show some errors (red dots) but those will go away after it auto-compiles the newly imported project. Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Comment Edited] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671305#comment-13671305 ] Tejas Patil edited comment on NUTCH-1577 at 5/31/13 10:22 AM: -- Here is a patch for trunk. How to use it: * on a SVN checkout of trunk, apply the patch * run ant eclipse * In eclipse: File - Import - Existing projects into workspace. Give the path of the trunk directory. Initially it would show some errors (red dots) but those will go away after it builds the workspace. was (Author: tejasp): Here is a patch for trunk. How to use it: * on a SVN checkout of trunk, apply the patch * run ant eclipse * In eclipse: File - Import - Existing projects into workspace. Give the path of the trunk directory. Initially it would show some errors (red dots) but those will go away after it auto-compiles the newly imported project. Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil updated NUTCH-1577: --- Attachment: NUTCH-1577.2.x.patch Patch for 2.x Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.2.x.patch, NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (NUTCH-1578) Upgrade to Hadoop 1.2.0
Markus Jelsma created NUTCH-1578: Summary: Upgrade to Hadoop 1.2.0 Key: NUTCH-1578 URL: https://issues.apache.org/jira/browse/NUTCH-1578 Project: Nutch Issue Type: Task Reporter: Markus Jelsma Assignee: Markus Jelsma Priority: Trivial Fix For: 1.7, 2.2 Hadoop 1.2.0 finally has the ability to run mappers in parallel when running in local mode. In trunk at least the generator seems to run slightly faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1578) Upgrade to Hadoop 1.2.0
[ https://issues.apache.org/jira/browse/NUTCH-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671448#comment-13671448 ] Markus Jelsma commented on NUTCH-1578: -- Everything runs fine as opposed to Hadoop 1.1.2 and all tests pass. Upgrade to Hadoop 1.2.0 --- Key: NUTCH-1578 URL: https://issues.apache.org/jira/browse/NUTCH-1578 Project: Nutch Issue Type: Task Reporter: Markus Jelsma Assignee: Markus Jelsma Priority: Trivial Fix For: 1.7, 2.2 Hadoop 1.2.0 finally has the ability to run mappers in parallel when running in local mode. In trunk at least the generator seems to run slightly faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Created] (NUTCH-1579) NPE when using solr indexing
shantharam anandkumar created NUTCH-1579: Summary: NPE when using solr indexing Key: NUTCH-1579 URL: https://issues.apache.org/jira/browse/NUTCH-1579 Project: Nutch Issue Type: Bug Components: crawldb Affects Versions: 2.4 Environment: MacOS Reporter: shantharam anandkumar Fix For: 2.1 NPE while running crawl with solar here is command bin/nutch crawl urls/ -solr http://localhost:8983/solr -depth 3 -topN 20 Exception in thread main java.lang.NullPointerException at java.util.Hashtable.put(Hashtable.java:394) at java.util.Properties.setProperty(Properties.java:143) at org.apache.hadoop.conf.Configuration.set(Configuration.java:438) at org.apache.nutch.indexer.IndexerJob.createIndexJob(IndexerJob.java:129) at org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:44) at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:69) at org.apache.nutch.crawl.Crawler.run(Crawler.java:194) at org.apache.nutch.crawl.Crawler.run(Crawler.java:252) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.nutch.crawl.Crawler.main(Crawler.java:261) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1579) NPE when using solr indexing
[ https://issues.apache.org/jira/browse/NUTCH-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671677#comment-13671677 ] kiran commented on NUTCH-1579: -- Hi anandkumar, You should post any questions to u...@nutch.apache.org first and if its a potential bug and on developers advise, you should create Jira's. You will get questions answered on mailing lists than here. Also, you should look at your logs (logs/hadoop.log) first. Please close the issue and shift it to mailing list. Thank you NPE when using solr indexing Key: NUTCH-1579 URL: https://issues.apache.org/jira/browse/NUTCH-1579 Project: Nutch Issue Type: Bug Components: crawldb Affects Versions: 2.4 Environment: MacOS Reporter: shantharam anandkumar Fix For: 2.1 NPE while running crawl with solar here is command bin/nutch crawl urls/ -solr http://localhost:8983/solr -depth 3 -topN 20 Exception in thread main java.lang.NullPointerException at java.util.Hashtable.put(Hashtable.java:394) at java.util.Properties.setProperty(Properties.java:143) at org.apache.hadoop.conf.Configuration.set(Configuration.java:438) at org.apache.nutch.indexer.IndexerJob.createIndexJob(IndexerJob.java:129) at org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:44) at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:69) at org.apache.nutch.crawl.Crawler.run(Crawler.java:194) at org.apache.nutch.crawl.Crawler.run(Crawler.java:252) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.nutch.crawl.Crawler.main(Crawler.java:261) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671746#comment-13671746 ] Lewis John McGibbney commented on NUTCH-1577: - Committed @revision 1488356 in 2.x HEAD. [~tejasp] the only reason I commit your patch is to push on wish hopefully getting 2.x RC out today. Sorry to burst you bubble my friend ;) Great work on the patch, imports in to Eclipse smooth, the same as well aged whisky goes down the throat :0) Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.2.x.patch, NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1576) Need to keep hotStore.flush() exception catching
[
https://issues.apache.org/jira/browse/NUTCH-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671748#comment-13671748
]
Lewis John McGibbney commented on NUTCH-1576:
-
Hi James, thanks for pointing this out.
I applied your patch, it applies with some fuzz, then I compile and I get
{code}
[javac] Compiling 1 source file to /home/lewismc/ASF/2.x/build/classes
[javac]
/home/lewismc/ASF/2.x/src/java/org/apache/nutch/host/HostDb.java:92: exception
java.io.IOException is never thrown in body of corresponding try statement
[javac] } catch (IOException e) {
[javac] ^
[javac] 1 error
BUILD FAILED
/home/lewismc/ASF/2.x/build.xml:101: Compile failed; see the compiler error
output for details.
{code}
Need to keep hotStore.flush() exception catching
Key: NUTCH-1576
URL: https://issues.apache.org/jira/browse/NUTCH-1576
Project: Nutch
Issue Type: Bug
Affects Versions: 2.2
Reporter: James Sullivan
Priority: Minor
Attachments: patch.txt
Still need exception checking for hoststorelflush() for those who have to use
gora-core 0.2.1 otherwise Nutch 2.x will not compile.
!-- Uncomment this to use SQL as Gora backend. It should be noted that the
gora-sql 0.1.1-incubating artifact is NOT compatable with gora-core 0.3.
Users should
downgrade to gora-core 0.2.1 in order to use SQL as a backend. --
Index: src/java/org/apache/nutch/host/HostDb.java
===
--- java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(revision 1487824)
+++ java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(working copy)
@@ -87,7 +87,11 @@
CacheHost removeFromCacheHost = notification.getValue();
if (removeFromCacheHost != NULL_HOST) {
if (removeFromCacheHost.timestamp lastFlush.get()) {
-hostStore.flush();
+try {
+ hostStore.flush();
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
lastFlush.set(System.currentTimeMillis());
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1576) Need to keep hotStore.flush() exception catching
[
https://issues.apache.org/jira/browse/NUTCH-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671752#comment-13671752
]
Lewis John McGibbney commented on NUTCH-1576:
-
I replaced the IOException with an Exception and it now compiles.
Can you do the same and check this out for me?
Need to keep hotStore.flush() exception catching
Key: NUTCH-1576
URL: https://issues.apache.org/jira/browse/NUTCH-1576
Project: Nutch
Issue Type: Bug
Affects Versions: 2.2
Reporter: James Sullivan
Priority: Minor
Attachments: patch.txt
Still need exception checking for hoststorelflush() for those who have to use
gora-core 0.2.1 otherwise Nutch 2.x will not compile.
!-- Uncomment this to use SQL as Gora backend. It should be noted that the
gora-sql 0.1.1-incubating artifact is NOT compatable with gora-core 0.3.
Users should
downgrade to gora-core 0.2.1 in order to use SQL as a backend. --
Index: src/java/org/apache/nutch/host/HostDb.java
===
--- java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(revision 1487824)
+++ java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(working copy)
@@ -87,7 +87,11 @@
CacheHost removeFromCacheHost = notification.getValue();
if (removeFromCacheHost != NULL_HOST) {
if (removeFromCacheHost.timestamp lastFlush.get()) {
-hostStore.flush();
+try {
+ hostStore.flush();
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
lastFlush.set(System.currentTimeMillis());
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1576) Need to keep hotStore.flush() exception catching
[
https://issues.apache.org/jira/browse/NUTCH-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lewis John McGibbney updated NUTCH-1576:
Fix Version/s: 2.2
Need to keep hotStore.flush() exception catching
Key: NUTCH-1576
URL: https://issues.apache.org/jira/browse/NUTCH-1576
Project: Nutch
Issue Type: Bug
Affects Versions: 2.2
Reporter: James Sullivan
Priority: Minor
Fix For: 2.2
Attachments: patch.txt
Still need exception checking for hoststorelflush() for those who have to use
gora-core 0.2.1 otherwise Nutch 2.x will not compile.
!-- Uncomment this to use SQL as Gora backend. It should be noted that the
gora-sql 0.1.1-incubating artifact is NOT compatable with gora-core 0.3.
Users should
downgrade to gora-core 0.2.1 in order to use SQL as a backend. --
Index: src/java/org/apache/nutch/host/HostDb.java
===
--- java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(revision 1487824)
+++ java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(working copy)
@@ -87,7 +87,11 @@
CacheHost removeFromCacheHost = notification.getValue();
if (removeFromCacheHost != NULL_HOST) {
if (removeFromCacheHost.timestamp lastFlush.get()) {
-hostStore.flush();
+try {
+ hostStore.flush();
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
lastFlush.set(System.currentTimeMillis());
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1576) Need to keep hotStore.flush() exception catching
[
https://issues.apache.org/jira/browse/NUTCH-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671771#comment-13671771
]
Lewis John McGibbney commented on NUTCH-1576:
-
I am applying this patch (with Exception) and pushing the RC.
we can check it it is all OK in the testing of the RC.
Thanks
Lewis
Need to keep hotStore.flush() exception catching
Key: NUTCH-1576
URL: https://issues.apache.org/jira/browse/NUTCH-1576
Project: Nutch
Issue Type: Bug
Affects Versions: 2.2
Reporter: James Sullivan
Priority: Minor
Fix For: 2.2
Attachments: patch.txt
Still need exception checking for hoststorelflush() for those who have to use
gora-core 0.2.1 otherwise Nutch 2.x will not compile.
!-- Uncomment this to use SQL as Gora backend. It should be noted that the
gora-sql 0.1.1-incubating artifact is NOT compatable with gora-core 0.3.
Users should
downgrade to gora-core 0.2.1 in order to use SQL as a backend. --
Index: src/java/org/apache/nutch/host/HostDb.java
===
--- java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(revision 1487824)
+++ java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(working copy)
@@ -87,7 +87,11 @@
CacheHost removeFromCacheHost = notification.getValue();
if (removeFromCacheHost != NULL_HOST) {
if (removeFromCacheHost.timestamp lastFlush.get()) {
-hostStore.flush();
+try {
+ hostStore.flush();
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
lastFlush.set(System.currentTimeMillis());
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (NUTCH-1576) Need to keep hotStore.flush() exception catching
[
https://issues.apache.org/jira/browse/NUTCH-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Lewis John McGibbney resolved NUTCH-1576.
-
Resolution: Fixed
Committed @r1488366 in 2.x HEAD thanks James.
As mentioned we can and should test this out more thoroughly during the RC
testing.
Need to keep hotStore.flush() exception catching
Key: NUTCH-1576
URL: https://issues.apache.org/jira/browse/NUTCH-1576
Project: Nutch
Issue Type: Bug
Affects Versions: 2.2
Reporter: James Sullivan
Priority: Minor
Fix For: 2.2
Attachments: patch.txt
Still need exception checking for hoststorelflush() for those who have to use
gora-core 0.2.1 otherwise Nutch 2.x will not compile.
!-- Uncomment this to use SQL as Gora backend. It should be noted that the
gora-sql 0.1.1-incubating artifact is NOT compatable with gora-core 0.3.
Users should
downgrade to gora-core 0.2.1 in order to use SQL as a backend. --
Index: src/java/org/apache/nutch/host/HostDb.java
===
--- java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(revision 1487824)
+++ java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(working copy)
@@ -87,7 +87,11 @@
CacheHost removeFromCacheHost = notification.getValue();
if (removeFromCacheHost != NULL_HOST) {
if (removeFromCacheHost.timestamp lastFlush.get()) {
-hostStore.flush();
+try {
+ hostStore.flush();
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
lastFlush.set(System.currentTimeMillis());
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1579) NPE when using solr indexing
[ https://issues.apache.org/jira/browse/NUTCH-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lewis John McGibbney updated NUTCH-1579: Fix Version/s: (was: 2.1) 2.3 NPE when using solr indexing Key: NUTCH-1579 URL: https://issues.apache.org/jira/browse/NUTCH-1579 Project: Nutch Issue Type: Bug Components: crawldb Affects Versions: 2.4 Environment: MacOS Reporter: shantharam anandkumar Fix For: 2.3 NPE while running crawl with solar here is command bin/nutch crawl urls/ -solr http://localhost:8983/solr -depth 3 -topN 20 Exception in thread main java.lang.NullPointerException at java.util.Hashtable.put(Hashtable.java:394) at java.util.Properties.setProperty(Properties.java:143) at org.apache.hadoop.conf.Configuration.set(Configuration.java:438) at org.apache.nutch.indexer.IndexerJob.createIndexJob(IndexerJob.java:129) at org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:44) at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:69) at org.apache.nutch.crawl.Crawler.run(Crawler.java:194) at org.apache.nutch.crawl.Crawler.run(Crawler.java:252) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.nutch.crawl.Crawler.main(Crawler.java:261) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Updated] (NUTCH-1578) Upgrade to Hadoop 1.2.0
[ https://issues.apache.org/jira/browse/NUTCH-1578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lewis John McGibbney updated NUTCH-1578: Fix Version/s: (was: 2.2) 2.3 Upgrade to Hadoop 1.2.0 --- Key: NUTCH-1578 URL: https://issues.apache.org/jira/browse/NUTCH-1578 Project: Nutch Issue Type: Task Reporter: Markus Jelsma Assignee: Markus Jelsma Priority: Trivial Fix For: 1.7, 2.3 Hadoop 1.2.0 finally has the ability to run mappers in parallel when running in local mode. In trunk at least the generator seems to run slightly faster. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671823#comment-13671823 ] Tejas Patil commented on NUTCH-1577: Committed to trunk at rev1488396. My next task is to update the wiki page with the new steps and then close this jira. Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.2.x.patch, NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1576) Need to keep hotStore.flush() exception catching
[
https://issues.apache.org/jira/browse/NUTCH-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671830#comment-13671830
]
James Sullivan commented on NUTCH-1576:
---
Thanks for fixing this. Just compiled it with gora-core 0.2.1 and it worked
fine.
Need to keep hotStore.flush() exception catching
Key: NUTCH-1576
URL: https://issues.apache.org/jira/browse/NUTCH-1576
Project: Nutch
Issue Type: Bug
Affects Versions: 2.2
Reporter: James Sullivan
Priority: Minor
Fix For: 2.2
Attachments: patch.txt
Still need exception checking for hoststorelflush() for those who have to use
gora-core 0.2.1 otherwise Nutch 2.x will not compile.
!-- Uncomment this to use SQL as Gora backend. It should be noted that the
gora-sql 0.1.1-incubating artifact is NOT compatable with gora-core 0.3.
Users should
downgrade to gora-core 0.2.1 in order to use SQL as a backend. --
Index: src/java/org/apache/nutch/host/HostDb.java
===
--- java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(revision 1487824)
+++ java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(working copy)
@@ -87,7 +87,11 @@
CacheHost removeFromCacheHost = notification.getValue();
if (removeFromCacheHost != NULL_HOST) {
if (removeFromCacheHost.timestamp lastFlush.get()) {
-hostStore.flush();
+try {
+ hostStore.flush();
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
lastFlush.set(System.currentTimeMillis());
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Closed] (NUTCH-1579) NPE when using solr indexing
[ https://issues.apache.org/jira/browse/NUTCH-1579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] shantharam anandkumar closed NUTCH-1579. Resolution: Fixed As per Kiran posting in user growup NPE when using solr indexing Key: NUTCH-1579 URL: https://issues.apache.org/jira/browse/NUTCH-1579 Project: Nutch Issue Type: Bug Components: crawldb Affects Versions: 2.4 Environment: MacOS Reporter: shantharam anandkumar Fix For: 2.3 NPE while running crawl with solar here is command bin/nutch crawl urls/ -solr http://localhost:8983/solr -depth 3 -topN 20 Exception in thread main java.lang.NullPointerException at java.util.Hashtable.put(Hashtable.java:394) at java.util.Properties.setProperty(Properties.java:143) at org.apache.hadoop.conf.Configuration.set(Configuration.java:438) at org.apache.nutch.indexer.IndexerJob.createIndexJob(IndexerJob.java:129) at org.apache.nutch.indexer.solr.SolrIndexerJob.run(SolrIndexerJob.java:44) at org.apache.nutch.crawl.Crawler.runTool(Crawler.java:69) at org.apache.nutch.crawl.Crawler.run(Crawler.java:194) at org.apache.nutch.crawl.Crawler.run(Crawler.java:252) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65) at org.apache.nutch.crawl.Crawler.main(Crawler.java:261) -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Resolved] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tejas Patil resolved NUTCH-1577. Resolution: Fixed Updated the documentation page [RunNutchInEclipse|http://wiki.apache.org/nutch/RunNutchInEclipse] to reflect the new steps. Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.2.x.patch, NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1576) Need to keep hotStore.flush() exception catching
[
https://issues.apache.org/jira/browse/NUTCH-1576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671961#comment-13671961
]
Hudson commented on NUTCH-1576:
---
Integrated in Nutch-nutchgora #627 (See
[https://builds.apache.org/job/Nutch-nutchgora/627/])
NUTCH-1576 Need to keep hotStore.flush() exception catching (Revision
1488366)
Result = SUCCESS
lewismc : http://svn.apache.org/viewvc/nutch/branches/2.x/?view=revrev=1488366
Files :
* /nutch/branches/2.x/CHANGES.txt
* /nutch/branches/2.x/src/java/org/apache/nutch/host/HostDb.java
Need to keep hotStore.flush() exception catching
Key: NUTCH-1576
URL: https://issues.apache.org/jira/browse/NUTCH-1576
Project: Nutch
Issue Type: Bug
Affects Versions: 2.2
Reporter: James Sullivan
Priority: Minor
Fix For: 2.2
Attachments: patch.txt
Still need exception checking for hoststorelflush() for those who have to use
gora-core 0.2.1 otherwise Nutch 2.x will not compile.
!-- Uncomment this to use SQL as Gora backend. It should be noted that the
gora-sql 0.1.1-incubating artifact is NOT compatable with gora-core 0.3.
Users should
downgrade to gora-core 0.2.1 in order to use SQL as a backend. --
Index: src/java/org/apache/nutch/host/HostDb.java
===
--- java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(revision 1487824)
+++ java/workspace/2.x/src/java/org/apache/nutch/host/HostDb.java
(working copy)
@@ -87,7 +87,11 @@
CacheHost removeFromCacheHost = notification.getValue();
if (removeFromCacheHost != NULL_HOST) {
if (removeFromCacheHost.timestamp lastFlush.get()) {
-hostStore.flush();
+try {
+ hostStore.flush();
+} catch (IOException e) {
+ throw new RuntimeException(e);
+}
lastFlush.set(System.currentTimeMillis());
}
}
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
[jira] [Commented] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671962#comment-13671962 ] Hudson commented on NUTCH-1577: --- Integrated in Nutch-nutchgora #627 (See [https://builds.apache.org/job/Nutch-nutchgora/627/]) NUTCH-1577 Add target for creating eclipse project (Revision 1488356) Result = SUCCESS lewismc : http://svn.apache.org/viewvc/nutch/branches/2.x/?view=revrev=1488356 Files : * /nutch/branches/2.x/CHANGES.txt * /nutch/branches/2.x/build.xml Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.2.x.patch, NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
MeetUp
Hi All, I am writing to both lists to see if there are any individuals around next week (in SFO/bay area or further afield) who would like to get together for a midweek or weekend meetup? Renato and myself are presenting on Gora at CassandraSummit in SFO so it is as good a time as any to break the ice and have a f2f meetup between communities. Either respond here or get me off list. Looking forward to having many beers during the week Renato is up from Peru, would be even better if we could touch base with others. Have a great weekend Lewis -- *Lewis*
[jira] [Commented] (NUTCH-1577) Add target for creating eclipse project
[ https://issues.apache.org/jira/browse/NUTCH-1577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=13671977#comment-13671977 ] Hudson commented on NUTCH-1577: --- Integrated in Nutch-trunk # (See [https://builds.apache.org/job/Nutch-trunk//]) NUTCH-1577 Add target for creating eclipse project (Revision 1488396) Result = SUCCESS tejasp : http://svn.apache.org/viewvc/nutch/trunk/?view=revrev=1488396 Files : * /nutch/trunk/CHANGES.txt * /nutch/trunk/build.xml Add target for creating eclipse project --- Key: NUTCH-1577 URL: https://issues.apache.org/jira/browse/NUTCH-1577 Project: Nutch Issue Type: Improvement Affects Versions: 1.6, 2.1 Reporter: Tejas Patil Assignee: Tejas Patil Priority: Minor Labels: build, eclipse Fix For: 1.7, 2.2 Attachments: NUTCH-1577.2.x.patch, NUTCH-1577.trunk.patch Currently, loading Nutch source code in Eclipse as a project is cumbersome and involves lot of manual steps as given over [wiki|http://wiki.apache.org/nutch/RunNutchInEclipse]. It would be great to automate this. Adding a ant target to do that would remove burden off from developers. -- This message is automatically generated by JIRA. If you think it was sent incorrectly, please contact your JIRA administrators For more information on JIRA, see: http://www.atlassian.com/software/jira
