Re: [Lucene-hadoop Wiki] Trivial Update of "Bigtable&Sawzall" by udanax

2007-02-15 Thread edward yoon
Greetings guys,I wrote some stuff in wiki page last night to share, to disscuss and get feedbacks about my views in BigTable. In Fact, The amount of writings that I would have posted wasn't small and that's why I created a new wiki page. My intention was to get feedbacks about my understandings of

[jira] Commented: (HADOOP-601) we need some rpc retry framework

2007-02-15 Thread Johan Oskarson (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473338 ] Johan Oskarson commented on HADOOP-601: --- Any news on this one? Whenever the jobtracker is busy and it makes ou

Re: [Lucene-hadoop Wiki] Trivial Update of "Bigtable&Sawzall" by udanax

2007-02-15 Thread Jim White
Seems to me that wiki pages are absurdly cheap and we're better off encouraging participation than discouraging it. As for noise from wiki update notifications, that must be a subscriber choice because I don't get them. For folks who don't like them I suggest turning that option off. Also wi

[jira] Updated: (HADOOP-1014) map/reduce is corrupting data between map and reduce

2007-02-15 Thread Devaraj Das (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Devaraj Das updated HADOOP-1014: Attachment: zero-size-inmem-fs.patch TestMapRed.java Riccardo, the problem with yo

Re: [jira] Updated: (HADOOP-1014) map/reduce is corrupting data between map and reduce

2007-02-15 Thread Nt Never
You are totally right, my bad. Your patched version passes all the JUnit tests now. I will now test it on my largest jobs and compare with 0.9.2. Should take about 7-8 hours. Thanks. On 2/15/07, Devaraj Das (JIRA) <[EMAIL PROTECTED]> wrote: [ https://issues.apache.org/jira/browse/HADOOP-1

new Hadoop committer: Tom White

2007-02-15 Thread Doug Cutting
I'm pleased to announce that the Lucene PMC has voted to add Tom White as a Hadoop committer. Welcome, Tom! The initiation ritual is to add yourself to the Credits page and re-publish the site. http://lucene.apache.org/hadoop/credits.html Doug

[jira] Updated: (HADOOP-990) Datanode doesn't retry when write to one (full)drive fail

2007-02-15 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated HADOOP-990: Status: Open (was: Patch Available) > This is not required. Datanode already considers only 98% of

[jira] Commented: (HADOOP-1014) map/reduce is corrupting data between map and reduce

2007-02-15 Thread Albert Chern (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473442 ] Albert Chern commented on HADOOP-1014: -- Disabling the in-mem merge has fixed this for me. I went back and chec

[jira] Commented: (HADOOP-1014) map/reduce is corrupting data between map and reduce

2007-02-15 Thread Mike Smith (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473441 ] Mike Smith commented on HADOOP-1014: Devaraj, As I email you yesterday, that patch solved my problem. But, I wi

[jira] Assigned: (HADOOP-492) Global counters

2007-02-15 Thread David Bowen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] David Bowen reassigned HADOOP-492: -- Assignee: David Bowen (was: Owen O'Malley) > Global counters > --- > >

[jira] Updated: (HADOOP-1014) map/reduce is corrupting data between map and reduce

2007-02-15 Thread Arun C Murthy (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Arun C Murthy updated HADOOP-1014: -- Status: Patch Available (was: Open) Submitting patch for review on behalf of Devaraj (whose I

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473451 ] Doug Cutting commented on HADOOP-492: - I talked to Owen about this last week. My concerns are: 1. We should onl

[jira] Commented: (HADOOP-1014) map/reduce is corrupting data between map and reduce

2007-02-15 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473453 ] Owen O'Malley commented on HADOOP-1014: --- This looks like a reasonable workaround for right now. Although it se

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread David Bowen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473458 ] David Bowen commented on HADOOP-492: This requirement is not an exact match with the Metrics API. A MetricsReco

[jira] Commented: (HADOOP-1014) map/reduce is corrupting data between map and reduce

2007-02-15 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473462 ] Hadoop QA commented on HADOOP-1014: --- +1, because http://issues.apache.org/jira/secure/attachment/12351227/zero-si

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473474 ] Doug Cutting commented on HADOOP-492: - That sounds like a great plan. > Do we need some sort of counter naming c

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread Runping Qi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473486 ] Runping Qi commented on HADOOP-492: --- If we change Reporter method to: void incrCounter(Enum key, long amount);

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473489 ] Doug Cutting commented on HADOOP-492: - > How does a user to accumulate on his/her specific counters? public enum

[jira] Commented: (HADOOP-442) slaves file should include an 'exclude' section, to prevent "bad" datanodes and tasktrackers from disrupting a cluster

2007-02-15 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473491 ] dhruba borthakur commented on HADOOP-442: - 0. Nice work on removing the STOPPED state. Makes the system less

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread David Bowen (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473496 ] David Bowen commented on HADOOP-492: I like the enum approach. It solves the namespace problem, and provides co

[jira] Updated: (HADOOP-990) Datanode doesn't retry when write to one (full)drive fail

2007-02-15 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-990?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated HADOOP-990: Attachment: HADOOP-990-3.patch finally I am able to write without errors to a datanode that has one

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473501 ] Doug Cutting commented on HADOOP-492: - > The only possible drawback I can see is the need to send longer strings

[jira] Created: (HADOOP-1023) better links to mailing list archives

2007-02-15 Thread Daniel Naber (JIRA)
better links to mailing list archives - Key: HADOOP-1023 URL: https://issues.apache.org/jira/browse/HADOOP-1023 Project: Hadoop Issue Type: Improvement Components: documentation Repor

[jira] Commented: (HADOOP-927) MapReduce is Broken for User-Defined Classes

2007-02-15 Thread Albert Chern (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473524 ] Albert Chern commented on HADOOP-927: - This issue was the same as Hadoop-964 so it can be closed now. > MapReduc

[jira] Resolved: (HADOOP-927) MapReduce is Broken for User-Defined Classes

2007-02-15 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-927?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Doug Cutting resolved HADOOP-927. - Resolution: Duplicate > MapReduce is Broken for User-Defined Classes > --

Re: new Hadoop committer: Tom White

2007-02-15 Thread Tom White
Thanks Doug. I'm looking forward to helping out with Hadoop! (I'm still figuring out how to generate the site from the xdocs. Doesn't seem to be in the top level build...) Tom

Re: new Hadoop committer: Tom White

2007-02-15 Thread Doug Cutting
Tom White wrote: (I'm still figuring out how to generate the site from the xdocs. Forrest. cd src/docs forrest cp -pr build/site/* ../../docs Doesn't seem to be in the top level build...) Yeah, it should be. Either that, or we should document it. But that would substantially reduc

[jira] Commented: (HADOOP-442) slaves file should include an 'exclude' section, to prevent "bad" datanodes and tasktrackers from disrupting a cluster

2007-02-15 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-442?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473544 ] dhruba borthakur commented on HADOOP-442: - Regarding comment 5 above, it actually might make sense to have a

[jira] Updated: (HADOOP-947) isReplicationInProgress() is very heavyweight

2007-02-15 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur updated HADOOP-947: Attachment: isReplicationInProgress.patch This patch fixes the logic that determines whethe

[jira] Assigned: (HADOOP-947) isReplicationInProgress() is very heavyweight

2007-02-15 Thread dhruba borthakur (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] dhruba borthakur reassigned HADOOP-947: --- Assignee: dhruba borthakur > isReplicationInProgress() is very heavyweight >

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread Runping Qi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473552 ] Runping Qi commented on HADOOP-492: --- As a user, I normally am interested only in the final accumulated values of m

[jira] Updated: (HADOOP-702) DFS Upgrade Proposal

2007-02-15 Thread Nigel Daley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nigel Daley updated HADOOP-702: --- Attachment: TestPlan-HdfsUpgrade.html Attached updated test plan for the latest design doc. There ar

[jira] Commented: (HADOOP-985) Namenode should identify DataNodes as ip:port instead of hostname:port

2007-02-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473558 ] Hairong Kuang commented on HADOOP-985: -- The open request takes the client host name as a parameter. Upon receivi

[jira] Updated: (HADOOP-985) Namenode should identify DataNodes as ip:port instead of hostname:port

2007-02-15 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated HADOOP-985: Attachment: HADOOP-985-3.patch attached 3.patch. Updated patch removed 'clientMachine' argument fro

[jira] Commented: (HADOOP-492) Global counters

2007-02-15 Thread Doug Cutting (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473562 ] Doug Cutting commented on HADOOP-492: - > I normally am interested only in the final accumulated values of my coun

[jira] Updated: (HADOOP-972) Improve the rack-aware replica placement performance

2007-02-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-972: - Attachment: (was: rack_performance.patch) > Improve the rack-aware replica placement performa

[jira] Updated: (HADOOP-972) Improve the rack-aware replica placement performance

2007-02-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-972: - Attachment: rack_performance.patch When I explained my patch to Dhruba this morning, he suggested

[jira] Updated: (HADOOP-972) Improve the rack-aware replica placement performance

2007-02-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-972: - Attachment: (was: rack_performance.patch) > Improve the rack-aware replica placement performa

[jira] Updated: (HADOOP-972) Improve the rack-aware replica placement performance

2007-02-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hairong Kuang updated HADOOP-972: - Attachment: rack_performance.patch > Improve the rack-aware replica placement performance > -

[jira] Updated: (HADOOP-985) Namenode should identify DataNodes as ip:port instead of hostname:port

2007-02-15 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Raghu Angadi updated HADOOP-985: Attachment: HADOOP-985-4.patch Thanks Hairong. minor change in 4.patch. > Namenode should identif

[jira] Commented: (HADOOP-972) Improve the rack-aware replica placement performance

2007-02-15 Thread Milind Bhandarkar (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473569 ] Milind Bhandarkar commented on HADOOP-972: -- A few comments: Replicator class needs to move out of FSNameSy

[jira] Commented: (HADOOP-1014) map/reduce is corrupting data between map and reduce

2007-02-15 Thread Owen O'Malley (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1014?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473570 ] Owen O'Malley commented on HADOOP-1014: --- We got a ConcurrentModificationException in the ramfs, which is likel

[jira] Commented: (HADOOP-985) Namenode should identify DataNodes as ip:port instead of hostname:port

2007-02-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473574 ] Hairong Kuang commented on HADOOP-985: -- The patch looks good. I have two comments: 1. ClientProtocolVersionNumb

[jira] Commented: (HADOOP-972) Improve the rack-aware replica placement performance

2007-02-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-972?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473576 ] Hairong Kuang commented on HADOOP-972: -- I agree that ReplicaChooser is a better name. But I am not sure if the c

[jira] Commented: (HADOOP-985) Namenode should identify DataNodes as ip:port instead of hostname:port

2007-02-15 Thread Raghu Angadi (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473585 ] Raghu Angadi commented on HADOOP-985: - Thanks Hairong. I will include both in a new patch. This changes the wha

[jira] Commented: (HADOOP-947) isReplicationInProgress() is very heavyweight

2007-02-15 Thread Wendy Chien (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473601 ] Wendy Chien commented on HADOOP-947: The logic looks fine, but the new name, hasReachedReplicationFactor, implies

Hadoop nightly build failure

2007-02-15 Thread hadoop-dev
init: [mkdir] Created dir: /tmp/hadoop-nightly/build [mkdir] Created dir: /tmp/hadoop-nightly/build/classes [mkdir] Created dir: /tmp/hadoop-nightly/build/src [mkdir] Created dir: /tmp/hadoop-nightly/build/webapps/task/WEB-INF [mkdir] Created dir: /tmp/hadoop-nightly/build/weba

[jira] Commented: (HADOOP-985) Namenode should identify DataNodes as ip:port instead of hostname:port

2007-02-15 Thread Hairong Kuang (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473630 ] Hairong Kuang commented on HADOOP-985: -- I also prefer option (a). I would open another jira issue to investigate

[jira] Updated: (HADOOP-1006) The "-local" option does work properly with test programs

2007-02-15 Thread Gautam Kowshik (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gautam Kowshik updated HADOOP-1006: --- Attachment: TestSeqFile.patch > The "-local" option does work properly with test programs >

[jira] Updated: (HADOOP-1006) The "-local" option does work properly with test programs

2007-02-15 Thread Gautam Kowshik (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gautam Kowshik updated HADOOP-1006: --- Affects Version/s: 0.11.1 Status: Patch Available (was: Open) Changed TestSe

[jira] Commented: (HADOOP-1006) The "-local" option does work properly with test programs

2007-02-15 Thread Hadoop QA (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473634 ] Hadoop QA commented on HADOOP-1006: --- -1, because the patch command could not apply the latest attachment (http://

[jira] Commented: (HADOOP-1006) The "-local" option does work properly with test programs

2007-02-15 Thread Gautam Kowshik (JIRA)
[ https://issues.apache.org/jira/browse/HADOOP-1006?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#action_12473635 ] Gautam Kowshik commented on HADOOP-1006: Changed only the TestSequenceFile program as the other test program

[jira] Created: (HADOOP-1024) Add stable version line to the website front page

2007-02-15 Thread Owen O'Malley (JIRA)
Add stable version line to the website front page - Key: HADOOP-1024 URL: https://issues.apache.org/jira/browse/HADOOP-1024 Project: Hadoop Issue Type: Improvement Reporter: Owen O'