I didn't see that one. That might explain the occasional errors. Thanks, guys.

On Thu, May 19, 2011 at 2:43 AM, Friso van Vollenhoven
<fvanvollenho...@xebia.com> wrote:
> I believe it's this:
>
> <property>
>  <name>mapred.submit.replication</name>
>  <value>10</value>
>  <description>The replication level for submitted job files.  This
>  should be around the square root of the number of nodes.
>  </description>
> </property>
>
> You can set it per job in the job specific conf and/or in mapred-site.xml.
>
>
> Friso
>
>
>
> On 19 mei 2011, at 03:42, Steve Cohen wrote:
>
>> Where is the default replication factor on job files set? Is it different 
>> then the dfs.replication setting in hdfs-site.xml?
>>
>> Sent from my iPad
>>
>> On May 18, 2011, at 9:10 PM, Joey Echeverria <j...@cloudera.com> wrote:
>>
>>> Did you run a map reduce job?
>>>
>>> I think the default replication factor on job files is 10, which
>>> obviously doesn't work well on a psuedo-distributed cluster.
>>>
>>> -Joey
>>>
>>> On Wed, May 18, 2011 at 5:07 PM, Steve Cohen <mail4st...@gmail.com> wrote:
>>>> Thanks for the answer. Earlier, I asked about why I get occasional not 
>>>> replicated yet errors. Now, I had dfs.replication set to one. What 
>>>> replication could it have been doing? Did the error messages actually mean 
>>>> that the file couldn't get created in the cluster?
>>>>
>>>> Thanks,
>>>> Steve Cohen
>>>>
>>>>
>>>>
>>>> On May 18, 2011, at 6:39 PM, Todd Lipcon <t...@cloudera.com> wrote:
>>>>
>>>>> Tried to send this, but apparently SpamAssassin finds emails about
>>>>> "replicas" to be spammy. This time with less rich text :)
>>>>>
>>>>> On Wed, May 18, 2011 at 3:35 PM, Todd Lipcon <t...@cloudera.com> wrote:
>>>>>>
>>>>>> Hi Steve,
>>>>>> Running setrep will indeed change those files. Changing 
>>>>>> "dfs.replication" just changes the default replication value for files 
>>>>>> created in the future. Replication level is a file-specific property.
>>>>>> Thanks
>>>>>> -Todd
>>>>>>
>>>>>> On Wed, May 18, 2011 at 3:32 PM, Steve Cohen <mail4st...@gmail.com> 
>>>>>> wrote:
>>>>>>>
>>>>>>> Say I add a datanode to a pseudo cluster and I want to change the
>>>>>>> replication factor to 2. I see that I can either run hadoop fs -setrep
>>>>>>> or change the hdfs-site.xml value for dfs.replication. But do either
>>>>>>> of these cause the existing blocks to replicate?
>>>>>>>
>>>>>>> Thanks,
>>>>>>> Steve Cohen
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Todd Lipcon
>>>>>> Software Engineer, Cloudera
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Todd Lipcon
>>>>> Software Engineer, Cloudera
>>>>
>>>
>>>
>>>
>>> --
>>> Joseph Echeverria
>>> Cloudera, Inc.
>>> 443.305.9434
>
>

Reply via email to