Re: NameNode sync using DUAL_ASYNC mode on IGFS

[dkarachentsev]

Hi,

You may use it in cases when you use temporary files that should be removed
after node stop, or any other data that don't require persistence.

-Dmitry.



--
View this message in context: 
http://apache-ignite-users.70518.x6.nabble.com/NameNode-sync-using-DUAL-ASYNC-mode-on-IGFS-tp11644p11756.html
Sent from the Apache Ignite Users mailing list archive at Nabble.com.

Re: NameNode sync using DUAL_ASYNC mode on IGFS

[Masayuki Takahashi]

Hi Dmitriy,

Thanks for explaining. I understand.
In situations using Name node, it's better to use DUAL_SYNC mode.

May I ask one more?

> Basically it means that every time a data is changed in Ignite, it will also 
> be changed in the underlying HDFS, either synchronously or asynchronously, 
> based on the IGFS configuration.

When using PRIMARY mode, I think that name node is not referenced.
What kind of use case is assumed for PRIMARY mode?

thanks.

2017-04-04 15:11 GMT+09:00 Dmitriy Setrakyan :
> Hi Massayuki,
>
> Ignite itself does not have a concept of a NameNode. It goes directly to the
> cluster node responsible for storing the data based on a key's hashcode.
>
> The only time when a NameNode would come into a play, is when Hadoop HDFS is
> configured as an underlying write-through file system. Basically it means
> that every time a data is changed in Ignite, it will also be changed in the
> underlying HDFS, either synchronously or asynchronously, based on the IGFS
> configuration. In this case, HDFS would contact the NameNode whenever the
> data is written into it, based on the its own native protocol, which has
> nothing to do with Ignite itself.
>
> Let me know if you have more questions.
>
> D.
>
> On Sun, Apr 2, 2017 at 5:47 AM, Masayuki Takahashi 
> wrote:
>>
>> Hi,
>>
>> I am trying to use IGFS on HDFS.
>>
>> If I set DUAL_ASYNC to IGFS mode and execute 'hdfs dfs -put ...', when
>> does the file info write to HDFS NameNode?
>>
>> Also, if I set PRIMARY to IGFS mode and put a new file, does the file
>> info write to HDFS NameNode?
>>
>> thanks.
>> --
>> Masayuki Takahashi
>
>



-- 
高橋 真之

Re: NameNode sync using DUAL_ASYNC mode on IGFS

[Dmitriy Setrakyan]

Hi Massayuki,

Ignite itself does not have a concept of a NameNode. It goes directly to
the cluster node responsible for storing the data based on a key's hashcode.

The only time when a NameNode would come into a play, is when Hadoop HDFS
is configured as an underlying write-through file system. Basically it
means that every time a data is changed in Ignite, it will also be changed
in the underlying HDFS, either synchronously or asynchronously, based on
the IGFS configuration. In this case, HDFS would contact the NameNode
whenever the data is written into it, based on the its own native protocol,
which has nothing to do with Ignite itself.

Let me know if you have more questions.

D.

On Sun, Apr 2, 2017 at 5:47 AM, Masayuki Takahashi 
wrote:

> Hi,
>
> I am trying to use IGFS on HDFS.
>
> If I set DUAL_ASYNC to IGFS mode and execute 'hdfs dfs -put ...', when
> does the file info write to HDFS NameNode?
>
> Also, if I set PRIMARY to IGFS mode and put a new file, does the file
> info write to HDFS NameNode?
>
> thanks.
> --
> Masayuki Takahashi
>

NameNode sync using DUAL_ASYNC mode on IGFS

[Masayuki Takahashi]

Hi,

I am trying to use IGFS on HDFS.

If I set DUAL_ASYNC to IGFS mode and execute 'hdfs dfs -put ...', when
does the file info write to HDFS NameNode?

Also, if I set PRIMARY to IGFS mode and put a new file, does the file
info write to HDFS NameNode?

thanks.
--
Masayuki Takahashi