Hi. If you're trying to use a regular file system, you should be aware of the
way the Linux kernel aggressively caches written data in RAM. It usually won't
flush out dirty data (from writes) until it feels like it, letting the writes
pile up in memory.
That's good for the usual case, but it's confusing to people who are used to a
single host to single disk relationship. Using an ethernet network instead of
an ATA cable provides more flexibility and exposes the reality behind the
illusion that transparent caching provides.
It is only safe to access the AoE target from other hosts if you are aware of
the implications of this caching. Specifically, only cluster file systems like
Red Hat's GFS are designed for sharing block storage to provide a single shared
file system. Other file systems like ext3 are "traditional" and rely on the
assumption that they have exclusive access to the block device. If other hosts
violate that assumption (e.g., by changing the filesystem metadata on the
shared block device), then data corruption is likely to occur.
Example:
Host "A" reads filesystem metadata from the block device, caching it in
its RAM.
Host "B" does the same.
Host "B" removes a file, changing the metadata in its RAM.
Host "B" flushes the data from RAM to the shared block device.
Host "A", unaware of host "B", attempts to read the file that B just
deleted.
Host "A" updates the atime in its cached metadata and flushes the
updated metadata to the shared block device.
You can see that the two hosts will create a jumbled mess of the file system if
they don't have a way to coordinate their actions. Cluster file systems
provide those coordination mechanisms.
You can force the Linux kernel to flush writes out to the backing storage by
using "sync" and other mechanisms, but performance will be affected, since a
disk, whether local or remote, cannot be as fast as RAM.
Jumbos are not required for many configurations but do increase throughput
significantly.
If I had to guess, the MAC address destinations are those of hosts on your
network, virtual or otherwise, that you didn't know about or have forgotten
about. The only address that is used without it needing to be on the network
is the broadcast address, ff:ff:ff:ff:ff:ff. Of course, you didn't say what
software you're using, so if you're using a program I don't know about, then
all bets are off. :)
On Oct 19, 2013, at 9:25 AM, 12navi...@gmail.com wrote:
> Hi, we set up AOE on our systems, then we mounted the exported partitions but
> unfortunately the changes on mounted partitions are not reflected on the main
> exported partition. Also other systems can not see the changes. Everything
> seems alright; except synchronization.
>
> Also i red in a blog post that we must use jumbo frames but i do not use
> jumbo frames. can this be the source of the problem??
> I checked the network traffic using "tcpdump ether proto 0x88a2" in the
> result of this command i saw bunch of frames with destionation addresses like
> bc:5f:f4:6b:af:58 that does not exist in my local network!!! A part of output
> looks like the following:
>
> 09:23:42.000905 30:85:a9:92:ba:ff (oui Unknown) > 1c:6f:65:7f:42:14 (oui
> Unknown), ethertype Unknown (0x88a2), length 1060:
> 0x0000: 1000 0000 0000 5a27 a1ed 4000 0224 0000 ......Z'..@..$..
> 0x0010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x00a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x00b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x00c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x00d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x00e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x00f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
> 0x0100: 0000 0000 0000 0000 0000 0000 0000 0000 ................
>
>
> As you can see most of the transferred frame is just zero!!! Any suggestions??
--
Ed Cashin
ecas...@coraid.com
------------------------------------------------------------------------------
October Webinars: Code for Performance
Free Intel webinars can help you accelerate application performance.
Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from
the latest Intel processors and coprocessors. See abstracts and register >
http://pubads.g.doubleclick.net/gampad/clk?id=60135031&iu=/4140/ostg.clktrk
_______________________________________________
Aoetools-discuss mailing list
Aoetools-discuss@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/aoetools-discuss
