Mohamed, Well, that’s interesting… and in direct conflict with what is written in the documentation<http://ceph.com/docs/master/rados/operations/cache-tiering/> (wherein it describes relative sizing as proportional to the cache pool’s capacity). I am presently reinstalling, so I’ll give that a try. Thanks very much.
-don- From: Mohamed Pakkeer [mailto:[email protected]] Sent: 30 April, 2015 11:45 To: Don Doerner Cc: [email protected] Subject: Re: [ceph-users] RHEL7/HAMMER cache tier doesn't flush or evict? Hi Don, You have to provide the target size through target_max_bytes. target_dirty_ratio and target_full_ratio values are based upon target_max_bytes. If you provide target_max bytes as 200 GB and target_dirty_ratio as 0.4, ceph will start the fulshing, when the cache tier has 60GB of dirty objects. Mohamed On Thu, Apr 30, 2015 at 11:56 PM, Don Doerner <[email protected]<mailto:[email protected]>> wrote: Hi Mohamed, I did not. But: • I confirmed that (by default) cache_target_dirty_ratio was set to 0.4 (40%) and cache_target_full_ratio was set to 0.8 (80%). • I did not set target_max_bytes (I felt that the simple, pure relative sizing controls were sufficient for my experiment). • I confirmed that (by default) cache_min_flush_age and cache_min_evict_age were set to 0 (so no required delay for either flushing or eviction). Given these settings, I expected to see: • Flushing begin to happen at 40% of my cache tier size (~200 GB, as it happened), or about 80 GB. Or earlier. • Eviction begin to happen at 80% of my cache tier size, or about 160 GB. Or earlier. • Cache tier capacity would only exceed 80% only if the flushing process couldn’t keep up with the ingest process for fairly long periods of time (at the observed ingest rate of ~400 MB/sec, a few hundred seconds). Am I misunderstanding something? Thank you very much for your assistance! -don- From: Mohamed Pakkeer [mailto:[email protected]<mailto:[email protected]>] Sent: 30 April, 2015 10:52 To: Don Doerner Cc: [email protected]<mailto:[email protected]> Subject: Re: [ceph-users] RHEL7/HAMMER cache tier doesn't flush or evict? Hi Don, Did you configure target_ dirty_ratio, target_full_ratio and target_max_bytes? K.Mohamed Pakkeer On Thu, Apr 30, 2015 at 10:26 PM, Don Doerner <[email protected]<mailto:[email protected]>> wrote: All, Synopsis: I can’t get cache tiering to work in HAMMER on RHEL7. Process: 1. Fresh install of HAMMER on RHEL7 went well. 2. Crush map adapted to provide two “root” level resources a. “ctstorage”, to use as a cache tier based on very high-performance, high IOPS SSD (intrinsic journal). 2 OSDs. b. “ecstorage”, to use as an erasure-coded poolbased on low-performance, cost effective storage (extrinsic journal). 12 OSDs. 3. Established a pool “ctpool”, 32 PGs on ctstorage (pool size = min_size = 1). Ran a quick RADOS bench test, all worked as expected. Cleaned up. 4. Established a pool “ecpool”, 256 PGs on ecstorage (5+3 profile). Ran a quick RADOS bench test, all worked as expected. Cleaned up. 5. Ensured that both pools were empty (i.e., “rados ls” shows no objects) 6. Put the cache tier on the erasure coded storage (one Bloom hit set, interval 900 seconds), set up the overlay. Used defaults for flushing and eviction. No errors. 7. Started a 3600-second write test to ecpool. Objects piled up in ctpool (as expected) – went past the 40% mark (as expected), then past the 80% mark (unexpected), then ran into the wall (95% full – very unexpected). Using “rados df”, I can see that the cache tier is full (duh!) but not one single object lives in the ecpool. Nothing was ever flushed, nothing was ever evicted. Thought I might be misreading that, so I went back to SAR data that I captured during the test: the SSDs were the only [ceph] devices that sustained I/O. I based this experiment on another (much more successful) experiment that I performed using GIANT (.1) on RHEL7 a couple of weeks ago (wherein I used RAM as a cache tier); that all worked. It seems there are at least three possibilities… • I forgot a critical step this time around. • The steps needed for a cache tier in HAMMER are different than the steps needed in GIANT (and different than the documentation online). • There is a problem with HAMMER in the area of cache tier. Has anyone successfully deployed cache-tiering in HAMMER? Did you have to do anything unusual? Do you see any steps that I missed? Regards, -don- ________________________________ The information contained in this transmission may be confidential. Any disclosure, copying, or further distribution of confidential information is not permitted unless such privilege is explicitly granted in writing by Quantum. Quantum reserves the right to have electronic communications, including email and attachments, sent across its networks filtered through anti virus and spam software programs and retain such messages in order to comply with applicable data security and retention requirements. Quantum is not responsible for the proper and complete transmission of the substance of this communication or for any delay in its receipt. _______________________________________________ ceph-users mailing list [email protected]<mailto:[email protected]> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com<https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.ceph.com_listinfo.cgi_ceph-2Dusers-2Dceph.com&d=BQMFaQ&c=8S5idjlO_n28Ko3lg6lskTMwneSC-WqZ5EBTEEvDlkg&r=DAW8QzIBpV_iBddECxqMq8sRPZPQOBqikPmeCBg26bM&m=Z-3d5aMnP4pxkHHCAf6pW_kRjxRPDF3dx6MfuVGZDgw&s=0C6rvtLPHnddUXaVLBff4sszXT6cKSkGmnZag2VVLfk&e=> -- Thanks & Regards K.Mohamed Pakkeer Mobile- 0091-8754410114 -- Thanks & Regards K.Mohamed Pakkeer Mobile- 0091-8754410114
_______________________________________________ ceph-users mailing list [email protected] http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
