Hello,
On Sun, 26 Aug 2018 22:23:53 +0400 James Watson wrote: > Hi CEPHers, > > I need to design an HA CEPH object storage system. The first question that comes to mind is why? Why does it need to be Ceph and why object based (RGW)? >From what's stated below it seems that nobody at your end has in depth experience with Ceph and related HW and the bit about getting that amount of data in via WiFi boggles the mind. HA is implied with Ceph, the bits that feed into it or read out if it will be your problem/responsibility. > The scenario is that we > are recording HD Videos and end of the day we need to copy all these video > files (each file has approx 15 TB ) to our storage system. > How many of these 15TB files, how long do you need to keep them, estimated future growth, etc? > 1)Which would be the best tech in storage to transfer these PBs size loads > of videos to CEPH based object storage wirelessly. > As you already figured out, wireless is going to be an issue. Are those 15TB files accumulated locally (sounds odd for remote cameras to have such storage)? Unless you deploy something beefier in the network department OR go to sneakernet this won't work well. > 2)How should I design my CEPH in the scale of PBs and make sure its future > proof. > Lots of examples around if you're looking around here and on google. Dense nodes (lots of HDDs) work if you have enough of them, otherwise smaller is preferable. > 3)What are the latest hardware components I might require to accomplish > this task? > You're entering the realm of paid consulting here, which I'm sure some people present might be willing to do. Vendors like Supermicro probably, too. Heck, you can buy it off the shelf even (not the only one out there): http://www.fujitsu.com/global/products/computing/storage/eternus-cd/s2/ In general your needs sound a lot like large streaming (in and out), which has less demands on the HW then lots of small IOPS. > > I am a software developer and am new to this domain. Kindly request all to > provide the name of even the most basic of hardware components required for > the setup so that I can do a cost estimation and compare with other techs. > You can do cheap with Ceph, _if_ you know what you're doing. OTOH Ceph emphasizes redundancy first, so compared with other solutions it may not be as cheap as it could be, depending on configuration choices and performance requirements. > My novice solution so far: > > 1. Transmitting module if using WiFi (802.11ac (aka Gigabit Wifi) max 200 > Mbps speed) to transfer a file of size 15 TB to CEPH Storage takes 7 days !! > > 2.CEPH needs to be configured with High Availability > A SAN with FC networking in place (GEN 6 SANS) using NVMe SSD with HBAs > that support NVMe over Fibre Channel giving a transfer rate of 16 Gbps to > Host Server. > Ceph is all about local storage per node, SAN and FC are anathema and NVMe is likely not needed in your scenario, at least not for actual storage space. Christian -- Christian Balzer Network/Systems Engineer ch...@gol.com Rakuten Communications _______________________________________________ ceph-users mailing list ceph-users@lists.ceph.com http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
