Ceph pools: a beginner’s guide
So you’ve got your Ceph cluster up and running but you don’t know your way around its innards. That’s perfectly fine, this guide is for you. Let’s learn how to manage a fundamental piece of Ceph: Ceph pools.
I don’t have a Ceph cluster
Any sufficiently advanced technology is indistinguishable from magic.Arthur C. Clarke
Ceph is pretty complicated as it is, I highly suggest you to follow this guide with a Ceph test cluster beneath your keyboard.
Ceph pools and RADOS
Ceph pools are the most basic and simple entity that you will encounter in Ceph. Simply put Ceph pools are logical groups of Ceph objects. Such objects live inside of Ceph, or rather they live inside RADOS. RADOS is Ceph’s core, it’s where all the magic happens and it may be difficult to grasp (link to rabbithole). You don’t need to understand what RADOS is to use Ceph pools, just remember that pools are logical, not physical.
There are two types of pools:
- Replicated pools: the default for Ceph, they store a set number of replicas.
- Erasure code pools: akin to RAID they store a number of chunks divided between actual data and erasure codes used to retrieve the object. This type of pool does not support the whole subset of Ceph operations, but it is usually cheaper to use (less disks needed, also more risks in some cases).
Now that you have an understanding of Ceph pools let’s get started. You will need to have a Ceph keyring on the host you’re using. The fastest way to get started is to ssh into a monitor node.
$ ceph osd lspools
To create a pool:
$ ceph osd pool create <pg-num> <pgp-num>
To create an erasure code pool:
$ ceph osd pool create <pg-num> <pgp-num> erasure
PG stands for placement groups. This is another complicated concept behind Ceph, know that you need to carefully select this value on a per-pool basis. Usually you can start with a low number and grow it along with the pool gaining size. High values might be tempting but they will create problems if not calculated. Refer to this paragraph for Ceph’s official recommendations.
Finally, pg-num and pgp-num should have the same value.
By default you can’t delete pools, you will need to enable a flag on monitors:
$ ceph tell mon.\* injectargs '--mon-allow-pool-delete=true' ## The following will delete the pool $ ceph osd pool delete <pool-name> <pool-name> --yes-i-really-really-mean-it $ ceph tell mon.\* injectargs '--mon-allow-pool-delete=false'
It is always a good idea to disable the flag once you have deleted what you needed.
Uploading data to Ceph pools
$ rados -p <pool-name> put <object-name> <file>
Downloading data from Ceph pools
$ rados -p <pool-name> get <object-name> <file>
Listing files in a pool
$ rados -p <pool-name> ls
Delete an object from a pool
$ rados -p <pool-name> rm <object-name>
Getting statistics about pools
$ rados df
This will output detailed statistics about the RADOS cluster. You may also use:
$ ceph df [detail]
This command will output similar statistics in a more user-friendly way (in my opinion). You can omit the “detail” command to reduce the number of information displayed.
Latest posts by mark (see all)
- Nvidia to acquire ARM for $40 billion - 16 September 2020
- All you need to know before buying a Raspberry Pi 4 - 9 September 2020
- How to set up Keepalived for High Availability and Load Balancing - 2 September 2020