Skip to content

Quick Start Guide for Ceph-CSI-Operator

1. Prerequisites

Before deploying the Ceph-CSI-Operator, ensure the following requirements are met:

  • A Kubernetes cluster (supported version recommended) or OpenShift 4.x cluster
  • Ceph cluster (supported version recommended)
  • kubectl CLI installed (or oc CLI for OpenShift)

Note: In this guide, we will use minimal configurations to deploy the Ceph-CSI-Operator and drivers. You can customize the configurations as per your requirements.

2. Install the Ceph-CSI Operator

2.1 Standard Kubernetes Installation

kubectl create -f deploy/all-in-one/install.yaml

2.2 OpenShift Installation

For OpenShift clusters, use the OpenShift-specific installer that includes the required SecurityContextConstraints:

kubectl create -f deploy/all-in-one/install-openshift.yaml

verify the installation:

kubectl get pods -n ceph-csi-operator-system
NAME                                                    READY   STATUS    RESTARTS   AGE
ceph-csi-operator-controller-manager-67d45fd9ff-zgst7   2/2     Running   0          40s

3. Deploy Ceph-CSI Drivers

Once the operator is installed, deploy the Ceph-CSI drivers:

3.1 Deploy the RBD Driver

echo '
apiVersion: csi.ceph.io/v1
kind: Driver
metadata:
  name: rbd.csi.ceph.com
  namespace: ceph-csi-operator-system
' | kubectl create -f -

3.2 Deploy the CephFS Driver

echo '
apiVersion: csi.ceph.io/v1
kind: Driver
metadata:
  name: cephfs.csi.ceph.com
  namespace: ceph-csi-operator-system
' | kubectl create -f -

3.3 Deploy the NVMe-oF Driver

echo '
apiVersion: csi.ceph.io/v1
kind: Driver
metadata:
  name: nvmeof.csi.ceph.com
  namespace: ceph-csi-operator-system
' | kubectl create -f -

3.4 Deploy the Ceph-NFS Driver

echo '
apiVersion: csi.ceph.io/v1
kind: Driver
metadata:
  name: nfs.csi.ceph.com
  namespace: ceph-csi-operator-system
' | kubectl create -f -

4. Verify Installation

To verify the installation, check the status of the Ceph-CSI components:

kubectl get pod -n ceph-csi-operator-system
NAME                                                    READY   STATUS    RESTARTS   AGE
ceph-csi-operator-controller-manager-744dc99cb5-scxxh   2/2     Running   0          45s
cephfs.csi.ceph.com-ctrlplugin-5847c998b5-xf85m         5/5     Running   0          27s
cephfs.csi.ceph.com-nodeplugin-r6pkt                    2/2     Running   0          27s
nvmeof.csi.ceph.com-ctrlplugin-6936b889a4-ya99h         5/5     Running   0          27s
nvmeof.csi.ceph.com-nodeplugin-ab67g                    2/2     Running   0          27s
nfs.csi.ceph.com-ctrlplugin-76fd4f5b4c-smk2g            5/5     Running   0          27s
nfs.csi.ceph.com-nodeplugin-kbzms                       2/2     Running   0          27s
rbd.csi.ceph.com-ctrlplugin-6965dcfdb8-w88kn            5/5     Running   0          4m35s
rbd.csi.ceph.com-nodeplugin-lnm4n                       2/2     Running   0          4m35s

5. Create CephConnection

Create a CephConnection CR to connect to the Ceph cluster:

echo '
apiVersion: csi.ceph.io/v1
kind: CephConnection
metadata:
  name: ceph-connection
  namespace: ceph-csi-operator-system
spec:
  monitors:
  - 10.98.44.171:6789
' | kubectl create -f -

6. Create ClientProfile

Create a ClientProfile CR to define the client configuration which points to the CephConnection CR and the CephFS and RBD configurations:

echo '
apiVersion: csi.ceph.io/v1
kind: ClientProfile
metadata:
  name: storage
  namespace: ceph-csi-operator-system
spec:
  cephConnectionRef:
    name: ceph-connection
  cephFs:
    subVolumeGroup: csi
' | kubectl create -f -

[!IMPORTANT] The ClientProfile name (storage in this example) will be used as the clusterID parameter in your StorageClass and VolumeSnapshotClass resources.

7. Create Ceph Secrets

Before creating storage classes, create Kubernetes Secrets with Ceph credentials for CSI operations.

For detailed instructions on creating Ceph users and Kubernetes Secrets, refer to the upstream Ceph-CSI documentation:

[!NOTE] - Create secrets in the namespace where your applications will create PVCs - NFS volumes use the same CephFS secret format since NFS is built on CephFS

8. Create StorageClasses

Create StorageClasses using the upstream Ceph-CSI examples:

[!IMPORTANT] ClusterID and ClientProfile Mapping

The clusterID parameter must match your ClientProfile CR name:

# In your StorageClass
parameters:
  clusterID: storage  # Must match the ClientProfile name from step 6

9. Create VolumeSnapshotClasses (Optional)

For snapshot support, use the upstream Ceph-CSI VolumeSnapshotClass examples:

Ensure the clusterID parameter matches your ClientProfile name:

parameters:
  clusterID: storage  # Must match your ClientProfile name

10. Test Storage Provisioning

Test your setup using the Ceph-CSI PVC examples:

The PVC should reach Bound status, indicating successful provisioning.

11. Clean Up Resources

To clean up the resources, delete the cepconnection, clientprofile and drivers:

kubectl delete cephconnection ceph-connection -n ceph-csi-operator-system
kubectl delete clientprofile storage -n ceph-csi-operator-system
kubectl delete driver rbd.csi.ceph.com -n ceph-csi-operator-system
kubectl delete driver cephfs.csi.ceph.com -n ceph-csi-operator-system
kubectl delete driver nvmeof.csi.ceph.com -n ceph-csi-operator-system
kubectl delete driver nfs.csi.ceph.com -n ceph-csi-operator-system

To uninstall the Ceph-CSI-Operator, delete the operator:

For standard Kubernetes: 

kubectl delete -f deploy/all-in-one/install.yaml

For OpenShift: 

kubectl delete -f deploy/all-in-one/install-openshift.yaml

Verify the deletion:

kubectl get pods -n ceph-csi-operator-system
No resources found in ceph-csi-operator-system namespace.