109 lines
6.5 KiB
Plaintext
109 lines
6.5 KiB
Plaintext
// image base directory::media/
|
|
ifndef::imagesdir[:imagesdir: ../media]
|
|
|
|
=== Kubernetes Cluster Setup for edX
|
|
|
|
This document describes the steps to set up a Kubernetes cluster for hosting the edX platform. Kubernetes is an open-source container orchestration platform that automates the deployment, scaling, and management of containerized applications.
|
|
The repository with all informations and the helm charts is available at link:https://github.com/openedx/openedx-k8s-harmony[openedx/openedx-k8s-harmony].
|
|
|
|
In particular, this project aims to provide the following benefits to Open edX operators:
|
|
|
|
* Ease of use and rapid deployment: This project aims to provide an Open edX hosting environment that just works out of the box, that can be easily upgraded, and that follows best practices for monitoring, security, etc.
|
|
* Lower costs by sharing resources where it makes sense. For example, by default Tutor's k8s feature will deploy a separate load balancer and ingress controller for each Open edX instance, instead of a shared ingress controller for all the instances in the cluster. Likewise for MySQL, MongoDB, ElasticSearch, and other resources. By using shared resources by default, costs can be dramatically reduced and operational monitoring and maintenance is greatly simplified.
|
|
** For setups with many small instances, this shared approach provides a huge cost savings with virtually no decrease in performance.
|
|
** For larger instances on the cluster that need dedicated resources, they can easily be configured to do so.
|
|
|
|
* Scalable hosting for instances of any size. This means for example that the default configuration includes autoscaling of LMS pods to handle increased traffic.
|
|
* Flexibility: this project aims to be "batteries included" and to support setting up all the resources that you need, with useful default configurations, but it is carefully designed so that operators can configure, replace, or disable any components as needed.
|
|
cite:[noauthor_tutor_nodate]
|
|
|
|
image:edX-k8s.drawio.svg[edX-K8s Deployment,align=center]
|
|
|
|
|
|
==== Setup Kubernetes Cluster ====
|
|
|
|
For setting up a Kubernetes cluster, you can use any cloud provider like AWS, GCP, Azure, or a self-hosted solution like Ubuntu with MicroK8s. The following steps are for setting up a Kubernetes cluster on Ubuntu with MicroK8s.
|
|
|
|
include::setup-k8s.asciidoc[]
|
|
|
|
==== Deploy Open edX on Kubernetes ====
|
|
|
|
*Caution:* This project is still in development and documentation is being updated. Please refer to the repository for the latest information. link:https://raw.githubusercontent.com/openedx/openedx-k8s-harmony/main/README.md[openedx/openedx-k8s-harmony/README.md]
|
|
|
|
### Step 1: Use Helm to provision a kubernetes cluster using this chart
|
|
|
|
#### Option 1a: Setting up Harmony Chart on a cloud-hosted Kubernetes Cluster (recommended)
|
|
|
|
For this recommended approach, you need to have a Kubernetes cluster in the cloud **with at least 12GB of usable
|
|
memory** (that's enough to test 2 Open edX instances).
|
|
|
|
1. Make sure you can access the cluster from your machine: run `kubectl cluster-info` and make sure it displays some
|
|
information about the cluster (e.g. two URLs).
|
|
2. Copy `values-example.yaml` to a new `values.yaml` file and edit it to put in your email address and customize
|
|
other settings. The email address is required for Lets Encrypt to issue HTTPS certificates. It is not shared
|
|
with anyone else. For a full configuration reference, see the `charts/harmony-chart/values.yaml` file.
|
|
3. Install [Helm](https://helm.sh/) if you don't have it already.
|
|
4. Add the Harmony Helm repository:
|
|
|
|
```shell
|
|
helm repo add openedx-harmony https://openedx.github.io/openedx-k8s-harmony
|
|
helm repo update
|
|
```
|
|
|
|
5. Install the cert-manager CRDs if using cert-manager:
|
|
|
|
```shell
|
|
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.10.1/cert-manager.crds.yaml --namespace=harmony
|
|
```
|
|
|
|
You can check the version of cert-manager that is going to be installed by the chart by checking the corresponding
|
|
line in the `charts/harmony-chart/Chart.yaml` file.
|
|
6. Install the Harmony chart by running:
|
|
|
|
```shell
|
|
helm install harmony --namespace harmony --create-namespace -f values.yaml openedx-harmony/harmony-chart
|
|
```
|
|
|
|
Note: in the future, if you apply changes to `values.yaml`, please run this command to update the deployment of the chart:
|
|
|
|
```shell
|
|
helm upgrade harmony --namespace harmony -f values.yaml openedx-harmony/harmony-chart
|
|
```
|
|
|
|
#### Option 1b: Setting up Harmony Chart locally on Minikube
|
|
|
|
*Note: if possible, it's preferred to use a cloud-hosted cluster instead (see previous section). But if you don't have a
|
|
cluster available in the cloud, you can use minikube to try this out locally. The minikube version does not support
|
|
HTTPS and is more complicated due to the need to use tunnelling.*
|
|
|
|
1. First, [install `minikube`](https://minikube.sigs.k8s.io/docs/start/) if you don't have it already.
|
|
2. Run `minikube start` (you can also use `minikube dashboard` to access the Kubernetes dashboard).
|
|
3. Add the Helm repository and install the Harmony chart using the `values-minikube.yaml` file as configuration:
|
|
|
|
```shell
|
|
helm repo add openedx-harmony https://openedx.github.io/openedx-k8s-harmony
|
|
helm repo update
|
|
helm install harmony --namespace harmony --create-namespace -f values-minikube.yaml openedx-harmony/harmony-chart
|
|
```
|
|
|
|
4. Run `minikube tunnel` (you may need to enter a password), and then you should be able to access the cluster (see
|
|
"External IP" below). If this approach is not working, an alternative is to run\
|
|
`minikube service harmony-ingress-nginx-controller -n harmony`\
|
|
and then go to the URL it says, e.g. `http://127.0.0.1:52806` plus `/cluster-echo-test`
|
|
(e.g. `http://127.0.0.1:52806/cluster-echo-test`)
|
|
5. In this case, skip step 2 ("Get the external IP") and use `127.0.0.1` as the external IP. You will need to remember
|
|
to include the port numbers shown above when accessing the instances.
|
|
|
|
### Step 2: Get the external IP
|
|
|
|
The [ingress NGINX Controller](https://kubernetes.github.io/ingress-nginx/) is used to automatically set up an HTTPS
|
|
reverse proxy for each Open edX instance as it gets deployed onto the cluster. There is just one load balancer with a
|
|
single external IP for all the instances on the cluster. To get its IP, use:
|
|
|
|
```shell
|
|
kubectl get svc -n harmony harmony-ingress-nginx-controller
|
|
```
|
|
|
|
To test that your load balancer is working, go to `http://<the external ip>/cluster-echo-test` .
|
|
You may need to ignore the HTTPS warnings, but then you should see a response with some basic JSON output.
|