Create Kubernetes Cluster with Terraform

This guide will walk you through the process of creating a Kubernetes Cluster with Terraform.

Prerequisites

Terraform installed on your machine.
Vietnix Cloud account.

Steps to Create Kubernetes Cluster with Terraform

1. Create a Terraform configuration file.

Terraform Configuration File

# main.tf
terraform {
  required_providers {
    openstack = {
      source  = "terraform-provider-openstack/openstack"
      version = ">= 1.45.0"
    }
  }
  required_version = ">= 1.6.0"
}

# auth provider
provider "openstack" {
  auth_url    = "https://auth.vietnix.cloud:5000/v3"
  user_name   = "your-username"
  password    = "your-password"
  tenant_name = "your-tenant-name"
  domain_id   = "your-domain-id"
}

# Create SSH keypair for Kubernetes cluster
resource "openstack_compute_keypair_v2" "terraform_k8s_keypair" {
  name       = "terraform_k8s_key"
  public_key = file("${var.ssh_key_file}.pub")
}

# Create network for Kubernetes cluster
resource "openstack_networking_network_v2" "terraform_k8s_network" {
  name        = "terraform_k8s_network"
  description = "Network for Terraform Kubernetes cluster"
}

# Create subnet for Kubernetes cluster
resource "openstack_networking_subnet_v2" "terraform_k8s_subnet" {
  name            = "terraform_k8s_subnet"
  network_id      = openstack_networking_network_v2.terraform_k8s_network.id
  cidr            = "10.0.0.0/24"
  gateway_ip      = "10.0.0.1"
  ip_version      = 4
  dns_nameservers = ["8.8.8.8", "8.8.4.4"]
}

# Create router for Kubernetes cluster
resource "openstack_networking_router_v2" "terraform_k8s_router" {
  name                = "terraform_k8s_router"
  external_network_id = var.external_network_id
  enable_snat         = true
}

# Connect router to subnet
resource "openstack_networking_router_interface_v2" "terraform_router_interface" {
  router_id = openstack_networking_router_v2.terraform_k8s_router.id
  subnet_id = openstack_networking_subnet_v2.terraform_k8s_subnet.id
}

# Create cluster template
resource "openstack_containerinfra_clustertemplate_v1" "terraform_k8s_template" {
  name                  = "terraform_k8s_template"
  coe                  = "kubernetes"
  image                = var.k8s_image_id
  flavor               = var.flavor_name
  external_network_id  = var.external_network_id
  server_type          = "vm"
  volume_driver        = "cinder"
  docker_storage_driver = "overlay2"
  docker_volume_size   = var.docker_volume_size
  network_driver       = "cilium"
  master_lb_enabled    = false
  floating_ip_enabled  = true
  fixed_network        = openstack_networking_network_v2.terraform_k8s_network.id
  fixed_subnet         = openstack_networking_subnet_v2.terraform_k8s_subnet.id
}

# Create Kubernetes cluster
resource "openstack_containerinfra_cluster_v1" "terraform_k8s_cluster" {
  name                = "terraform_k8s_cluster"
  cluster_template_id = openstack_containerinfra_clustertemplate_v1.terraform_k8s_template.id
  keypair            = openstack_compute_keypair_v2.terraform_k8s_keypair.name
  master_count       = 1
  node_count         = 2
  master_flavor      = "medium.2c.4g"
  flavor             = "small.1c.2g"
  create_timeout     = 60
  docker_volume_size = 10
  
  labels = {
    boot_volume_type         = "nvmer3"
    docker_volume_type       = "nvmer3"
    boot_volume_size         = "20"
    etcd_lb_disabled         = "true"
    heat_container_agent_tag = "5.3.11"
    cloud_provider_tag       = "v1.31.1"
    cloud_provider_enabled   = "true"
    cinder_csi_plugin_tag    = "v1.31.1"
    csi_snapshotter_tag      = "v8.0.1"
    csi_attacher_tag         = "v4.6.1"
    flannel_tag              = "v0.11.0-amd64"
    cinder_csi_enabled       = "true"
    availability_zone        = "nova"
    use_podman               = "true"
    etcd_tag                 = "v3.5.13"
    cgroup_driver            = "systemd"
    octavia_api_lb_flavor    = ""
    octavia_default_flavor   = ""
    auto_scaling_enabled     = "True"
    cluster_upgrade_method   = "replace"
    network_driver           = "cilium"
    kube_tag                 = "v1.31.2"
    kube_version             = "v1.31.2"
    hyperkube_image          = "docker.io/virtuozzo/hci-binary-hyperkube"
    autoscaler_tag           = "1.31.0"
    coredns_tag              = "1.11.3"
    k8s_keystone_auth_tag    = "v1.31.1"
    cilium_tag               = "v1.16.3"
    container_runtime        = "containerd"
    monitoring_enabled       = "False"
    minion_floating_ip_disabled = "True"
    cilium_routing_mode      = "native"
  }
}

Variables File

Create a variables.tf file to define your variables:

# variables.tf
variable "ssh_key_file" {
  description = "Path to the SSH private key file"
  type        = string
  default     = "~/.ssh/id_rsa"
}

variable "external_network_id" {
  description = "ID of the external network"
  type        = string
}

variable "k8s_image_id" {
  description = "ID of the image to use for Kubernetes nodes"
  type        = string
}

variable "flavor_name" {
  description = "Name of the flavor for the cluster template"
  type        = string
  default     = "small.1c.2g"
}

variable "docker_volume_size" {
  description = "Size of the Docker volume in GB"
  type        = number
  default     = 10
}

variable "master_count" {
  description = "Number of master nodes"
  type        = number
  default     = 1
}

variable "node_count" {
  description = "Number of worker nodes"
  type        = number
  default     = 2
}

variable "master_flavor" {
  description = "Flavor for master nodes"
  type        = string
  default     = "medium.2c.4g"
}

variable "worker_flavor" {
  description = "Flavor for worker nodes"
  type        = string
  default     = "small.1c.2g"
}

Outputs File

Create an outputs.tf file to display important information:

# outputs.tf
output "k8s_cluster_id" {
  description = "ID of the created Kubernetes cluster"
  value       = openstack_containerinfra_cluster_v1.terraform_k8s_cluster.id
}

output "k8s_cluster_name" {
  description = "Name of the created Kubernetes cluster"
  value       = openstack_containerinfra_cluster_v1.terraform_k8s_cluster.name
}

output "k8s_cluster_status" {
  description = "Status of the Kubernetes cluster"
  value       = openstack_containerinfra_cluster_v1.terraform_k8s_cluster.status
}

output "k8s_cluster_api_address" {
  description = "API address of the Kubernetes cluster"
  value       = openstack_containerinfra_cluster_v1.terraform_k8s_cluster.api_address
}

output "k8s_cluster_master_addresses" {
  description = "Master node addresses"
  value       = openstack_containerinfra_cluster_v1.terraform_k8s_cluster.master_addresses
}

output "k8s_cluster_node_addresses" {
  description = "Worker node addresses"
  value       = openstack_containerinfra_cluster_v1.terraform_k8s_cluster.node_addresses
}

output "k8s_cluster_kubeconfig" {
  description = "Kubeconfig content for the cluster"
  value       = openstack_containerinfra_cluster_v1.terraform_k8s_cluster.kubeconfig
  sensitive   = true
}

2. Run `terraform init` to initialize the Terraform configuration.

terraform init

3. Run `terraform plan` to plan the Terraform configuration.

terraform plan

tip

You can add variables to the plan command to see the plan with specific values.

terraform plan -var="ssh_key_file=~/.ssh/id_rsa" -var="external_network_id=your-external-network-id" -var="k8s_image_id=your-k8s-image-id" -var="flavor_name=small.1c.2g" -var="docker_volume_size=10" -var="master_count=1" -var="node_count=2"

4. Run `terraform apply` to create the Kubernetes cluster.

terraform apply

tip

You can add variables to the apply command to create the cluster with specific values.

terraform apply -var="ssh_key_file=~/.ssh/id_rsa" -var="external_network_id=your-external-network-id" -var="k8s_image_id=your-k8s-image-id" -var="flavor_name=small.1c.2g" -var="docker_volume_size=10" -var="master_count=1" -var="node_count=2"

5. Run `terraform destroy` to destroy the Kubernetes cluster.

terraform destroy

Verify the Kubernetes Cluster

After the Kubernetes cluster is created, you can verify it by:

1. Get the kubeconfig

# Get the kubeconfig content
terraform output k8s_cluster_kubeconfig

# Save kubeconfig to a file
terraform output -raw k8s_cluster_kubeconfig > kubeconfig
export KUBECONFIG=./kubeconfig

2. Check cluster status

kubectl cluster-info
kubectl get nodes
kubectl get pods --all-namespaces

3. Deploy a test application

kubectl run nginx --image=nginx --port=80
kubectl expose pod nginx --type=NodePort --port=80
kubectl get services

Other Useful Guides

Create VM with Terraform

Prerequisites​

Steps to Create Kubernetes Cluster with Terraform​

1. Create a Terraform configuration file.​

Terraform Configuration File​

Variables File​

Outputs File​

2. Run terraform init to initialize the Terraform configuration.​

3. Run terraform plan to plan the Terraform configuration.​

4. Run terraform apply to create the Kubernetes cluster.​

5. Run terraform destroy to destroy the Kubernetes cluster.​

Verify the Kubernetes Cluster​

1. Get the kubeconfig​

2. Check cluster status​

3. Deploy a test application​

Other Useful Guides​