
The module creates a managed Kubernetes cluster (GKE) in GCP.
Supported Kubernetes versions are 1.29
and newer.
Once you have a Corewide Solutions Portal account, this one-time action will use your browser session to retrieve credentials:
shellterraform login solutions.corewide.com
Initialize mandatory providers:
Copy and paste into your Terraform configuration and insert the variables:
hclmodule "tf_gcp_k8s_gke" {
source = "solutions.corewide.com/google-cloud/tf-gcp-k8s-gke/google"
version = "~> 4.1.0"
# specify module inputs here or try one of the examples below
...
}
Initialize the setup:
shellterraform init
Corewide DevOps team strictly follows Semantic Versioning
Specification
to
provide our clients with products that have predictable upgrades between versions. We
recommend
pinning
patch versions of our modules using pessimistic
constraint operator (~>
) to prevent breaking changes during upgrades.
To get new features during the upgrades (without breaking compatibility), use
~> 4.1
and run
terraform init -upgrade
For the safest setup, use strict pinning with version = "4.1.0"
All notable changes to this project are documented here.
The format is based on Keep a Changelog, and this project adheres to Semantic Versioning.
gateway_api_config_channel
variable, which defines the configuration options for API Gateway channelBREAKING CHANGE: all the node pools will have the unique suffix appended to their name. To incorporate the suffixes for the node pool names, node pool redeployment is needed. Please, see the Upgrade notes section.
default_node_pool
variable which defines the configurational options for the maintenance
node pool that is created unconditionally<cluster_name>/node_pool_name
label to all node poolscreate_before_destroy
lifecycle option to node poolsname
by adding a random suffixvpc_native
and cluster_private_nodes_enabled
property values to true
enabled
observability
variable which allows the user to configure Google-managed cluster monitoring and deploy Google Cloud Managed Service for Prometheus (GMP) as GKE add-ondeletion_protection_enabled
variable which allows the user to destroy the cluster, by default, is true
. Set to false
to destroy the clusternullable
parameter to all variablescluster_version
to 1.29
google
and google-beta
providers' version to 6.2
auto_upgrade
option to true
if STABLE
release channel is selected (required by provider)(Last version compatible with Terraform Google v4)
min_master_version
parameter to google_container_cluster
resourcecluster_version
to 1.27
node_pools.image
to COS_CONTAINERD
cluster_version
to 1.23
vpc_native
variable, to fix absence of IP aliasingcluster_master_cidr
variable, to fix unrestricted inter-cluster communicationallowed_mgmt_networks
, cluster_private_nodes_enabled
variables, to fix unrestricted access to clusterBREAKING CHANGE: now all node pools have corresponding names in the state instead of abstract indexes which aren't compatible with a new version. Upgrade from an older version is possible with manual changes, see Upgrade Notes section
for_each
meta-argument instead of count
. With this change, it is now possible to store and reference Node Pool resources using their corresponding names instead of abstract indexestags
parameter to identify valid sources or targets for network firewalls1.20+
v1.x
to v2.x
Module from v2.0
has changed handling of node pool copies from count
meta-argument to for_each
, which isn't compatible with an old version. After the module version is upgraded, re-init module and update naming in Terraform state of managed indexed resources:
As an example, node_pools
variable contains two node pools, where:
node_pools[0].name = "maintenance"
node_pools[1].name = "app"
Then, you need to move their state this way:
bashterraform state mv 'module.gke.google_container_node_pool.gke[0]' 'module.gke.google_container_node_pool.gke["maintenance"]'
terraform state mv 'module.gke.google_container_node_pool.gke[1]' 'module.gke.google_container_node_pool.gke["app"]'
v2.x
to v3.x
Module from v3.0
has changed Google providers version which isn't compatible with an old version. After the module version is upgraded, re-init module to upgrade Google providers version.
Upgrade Google providers version on project level to ~> 6.2:
hclrequired_providers {
google = {
source = "hashicorp/google"
version = "~> 6.2"
}
google-beta = {
source = "hashicorp/google-beta"
version = "~> 6.2"
}
}
Upgrade project dependencies:
bashterraform init --upgrade
v3.x
to v4.x
Module from v4.0
introduced the maintenance
node pool, which is created unconditionally. If the maintenance
node pool was configured using the previous module version, move the supported parameters of the maintenance
node pool from the node_pools
parameter to the default_node_pool
parameter.
Module from v4.0
enabled the cluster_private_nodes_enabled
parameter by default. To avoid cluster re-creation, set the cluster_private_nodes_enabled
parameter to false. Alternatively, if the cluster_private_nodes_enabled
parameter will be set to true
, the cluster must be re-created using terraform destroy --target module.<module-name>
and terraform apply --target module.<module-name>
Module from v4.0
has changed the node pool property name
by adding a random suffix. For the changes to take effect node pools must be redeployed.
Replace the <module-name>
with the name, assigned to the name of module
resource, that utilizes this module in the following command and execute it to replace the node pools:
bashterraform apply $(tf state list | grep module.<module-name>.google_container_node_pool | xargs -I{} echo "-replace={} " | sed -E 's/\[/["/g; s/\]/"]/g')
If the cloud.google.com/gke-nodepool
label was used in the application configuration, it has to be replaced with <name_prefix>/node-pool-name
For example, if the application pod used the following configuration on the cluster with name_prefix = production
:
bashnodeSelector:
cloud.google.com/gke-nodepool: application
It should be replaced with the following configuration, to ensure that the pods will be scheduled both before and after the module version migration:
bashnodeSelector:
production/node-pool-name: application
GKE with one node pool: node pool is created unconditionally under maintenance
pool name. The parameters like the node size can be customized using the default_node_pool
variable:
hclmodule "gke" {
source = "solutions.corewide.com/google-cloud/tf-gcp-k8s-gke/google"
version = "~> 4.1"
name_prefix = "foo"
vpc = google_compute_network.main.self_link
release_channel = "STABLE"
cluster_version = "1.29"
}
Creates GKE cluster with workload identity enabled, default node pool with the custom node_size
, and the second node pool with autoscaling and workload identity pool enabled:
hclmodule "gke" {
source = "solutions.corewide.com/google-cloud/tf-gcp-k8s-gke/google"
version = "~> 4.0"
name_prefix = "foo"
vpc = google_compute_network.main.self_link
release_channel = "STABLE"
cluster_version = "1.29"
workload_identity_enabled = true
create_workload_identity_pool = true
node_pools = [
{
name = "application"
min_size = 2
max_size = 5
preemptible = true
image = "cos_containerd"
tags = ["application"]
},
]
}
GKE with IP aliasing enabled, restricted connection to cluster API, disabled deletion protection, and observability configured:
hclmodule "gke" {
source = "solutions.corewide.com/google-cloud/tf-gcp-k8s-gke/google"
version = "~> 4.1"
name_prefix = "foo"
vpc = google_compute_network.main.self_link
release_channel = "STABLE"
gateway_api_config_channel = "CHANNEL_STANDARD"
cluster_version = "1.29"
deletion_protection_enabled = false
allowed_mgmt_networks = {
office = "104.22.0.0/24"
}
vpc_native = true
default_node_pool = {
node_size = "e2-standard-4"
}
node_pools = [
{
name = "application"
min_size = 2
max_size = 5
preemptible = true
tags = [
"application",
]
},
]
observability = {
enabled = true
managed_prometheus_enabled = true
components = [
"SYSTEM_COMPONENTS",
"APISERVER",
"SCHEDULER",
"CONTROLLER_MANAGER",
"STORAGE",
"HPA",
"DEPLOYMENT",
]
}
}
Variable | Description | Type | Default | Required | Sensitive |
---|---|---|---|---|---|
name_prefix |
Name prefix for Google service account and GKE cluster | string |
yes | no | |
vpc |
VPC network self_link which will be attached to the Kubernetes cluster | string |
yes | no | |
allowed_mgmt_networks |
Map of CIDR blocks allowed to connect to cluster API | map(string) |
no | no | |
cluster_master_cidr |
CIDR block to be used for control plane components | string |
172.16.0.0/28 |
no | no |
cluster_private_nodes_enabled |
Indicates whether cluster private nodes should be enabled | bool |
true |
no | no |
cluster_version |
Kubernetes version (Major.Minor ) |
string |
1.29 |
no | no |
create_workload_identity_pool |
Indicates whether to create a GKE workload identity pool or use the existing one (one pool per project) | bool |
true |
no | no |
default_node_pool |
Configuration of the maintenance node pool, that is created unconditionally |
object |
{} |
no | no |
default_node_pool.disk_size |
Disk size of a node | number |
20 |
no | no |
default_node_pool.max_size |
Maximum number of nodes in the pool | number |
no | no | |
default_node_pool.min_size |
Minimum number of nodes in the pool | number |
1 |
no | no |
default_node_pool.node_size |
Instance type to use for node creation | string |
e2-standard-2 |
no | no |
deletion_protection_enabled |
Prevent cluster deletion by Terraform | bool |
true |
no | no |
gateway_api_config_channel |
Configuration options for the Gateway API config feature | string |
CHANNEL_DISABLED |
no | no |
node_pools |
List of node pools to create | list(object) |
[] |
no | no |
node_pools[*].disk_size |
Disk size of a node | number |
20 |
no | no |
node_pools[*].image |
Image type of node pools | string |
COS_CONTAINERD |
no | no |
node_pools[*].max_size |
Maximum number of nodes in the pool | number |
no | no | |
node_pools[*].min_size |
Minimum number of nodes in the pool | number |
1 |
no | no |
node_pools[*].name |
Name of the node pool | string |
yes | no | |
node_pools[*].node_size |
Instance type to use for node creation | string |
e2-standard-2 |
no | no |
node_pools[*].preemptible |
Whether the nodes should be preemptible | bool |
false |
no | no |
node_pools[*].tags |
The list of instance tags to identify valid sources or targets for network firewalls (When is not set, the default rule set is applied) | list(string) |
[] |
no | no |
observability |
Cluster observability configuration | object |
{} |
no | no |
observability.components |
List of Kubernetes components exposing metrics to monitor | list(string) |
['SYSTEM_COMPONENTS', 'APISERVER', 'SCHEDULER', 'CONTROLLER_MANAGER', 'STORAGE', 'HPA', 'POD', 'DAEMONSET', 'DEPLOYMENT', 'STATEFULSET', 'KUBELET', 'CADVISOR', 'DCGM'] |
no | no |
observability.enabled |
Indicates whether cluster observability is enabled | bool |
false |
no | no |
observability.managed_prometheus_enabled |
Indicates whether Google Cloud Managed Service for Prometheus (GMP) should be deployed | bool |
true |
no | no |
region |
Specific zone which exists within the region or a single region | string |
no | no | |
release_channel |
Configuration options for the Release channel feature | string |
UNSPECIFIED |
no | no |
subnet_id |
The name or self_link of the Google Compute Engine subnetwork in which the cluster's instances are launched | string |
no | no | |
vpc_native |
Indicates whether IP alliasing should be enabled | bool |
true |
no | no |
workload_identity_enabled |
Indicates whether workload identity is enabled and whether nodes should store their metadata on the GKE metadata server | bool |
false |
no | no |
Output | Description | Type | Sensitive |
---|---|---|---|
cluster |
GKE cluster resource | resource |
no |
node_pools |
List of created node pools | resource |
no |
workload_identity_pool |
GKE workload identity pool data | computed |
no |
Dependency | Version | Kind |
---|---|---|
terraform |
>= 1.3 |
CLI |
hashicorp/google |
~> 6.2 |
provider |
hashicorp/google-beta |
~> 6.2 |
provider |
PodMonitoring
CRD documentation