Kubernetes ESO

Helm-based setup of External Secrets Operator and Reloader with Terraform. External Secrets Operator is a Kubernetes operator that integrates external secret management systems. The operator reads information from external APIs and automatically injects the values into a Kubernetes Secret. Reloader is a tool that can track the changes in ConfigMap and Secret and apply rolling upgrades on Pods with their associated DeploymentConfigs, Deployments, Daemonsets Statefulsets and Rollouts.

For cloud-specific implementations, the module does not support creation of a ClusterSecretStore Custom Resource Definition. This module is supposed to be used as a basis for the cloud-specific implementations that cover integration with cloud secrets storages.

Once you have a Corewide Solutions Portal account, this one-time action will use your browser session to retrieve credentials:

 shellterraform login solutions.corewide.com

Provision instructions

Initialize mandatory providers:

Copy and paste into your Terraform configuration and insert the variables:

 hclmodule "tf_k8s_eso" {
  source  = "solutions.corewide.com/kubernetes/tf-k8s-eso/helm"
  version = "~> 1.0.1"

  # specify module inputs here or try one of the examples below
  ...
}

Initialize the setup:

 shellterraform init

Define update strategy

Corewide DevOps team strictly follows Semantic Versioning Specification to provide our clients with products that have predictable upgrades between versions. We recommend pinning patch versions of our modules using pessimistic constraint operator (~>) to prevent breaking changes during upgrades.

To get new features during the upgrades (without breaking compatibility), use ~> 1.0 and run terraform init -upgrade

For the safest setup, use strict pinning with version = "1.0.1"

Module 55% off	~~€410~~ €190
TOTAL	€190

BUY

2,996

Corewide Proprietary License

v1.0.1 released 1 year, 9 months ago

New version approx. every 19 weeks

Upgrade Notes

Changelog

From `v1.x` to `v2.x`

The module from v2.0 utilizes a fundamentally new concept of inputs management and processing:

node_selector
eso.serviceaccount.annotations
eso.custom_values
reloader.namespace_selector
reloader.resource_selector

Update the declaration of the module according to the requirements and examples to match the designed configuration. Make sure that the mentioned inputs are updated with the new format:

update the format of reloader.namespace_selector and reloader.resource_selector inputs' type from list to map:

 hclmodule "eso" {
  ...
  # from v1.x format
  reloader = {
    namespace_selector = ["kubernetes.io/metadata.name=application"]
    resource_selector  = ["app-secret=yes"]
  }

  # to v2.0 format
  reloader = {
    namespace_selector = {
      "kubernetes.io/metadata.name" = "application"
    }

    resource_selector = {
      app-secret = "yes"
    }
  }
  ...
}

node_selector and eso.serviceaccount.annotations inputs don't require anymore escaping of the dot (.) symbol:

 hclmodule "eso" {
  ...
  # from v1.x format
  node_selector = {
    "eks\\.amazonaws\\.com/nodegroup" = "maintenance"
  }

  eso = {
    serviceaccount = {
      annotations = {
        "eks\\.amazonaws\\.com/role-arn" = data.aws_iam_role.allow_k8s_manage_secrets.arn
      }
    }
  }

  # to v2.0 format
  node_selector = {
    "eks.amazonaws.com/nodegroup" = "maintenance"
  }

  eso = {
    serviceaccount = {
      annotations = {
        "eks.amazonaws.com/role-arn" = data.aws_iam_role.allow_k8s_manage_secrets.arn
      }
    }
  }
  ...
}

eso.custom_values must be supplied as a single map with all subparameters instead of a map of Helm set array entities

 hclmodule "eso" {
  ...
  # from v1.x format
  eso = {
    custom_values = {
      "serviceAccount.extraLabels.managed-by"  = "terraform"
      "serviceAccount.extraLabels.environment" = "development"
    }
  }

  # to v2.0 format
  ...
  eso = {
    custom_values = {
      serviceAccount = {
        extraLabels = {
          managed-by  = "terraform"
          environment = "development"
        }
      }
    }
  }
}

From `v2.x` to `v3.x`

Module from v3.0 has changed hashicorp/kubernetes and hashicorp/helm Terraform providers versions, which aren't compatible with an old version. After the module version is upgraded, re-init the module to upgrade the provider versions. Additionally module has introduced Reloader and ESO version updates. Minimum Reloader Helm chart version has been set to 2.1.1.

Upgrade the following Terraform provider versions on the project level:
* hashicorp/kubernetes to ~> 3.0
* hashicorp/helm to ~> 3.1

 hclterraform {
  required_providers {
    helm = {
      source  = "hashicorp/helm"
      version = "~> 3.1"
    }

    kubernetes = {
      source  = "hashicorp/kubernetes"
      version = "~> 3.0"
    }
  }
}

If using non-default version of the Reloader Helm chart, update the Helm chart to version 2.2.7 or later:

 hclmodule "eso" {
  ...
  reloader = {
    enabled       = true
    chart_version = "2.2.7"
    ...
  }
  ...
}

NOTE Before the upgrade make sure to review the official upgrade guide for each version and perform required actions for your configuration. The upgrade guide is accessible separatly for each version on the releases page.

Update the ESO to the 0.16.2 version:

 hclmodule "eso" {
  ...
  eso = {
    enabled       = true
    chart_version = "0.16.2"
    app_version   = "v0.16.2"
    ...
  }
  ...
}

NOTE ESO version 0.16.0 stops serving v1alpha1 Kubernetes Custom Resources APIs. Prior to performing the upgrade to the 0.16.0 version make sure you are not using any v1alpha1 ESO Kubernetes resources across all of your infrastructure.

NOTE If using AWS Secrets Manager with ESO make sure to add secretsmanager:BatchGetSecretValue permission to the ESO IAM role prior to upgrading to the 0.16.0 version.

Upgrade project dependencies:

 bashterraform init --upgrade

Apply the changes to make sure the state of the resources is up-to-date with the new provider version and ESO is upgraded to the 0.16.2 version:

 bashterraform apply

ESO from version 0.17.0 stops serving v1beta1 Kubernetes Custom Resources APIs. Update the ESO Kubernetes Custom Resources API version from v1beta1 to v1 prior to updating from version 0.16.2 to 0.17.0.

Update the ESO to the 2.0.0 version:

 hclmodule "eso" {
  ...
  eso = {
    enabled       = true
    chart_version = "2.0.0"
    app_version   = "v2.0.0"
    ...
  }
  ...
}

Apply the changes:

 bashterraform apply

Deploy External Secrets Operator and Reloader with default parameters:

 hclmodule "eso" {
  source  = "solutions.corewide.com/kubernetes/tf-k8s-eso/helm"
  version = "~> 1.0"
}

Deploy External Secrets Operator only (without Reloader service):

 hclmodule "eso" {
  source  = "solutions.corewide.com/kubernetes/tf-k8s-eso/helm"
  version = "~> 1.0"

  reloader = {
    enabled = false
  }
}

Deploy External Secrets Operator with a custom service account and Reloader selectors:

 hclmodule "eso" {
  source  = "solutions.corewide.com/kubernetes/tf-k8s-eso/helm"
  version = "~> 1.0"

  eso = {
    serviceaccount = {
      create = true
      name   = "secrets-operator"

      annotations = {
        "eks\\.amazonaws\\.com/role-arn" = data.aws_iam_role.allow_k8s_manage_secrets.arn
      }
    }
  }

  reloader = {
    namespace_selector = ["foo=bar"]
    resource_selector  = ["reload=yes"]
  }
}

Deploy External Secrets Operator and Reloader with full customization:

 hclmodule "eso" {
  source  = "solutions.corewide.com/kubernetes/tf-k8s-eso/helm"
  version = "~> 1.0"

  name_prefix      = "stage"
  namespace        = "eso"
  create_namespace = true

  node_selector = {
    node = "maintenance"
  }

  eso = {
    enabled       = true
    chart_version = "yyyyy"
    app_version   = "xxxxx"
    install_crds  = true

    custom_values = {
      foo = "bar"
    }

    serviceaccount = {
      create = true
      name   = "secrets-operator"

      annotations = {
        "eks\\.amazonaws\\.com/role-arn" = data.aws_iam_role.allow_k8s_manage_secrets.arn
      }
    }
  }

  reloader = {
    enabled            = true
    chart_version      = "yyyyy"
    app_version        = "xxxxx"
    namespace_selector = ["foo=bar"]
    resource_selector  = ["reload=yes"]
  }
}

Variable	Description	Type	Default	Required	Sensitive
`name_prefix`	Naming prefix for all the resources created by the module	`string`		yes	no
`create_namespace`	Indicates whether to create a dedicated namespace for the stack deployment	`bool`	`true`	no	no
`eso`	External Secrets Operator parameters	`object`	`{}`	no	no
`eso.app_version`	External Secrets Operator version to deploy (image tag). If not set, the chart uses its corresponding default	`string`		no	no
`eso.chart_version`	External Secrets Operator Helm chart version to deploy	`string`	`0.10.7`	no	no
`eso.custom_values`	External Secrets Operator Helm chart custom values	`map(string)`	`{}`	no	no
`eso.enabled`	Indicates either to deploy the External Secrets Operator Helm chart into the Kubernetes cluster	`bool`	`true`	no	no
`eso.install_crds`	Indicates either to manage CRDs through Helm chart	`bool`	`true`	no	no
`eso.serviceaccount`	External Secrets Operator service account parameters	`object`		no	no
`eso.serviceaccount.annotations`	External Secrets Operator service account annotations	`map(string)`	`{}`	no	no
`eso.serviceaccount.create`	Indicates creation of External Secrets Operator service account	`bool`	`false`	no	no
`eso.serviceaccount.name`	External Secrets Operator service account name	`string`	`secrets-operator`	no	no
`namespace`	The namespace to install the External Secrets Operator and Reloader into	`string`	`external-secrets-operator`	no	no
`node_selector`	Node selector for the stack components	`map(string)`	`{}`	no	no
`reloader`	Reloader parameters	`object`	`{}`	no	no
`reloader.app_version`	Reloader version to deploy (image tag)	`string`	`v1.0.64`	no	no
`reloader.chart_version`	Reloader Helm chart version to deploy	`string`	`1.0.65`	no	no
`reloader.enabled`	Indicates either to deploy the Reloader Helm chart into the Kubernetes cluster	`bool`	`true`	no	no
`reloader.namespace_selector`	Comma separated list of k8s label selectors for Namespace selection	`list(string)`	`[]`	no	no
`reloader.resource_selector`	Comma separated list of k8s label selectors for ConfigMap/Secret selection	`list(string)`	`[]`	no	no