All products Terraform Google Cloud google
tf-gcp-kafka
Diagram of product resources

This module manages KMS-encrypted Kafka cluster and its topics in GCP Managed Service for Apache Kafka.

NOTE: This module is meant to be used with an already created VPC.

Log in to Corewide IaC registry

Once you have a Corewide Solutions Portal account, this one-time action will use your browser session to retrieve credentials:

 shellterraform login solutions.corewide.com
Provision instructions

Initialize mandatory providers:

Copy and paste into your Terraform configuration and insert the variables:

 hclmodule "tf_gcp_kafka" {
  source  = "solutions.corewide.com/google-cloud/tf-gcp-kafka/google"
  version = "~> 1.0.0"

  # specify module inputs here or try one of the examples below
  ...
}

Initialize the setup:

 shellterraform init
Define update strategy

Corewide DevOps team strictly follows Semantic Versioning Specification to provide our clients with products that have predictable upgrades between versions. We recommend pinning patch versions of our modules using pessimistic constraint operator (~>) to prevent breaking changes during upgrades.

To get new features during the upgrades (without breaking compatibility), use ~> 1.0 and run terraform init -upgrade

For the safest setup, use strict pinning with version = "1.0.0"

Managed Kafka cluster in GCP

Creates a Kafka cluster in GCP Managed Service for Apache Kafka
$600
Version:
1.0.0
40
40
v1.0.0
BUY
40
License
v1.0.0 released 3 months ago
Changelog
Diagram of product resources

This module manages KMS-encrypted Kafka cluster and its topics in GCP Managed Service for Apache Kafka.

NOTE: This module is meant to be used with an already created VPC.

Deploy Kafka cluster with 4 CPU threads, 4 GiB memory, single topic with custom configuration, and custom labels:

 hclmodule "vpc" {
  source  = "solutions.corewide.com/google-cloud/tf-gcp-vpc/google"
  version = "~> 1.0"

  name_prefix = "foo"

  subnets = {
    data = {
      cidr_block = "10.0.0.0/8"
      region     = "us-west1"
    }
  }
}

module "kafka" {
  source  = "solutions.corewide.com/google-cloud/tf-gcp-kafka/google"
  version = "~> 1.0"

  name_prefix     = "bar"
  region          = "us-west1"
  subnet_id       = module.vpc.subnets["data"].id
  vcpu_count      = 4
  memory_size_gib = 4
  rebalance_mode  = "NO_REBALANCE"

  topics = [
    {
      name               = "biz"
      partition_count    = 3
      replication_factor = 3

      config = {
        "cleanup.policy" = "compact"
      }
    },
  ]

  labels = {
    env = "dev"
  }
}

Minimal setup with mandatory parameters only:

 hclmodule "vpc" {
  source  = "solutions.corewide.com/google-cloud/tf-gcp-vpc/google"
  version = "~> 1.0"

  name_prefix = "foo"

  subnets = {
    data = {
      cidr_block = "10.0.0.0/8"
      region     = "us-west1"
    }
  }
}

module "kafka" {
  source  = "solutions.corewide.com/google-cloud/tf-gcp-kafka/google"
  version = "~> 1.0"

  name_prefix = "bar"
  region      = "us-west1"
  subnet_id   = module.vpc.subnets["data"].id
}
Variable Description Type Default Required Sensitive
name_prefix Name for Kafka cluster and name prefix for related resources string yes no
region GCP region to create resources in string yes no
subnet_id The name or self_link of the Google Compute Engine network subnet to create the cluster endpoint in string yes no
labels Map of labels to be assigned to Kafka cluster map(string) {} no no
memory_size_gib The memory to provision for the cluster (GiB). The value must be between 1 GiB and 8 GiB per one vCPU number 3 no no
rebalance_mode The rebalance behavior for the cluster. Possible values are: NO_REBALANCE, AUTO_REBALANCE_ON_SCALE_UP string NO_REBALANCE no no
topics List of Kafka topics to create inside the cluster list(object) [] no no
topics[*].config Map containing Kafka topic configuration parameters map(any) {} no no
topics[*].name Name of Kafka topic string yes no
topics[*].partition_count The number of partitions for the topic number 3 no no
topics[*].replication_factor The number of nodes that topics are replicated across. 3 is recommended for high availability number 3 no no
vcpu_count Number of vCPU threads to provision for Kafka. Minimal value is 3 number 3 no no
Output Description Type Sensitive
bootstrap_host Kafka bootstrap host computed no
bootstrap_port Kafka bootstrap port computed no
cluster Kafka cluster resource attributes resource yes
topics Kafka Topic resources' attributes computed no
Dependency Version Kind
terraform >= 1.3 CLI
hashicorp/google ~> 6.2 provider
hashicorp/google-beta ~> 6.2 provider

References

Not sure where to start?
Let's find your perfect match.