OpenObserve

Why it mattersCopy Anchor Link

Teams need to search logs and traces without shipping data to a third party service that a federal customer has not approved. This module runs OpenObserve inside the VPC, stores telemetry in an S3 bucket the account owns, and ships with FIPS TLS, vendor telemetry turned off, and 90 day log retention.

Use it whenCopy Anchor Link

• A federal or air gapped deployment needs log search that stays inside the account.
• You want to route app logs through Fluent Bit with ecs-service's main_container_log_configuration.

Reach for something else whenCopy Anchor Link

• CloudWatch Logs from ecs-service is enough. It needs no extra infrastructure. Use ecs-service.
• You want browser side performance and error data. Use rum.

What it createsCopy Anchor Link

• aws_ecs_task_definition and aws_ecs_service (OpenObserve plus a NATS coordinator)
• aws_lb, aws_lb_target_group, and HTTPS and HTTP redirect listeners (internal by default)
• aws_security_group (load balancer and task) with rules
• aws_s3_bucket for telemetry (versioning, encryption, public access block, lifecycle)
• aws_iam_role (task and execution, partition aware policy ARN)
• aws_cloudwatch_log_group (encrypted with the required kms_key_arn)
• aws_route53_record (A record in a private hosted zone)

Secure by defaultCopy Anchor Link

• The load balancer is internal (alb_internal = true) and uses ELBSecurityPolicy-TLS13-1-2-FIPS-2023-04.
• The log group requires a customer managed key (kms_key_arn has no default).
• OpenObserve telemetry to the vendor is off (telemetry_enabled = false).
• The image is pinned to a release tag, never latest.
• Logs and telemetry are kept for 90 days, and the telemetry bucket refuses force deletion.

Commercial and GovCloudCopy Anchor Link

module "openobserve" {
  source = "git::ssh://git@github.com/the-kaizen-labs/terraform-modules.git//openobserve?ref=v1.7.0"

  name_prefix        = local.app_name
  vpc_id             = module.networking.vpc_id
  private_subnet_ids = module.networking.private_subnet_ids
  alb_subnet_ids     = module.networking.private_subnet_ids
  ecs_cluster_id     = module.ecs_cluster.cluster_id
  kms_key_arn        = aws_kms_key.this.arn
  certificate_arn    = local.certificate_arn

  private_hosted_zone_id = local.private_zone_id
  parent_domain          = "internal.example.com"

  metadata_postgres_dsn_ssm_arn = aws_ssm_parameter.o2_meta_db_dsn.arn
  admin_email                   = "devops@example.com"
  admin_password_ssm_arn        = local.o2_admin_password_arn

  caller_ingress_security_group_ids = [module.ecs_service.security_group_id]

  tags = local.tags
}

How it connectsCopy Anchor Link

• From networking: vpc_id, private_subnet_ids
• From ecs-cluster: cluster_id (as ecs_cluster_id)
• From rds-postgres: db_host, in the SSM DSN passed as metadata_postgres_dsn_ssm_arn
• From ecs-service: security_group_id (as caller_ingress_security_group_ids)
• To ecs-service: ingest_host, for Fluent Bit log routing
• To rds-postgres: task_security_group_id (as allowed_security_group_ids)

InputsCopy Anchor Link

OutputsCopy Anchor Link

Used byCopy Anchor Link

No Kaizen app uses this module yet.