DevOps & Scaling

Terraform + CaptchaAI:验证码识别 worker 的基础设施即代码

worker 开几台、每台跑多少并发、API Key 从哪里来——写进 Terraform 之后,环境之间只差一个 tfvars:terraform apply 上线,terraform destroy 拆干净。下面这套配置跑在 AWS ECS Fargate 上,密钥放 Secrets Manager。先记住一条:worker_count × captchaai_concurrency 就是你的峰值并发,必须落在 CaptchaAI 套餐的线程数以内。

为什么验证码 worker 值得写成代码

识别 worker 无状态、可横向扩、按并发计费,正好落在 IaC 的舒适区。

环境差异收敛到 tfvars,调并发就是一次可评审的 PR。

目录结构

terraform/
├── main.tf              # Provider config
├── variables.tf         # Input variables
├── outputs.tf           # Output values
├── modules/
│   └── captcha-worker/
│       ├── main.tf      # ECS/EC2 resources
│       ├── variables.tf # Module inputs
│       └── outputs.tf   # Module outputs
├── environments/
│   ├── dev.tfvars
│   ├── staging.tfvars
│   └── production.tfvars

根目录只做参数编排,资源定义收在 modules/captcha-workerenvironments/ 下每个 tfvars 一套参数,state 分环境存。

核心配置

provider 与远程 state

# main.tf
terraform {
  required_version = ">= 1.5"

  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }

  backend "s3" {
    bucket         = "my-terraform-state"
    key            = "captcha-workers/terraform.tfstate"
    region         = "us-east-1"
    dynamodb_table = "terraform-locks"
    encrypt        = true
  }
}

provider "aws" {
  region = var.aws_region
}

state 放 S3、锁放 DynamoDB 是协作下限。

没有锁,两个人同时 apply 就会互相覆盖。

变量定义

# variables.tf
variable "aws_region" {
  description = "AWS region for deployment"
  type        = string
  default     = "us-east-1"
}

variable "environment" {
  description = "Environment name (dev, staging, production)"
  type        = string
}

variable "worker_count" {
  description = "Number of CAPTCHA solving workers"
  type        = number
  default     = 3
}

variable "worker_cpu" {
  description = "CPU units for each worker (1024 = 1 vCPU)"
  type        = number
  default     = 512
}

variable "worker_memory" {
  description = "Memory in MB for each worker"
  type        = number
  default     = 1024
}

variable "max_workers" {
  description = "Maximum workers for auto-scaling"
  type        = number
  default     = 10
}

variable "captchaai_concurrency" {
  description = "Concurrent CAPTCHA tasks per worker"
  type        = number
  default     = 10
}

影响账单的只有两个:worker_count 是常驻容器数,captchaai_concurrency 是单机同时处理的验证码数。

轮询型 worker 吃不满 CPU,worker_cpu 从 512 起步够用。

API Key 交给 Secrets Manager

# secrets.tf — Store API key in AWS Secrets Manager
resource "aws_secretsmanager_secret" "captchaai_api_key" {
  name        = "${var.environment}/captchaai-api-key"
  description = "CaptchaAI API key for CAPTCHA solving workers"
}

# Reference secret in ECS task (never in plain text)
data "aws_secretsmanager_secret_version" "captchaai_api_key" {
  secret_id = aws_secretsmanager_secret.captchaai_api_key.id
}

API Key 不进 tfvars、不进仓库,也不写成普通环境变量。Terraform 只建密钥的壳,值由 CI 写入,state 里不留明文。

ECS Fargate worker 集群

# ecs.tf — Fargate-based CAPTCHA workers
resource "aws_ecs_cluster" "captcha" {
  name = "captcha-workers-${var.environment}"

  setting {
    name  = "containerInsights"
    value = "enabled"
  }
}

resource "aws_ecs_task_definition" "captcha_worker" {
  family                   = "captcha-worker-${var.environment}"
  network_mode             = "awsvpc"
  requires_compatibilities = ["FARGATE"]
  cpu                      = var.worker_cpu
  memory                   = var.worker_memory
  execution_role_arn       = aws_iam_role.ecs_execution.arn
  task_role_arn            = aws_iam_role.ecs_task.arn

  container_definitions = jsonencode([
    {
      name  = "captcha-worker"
      image = "${aws_ecr_repository.captcha_worker.repository_url}:latest"

      environment = [
        { name = "CAPTCHAAI_CONCURRENCY", value = tostring(var.captchaai_concurrency) },
        { name = "CAPTCHAAI_POLL_INTERVAL", value = "5" },
        { name = "ENVIRONMENT", value = var.environment },
      ]

      secrets = [
        {
          name      = "CAPTCHAAI_API_KEY"
          valueFrom = aws_secretsmanager_secret.captchaai_api_key.arn
        }
      ]

      logConfiguration = {
        logDriver = "awslogs"
        options = {
          "awslogs-group"         = aws_cloudwatch_log_group.captcha.name
          "awslogs-region"        = var.aws_region
          "awslogs-stream-prefix" = "worker"
        }
      }
    }
  ])
}

resource "aws_ecs_service" "captcha_worker" {
  name            = "captcha-workers"
  cluster         = aws_ecs_cluster.captcha.id
  task_definition = aws_ecs_task_definition.captcha_worker.arn
  desired_count   = var.worker_count
  launch_type     = "FARGATE"

  network_configuration {
    subnets         = var.private_subnets
    security_groups = [aws_security_group.captcha_worker.id]
  }
}

非敏感项走 environment,API Key 走 secrets 引用 ARN。日志统一进 CloudWatch,排错按 worker 前缀过滤。

弹性伸缩

# autoscaling.tf
resource "aws_appautoscaling_target" "captcha" {
  max_capacity       = var.max_workers
  min_capacity       = var.worker_count
  resource_id        = "service/${aws_ecs_cluster.captcha.name}/${aws_ecs_service.captcha_worker.name}"
  scalable_dimension = "ecs:service:DesiredCount"
  service_namespace  = "ecs"
}

# Scale up when queue is deep
resource "aws_appautoscaling_policy" "scale_up" {
  name               = "captcha-scale-up"
  policy_type        = "StepScaling"
  resource_id        = aws_appautoscaling_target.captcha.resource_id
  scalable_dimension = aws_appautoscaling_target.captcha.scalable_dimension
  service_namespace  = aws_appautoscaling_target.captcha.service_namespace

  step_scaling_policy_configuration {
    adjustment_type         = "ChangeInCapacity"
    cooldown                = 120

    step_adjustment {
      scaling_adjustment          = 2
      metric_interval_lower_bound = 0
    }
  }
}

# Scale down when idle
resource "aws_appautoscaling_policy" "scale_down" {
  name               = "captcha-scale-down"
  policy_type        = "StepScaling"
  resource_id        = aws_appautoscaling_target.captcha.resource_id
  scalable_dimension = aws_appautoscaling_target.captcha.scalable_dimension
  service_namespace  = aws_appautoscaling_target.captcha.service_namespace

  step_scaling_policy_configuration {
    adjustment_type         = "ChangeInCapacity"
    cooldown                = 300

    step_adjustment {
      scaling_adjustment          = -1
      metric_interval_upper_bound = 0
    }
  }
}

扩容加 2 个、冷却 120 秒;缩容减 1 个、冷却 300 秒——扩得快、缩得慢。识别任务要几秒到几十秒,缩太急会把在跑的任务一起干掉。

每个环境一份 tfvars

# environments/dev.tfvars
environment           = "dev"
worker_count          = 1
max_workers           = 3
worker_cpu            = 256
worker_memory         = 512
captchaai_concurrency = 3
# environments/production.tfvars
environment           = "production"
worker_count          = 5
max_workers           = 20
worker_cpu            = 1024
worker_memory         = 2048
captchaai_concurrency = 20

dev 求跑通,production 求扛峰值,共用模块,改的只是数字。

并发数怎么和套餐线程数对齐

CaptchaAI 按线程计费:一个线程等于一个正在处理中的验证码,完成即释放,线程内识别次数不限。要算的不是“一个月多少次”,而是“同一时刻几个在跑”。

环境 worker 数 单机并发 峰值并发 对应套餐
dev 1 3 3 BASIC($15/月,5 线程)
production 常驻 5 20 100 PREMIUM($170/月,100 线程)
扩容到 10 台 10 20 200 ENTERPRISE($300/月,200 线程)

按默认值,production 常驻 100 并发正好吃满 PREMIUM;而 max_workers = 20 意味着峰值能冲到 400,超出的部分会堆在自己队列里排队。要么压 max_workers,要么升套餐。

worker 容器里跑什么

"""captcha_worker.py — The container runs this."""
import os
import time
import signal
import requests

API_KEY = os.environ["CAPTCHAAI_API_KEY"]
CONCURRENCY = int(os.environ.get("CAPTCHAAI_CONCURRENCY", "10"))
POLL_INTERVAL = int(os.environ.get("CAPTCHAAI_POLL_INTERVAL", "5"))

running = True

def shutdown_handler(signum, frame):
    global running
    print("Graceful shutdown initiated")
    running = False

signal.signal(signal.SIGTERM, shutdown_handler)
signal.signal(signal.SIGINT, shutdown_handler)

session = requests.Session()

def solve_captcha(sitekey, pageurl):
    resp = session.post("https://ocr.captchaai.com/in.php", data={
        "key": API_KEY,
        "method": "userrecaptcha",
        "googlekey": sitekey,
        "pageurl": pageurl,
        "json": 1
    })
    data = resp.json()
    if data.get("status") != 1:
        return {"error": data.get("request")}

    captcha_id = data["request"]
    for _ in range(60):
        time.sleep(POLL_INTERVAL)
        result = session.get("https://ocr.captchaai.com/res.php", params={
            "key": API_KEY, "action": "get", "id": captcha_id, "json": 1
        }).json()
        if result.get("status") == 1:
            return {"solution": result["request"]}
        if result.get("request") != "CAPCHA_NOT_READY":
            return {"error": result.get("request")}

    return {"error": "TIMEOUT"}

# Main loop — pull tasks from SQS or Redis
print(f"Worker started: concurrency={CONCURRENCY}")
while running:
    # Pull tasks from your queue here
    time.sleep(1)

print("Worker shutdown complete")

提交到 in.php 拿到 ID 后,每 5 秒查一次 res.php,最多 60 次。换识别类型只改 method,Terraform 一行不动。

缩容时 Fargate 先发 SIGTERM,worker 收到后不再领新任务、把手上的做完再退出。国内 CI 构建镜像时,给 pip install-i 指向清华 TUNA 镜像源会快很多。

Terraform 上线与回滚

# Initialize
terraform init

# Plan for production
terraform plan -var-file=environments/production.tfvars

# Apply
terraform apply -var-file=environments/production.tfvars

# Destroy (dev cleanup)
terraform destroy -var-file=environments/dev.tfvars

流程是 plan → 评审 → apply。destroy 只对 dev 用,production 的删除走单独审批。

部署排错

现象 原因 处理方式
提示密钥不存在 只建了壳,没写值 先写值再 apply
worker 启动即退出 环境变量缺失或镜像标签不对 查 CloudWatch 日志,核对 ECR tag
state 被锁住 上次 apply 断了 terraform force-unlock <lock-id>

换云厂商与选区域

分层方式不依赖 AWS:registry 上有 alicloudtencentcloud provider,把 Fargate 换成阿里云 ACK / ECI 或腾讯云 TKE,“变量、密钥、伸缩”三层照搬即可。

区域按目标站点选。国际站点的 reCAPTCHA、Turnstile 任务放 ap-southeast-1(新加坡)或 ap-northeast-1(东京),顺带避开 Google 脚本在国内加载不稳的调试麻烦。

国内站点以极验(GeeTest)为主,CaptchaAI 支持 GeeTest v3,易盾、防水墙不在覆盖范围内。

常见问题

captchaai_concurrency 设成多少合适?

套餐线程数 ÷ 峰值 worker 数 倒推,再留 10%~20% 余量给重试。512 MB 的任务跑 20 并发是稳妥起点。

API Key 可以直接写进 tfvars 吗?

不要。tfvars 常被提交进仓库,apply 之后这个值还会以明文落在 state 里。让 Terraform 只创建密钥对象,值由 CI 写入。

缩容的时候,正在识别的任务会丢吗?

只要 worker 处理了 SIGTERM 就不会:收到信号后不再领新任务,手上的做完再退出。队列再配上可见性超时即可兜底。

Fargate 和 EC2 怎么选?

求省心选 Fargate,节点不用自己维护;常驻负载稳定、量大的部分可以迁到 EC2 用预留实例压单价。

下一步

把识别基础设施变成可评审、可回滚的代码:获取 CaptchaAI API Key,写进 Secrets Manager,再 apply。

相关阅读:

该文章已禁用评论。