BFF与AI Gateway架构：2026年统一LLM接入层设计

AI时代，为什么需要AI Gateway？

当你的系统同时接入OpenAI、Claude、Gemini、通义千问、DeepSeek……每个LLM有不同的API格式、计费方式、限流策略。没有统一接入层，你的业务代码会被LLM供应商彻底绑架。

真实案例：某公司从OpenAI迁移到Claude，因为API格式不同，改动了47个文件，耗时2周。有了AI Gateway，迁移只需改1行配置。

BFF模式的三次进化

传统BFF（2018）
  为不同前端聚合后端API，解決过度获取問題

AI-Enhanced BFF（2024）
  BFF层加入AI能力：摘要、翻译、内容生成
  AI只是BFF的一个下游服务

AI Gateway（2026）
  AI成为核心，BFF围绕AI重构
  统一接入多LLM，管理路由、计费、安全
  业务代码只对接AI Gateway，不直接调用LLM

---

AI Gateway核心能力

┌──────────────────────────────────────────────────────┐
│                    业务服务层                          │
│   OrderService │ UserService │ ContentService         │
├──────────────────────────────────────────────────────┤
│                    AI Gateway                         │
│  ┌──────────┬──────────┬──────────┬───────────────┐  │
│  │ 路由策略  │ 限流熔断 │ 缓存管理  │ 降级策略      │  │
│  ├──────────┼──────────┼──────────┼───────────────┤  │
│  │ Prompt   │ Token    │ 审计日志  │ 安全防护      │  │
│  │ 管理     │ 计费     │          │               │  │
│  ├──────────┴──────────┴──────────┴───────────────┤  │
│  │              流式响应代理（SSE/WebSocket）         │  │
│  ├─────────────────────────────────────────────────┤  │
│  │              多模型适配层                          │  │
│  └──┬─────────┬─────────┬─────────┬───────────────┘  │
├─────┼─────────┼─────────┼─────────┼──────────────────┤
│  OpenAI │ Claude  │ Gemini  │ 通义千问 │ DeepSeek     │
└──────────────────────────────────────────────────────┘

---

多模型路由策略

按成本/延迟/质量智慧选擇LLM

@Configuration
public class AiGatewayConfig {

    @Bean
    public ModelRouter modelRouter() {
        return ModelRouter.builder()
            .addStrategy(new CostOptimizedStrategy())
            .addStrategy(new LatencyOptimizedStrategy())
            .addStrategy(new QualityOptimizedStrategy())
            .addStrategy(new FallbackStrategy())
            .build();
    }
}

public class CostOptimizedStrategy implements RoutingStrategy {

    private static final Map<String, ModelPricing> PRICING = Map.of(
        "gpt-4o",           new ModelPricing(0.005, 0.015),
        "gpt-4o-mini",      new ModelPricing(0.00015, 0.0006),
        "claude-3.5-sonnet", new ModelPricing(0.003, 0.015),
        "deepseek-v3",      new ModelPricing(0.00027, 0.0011)
    );

    @Override
    public ModelSelection select(RoutingContext context) {
        String taskType = context.getTaskType();
        int estimatedTokens = context.getEstimatedTokens();

        return switch (taskType) {
            case "simple_qa"     -> selectModel("gpt-4o-mini", estimatedTokens);
            case "code_review"   -> selectModel("claude-3.5-sonnet", estimatedTokens);
            case "creative"      -> selectModel("gpt-4o", estimatedTokens);
            case "chinese_nlp"   -> selectModel("deepseek-v3", estimatedTokens);
            default              -> selectModel("gpt-4o", estimatedTokens);
        };
    }
}

---

Prompt模板管理与版本控制

@Service
public class PromptTemplateService {

    private final PromptTemplateRepository templateRepo;

    public PromptRenderResult render(String templateId, Map<String, String> variables) {
        PromptTemplate template = templateRepo.findLatestVersion(templateId);

        String renderedPrompt = template.getContent();
        for (Map.Entry<String, String> entry : variables.entrySet()) {
            renderedPrompt = renderedPrompt.replace("{{" + entry.getKey() + "}}", entry.getValue());
        }

        return PromptRenderResult.builder()
            .templateId(templateId)
            .version(template.getVersion())
            .renderedPrompt(renderedPrompt)
            .estimatedTokens(estimateTokens(renderedPrompt))
            .build();
    }
}

@Entity
@Table(name = "prompt_templates")
public class PromptTemplate {
    @Id
    @GeneratedValue(strategy = GenerationType.IDENTITY)
    private Long id;

    private String templateId;
    private Integer version;
    private String content;
    private String description;
    private Boolean isActive;

    @Column(name = "created_at")
    private LocalDateTime createdAt;
}

---

Token计费与用量追踪

@Service
public class TokenBillingService {

    private final UsageRepository usageRepo;

    public UsageRecord recordUsage(UsageRequest request) {
        BigDecimal cost = calculateCost(
            request.getModel(),
            request.getInputTokens(),
            request.getOutputTokens()
        );

        UsageRecord record = UsageRecord.builder()
            .tenantId(request.getTenantId())
            .model(request.getModel())
            .inputTokens(request.getInputTokens())
            .outputTokens(request.getOutputTokens())
            .cost(cost)
            .promptTemplateId(request.getPromptTemplateId())
            .latencyMs(request.getLatencyMs())
            .build();

        return usageRepo.save(record);
    }

    public BillingSummary getMonthlySummary(String tenantId, YearMonth month) {
        List<UsageRecord> records = usageRepo.findByTenantIdAndMonth(tenantId, month);

        return BillingSummary.builder()
            .tenantId(tenantId)
            .month(month)
            .totalTokens(records.stream().mapToLong(r -> r.getInputTokens() + r.getOutputTokens()).sum())
            .totalCost(records.stream().map(UsageRecord::getCost).reduce(BigDecimal.ZERO, BigDecimal::add))
            .byModel(records.stream().collect(Collectors.groupingBy(UsageRecord::getModel, Collectors.summingLong(r -> r.getInputTokens() + r.getOutputTokens()))))
            .avgLatencyMs(records.stream().mapToLong(UsageRecord::getLatencyMs).average().orElse(0))
            .build();
    }
}

---

流式响应代理：SSE透传

@RestController
@RequestMapping("/api/ai")
public class StreamingAiController {

    private final AiGatewayService gatewayService;

    @PostMapping(value = "/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<ServerSentEvent<String>> streamChat(@RequestBody ChatRequest request) {
        return gatewayService.streamChat(request)
            .map(chunk -> ServerSentEvent.<String>builder()
                .id(chunk.getId())
                .event("delta")
                .data(chunk.getContent())
                .build())
            .concatWith(Flux.just(
                ServerSentEvent.<String>builder()
                    .event("done")
                    .data("[DONE]")
                    .build()
            ));
    }
}

@Service
public class AiGatewayService {

    public Flux<StreamChunk> streamChat(ChatRequest request) {
        ModelSelection selection = modelRouter.select(RoutingContext.from(request));
        LlmProvider provider = providerFactory.getProvider(selection.getModel());

        return provider.stream(request.getPrompt(), selection.getModel())
            .doOnNext(chunk -> tokenBillingService.recordAsync(
                request.getTenantId(), selection.getModel(), chunk))
            .onErrorResume(e -> fallbackProvider.stream(request.getPrompt()));
    }
}

---

Spring Cloud Gateway + AI扩展实战

# application.yml
spring:
  cloud:
    gateway:
      routes:
        - id: openai-route
          uri: https://api.openai.com
          predicates:
            - Path=/api/ai/openai/**
          filters:
            - name: AiGateway
              args:
                provider: openai
                model: gpt-4o
                rateLimit: 100/s
                timeout: 30s

        - id: claude-route
          uri: https://api.anthropic.com
          predicates:
            - Path=/api/ai/claude/**
          filters:
            - name: AiGateway
              args:
                provider: anthropic
                model: claude-3.5-sonnet
                rateLimit: 50/s
                timeout: 60s

@Component
public class AiGatewayFilter implements GlobalFilter, Ordered {

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        String provider = exchange.getRequest().getHeaders().getFirst("X-AI-Provider");

        if (!rateLimiter.tryAcquire(provider)) {
            exchange.getResponse().setStatusCode(HttpStatus.TOO_MANY_REQUESTS);
            return exchange.getResponse().setComplete();
        }

        auditService.log(exchange.getRequest());

        return chain.filter(exchange)
            .doOnSuccess(v -> billingService.record(exchange))
            .onErrorResume(e -> fallbackService.handle(exchange, e));
    }

    @Override
    public int getOrder() {
        return -1;
    }
}

---

安全：Prompt注入防护与输出过滤

@Service
public class AiSecurityService {

    private static final List<Pattern> INJECTION_PATTERNS = List.of(
        Pattern.compile("(?i)ignore\\s+(all\\s+)?previous\\s+instructions"),
        Pattern.compile("(?i)system\\s*:\\s*you\\s+are"),
        Pattern.compile("(?i)forget\\s+everything"),
        Pattern.compile("(?i)pretend\\s+you\\s+are")
    );

    private static final List<Pattern> SENSITIVE_PATTERNS = List.of(
        Pattern.compile("\\b\\d{16}\\b"),
        Pattern.compile("\\b\\d{17}[\\dXx]\\b"),
        Pattern.compile("[\\w.-]+@[\\w.-]+\\.\\w+")
    );

    public SecurityCheckResult checkInput(String prompt) {
        for (Pattern pattern : INJECTION_PATTERNS) {
            if (pattern.matcher(prompt).find()) {
                return SecurityCheckResult.blocked("疑似Prompt注入攻击");
            }
        }
        return SecurityCheckResult.passed();
    }

    public String sanitizeOutput(String output) {
        String sanitized = output;
        for (Pattern pattern : SENSITIVE_PATTERNS) {
            sanitized = pattern.matcher(sanitized).replaceAll("[REDACTED]");
        }
        return sanitized;
    }
}

---

总结

1. AI Gateway是AI时代的基础设施 — 统一接入多LLM，业务代码零耦合
2. 多模型路由讓成本降低40% — 按任务類型智慧选擇最优模型
3. 安全是底線 — Prompt注入防护、输出过滤、敏感信息脱敏缺一不可
4. Spring Cloud Gateway + AI扩展是最佳实踐 — 网关层统一管控，业务层無感知

AI Gateway不是可选架构，而是AI时代的必修课。越早建设，越早摆脱LLM供应商锁定。