spring gateway 动态uri

需求背景:

业务背景通过k8s的容器是为用户提供执行环境, 并把容器里的端口通过ingress暴露出来
原始实现是动态创建ingress和service路由到用户容器
出现的问题是用户量巨大时, 由于ingress规则太多, nginx-ingress-controller会占用巨大内存(超过20G), 导致ingress实例相继重启, 服务短暂不可用.
所以修改了实现方式, 只配置一个泛域名ingress, 指到gateway, 然后gateway动态路由到service

实现方案:

实现方式上面已经说了, 这里说一些实现细节和遇到的问题
第一是自定义了一个filter, 根据域名转发到service.

问题1:修改uri不生效

最开始通过lambda写的filter, 修改目标uri

exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR, newUri);

发现不生效, 缘由是lambda里自定义的filter顺序在NettyRoutingFilter后面, 实际请求已经完成了, 这时候修改uri就没有意义了. 解决方法就是把lambda抽成GatewayFilter的实现类. 并且实现order方法:

    @Override
    public int getOrder() {
        // Before NettyRoutingFilter so the URL is
        // changed just before the actual request is made.
        return Ordered.LOWEST_PRECEDENCE - 1;
    }

问题2:修改ws请求转发成了http

ws转发无响应, 浏览器里看不出来问题缘由, python手搓ws请求发现

>>> # 运行 WebSocket 连接
>>> asyncio.run(connect())
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
    return loop.run_until_complete(main)
  File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
    return future.result()
  File "<stdin>", line 2, in connect
  File "/usr/local/lib/python3.10/dist-packages/websockets/asyncio/client.py", line 587, in __aenter__
    return await self
  File "/usr/local/lib/python3.10/dist-packages/websockets/asyncio/client.py", line 543, in __await_impl__
    await self.connection.handshake(
  File "/usr/local/lib/python3.10/dist-packages/websockets/asyncio/client.py", line 114, in handshake
    raise self.protocol.handshake_exc
  File "/usr/local/lib/python3.10/dist-packages/websockets/client.py", line 325, in parse
    self.process_response(response)
  File "/usr/local/lib/python3.10/dist-packages/websockets/client.py", line 142, in process_response
    raise InvalidStatus(response)
websockets.exceptions.InvalidStatus: server rejected WebSocket connection: HTTP 200

HTTP 200, 这是把ws请求当成http了, 这个问题卡了我大半天, chatgpt总往歪路上引, 走了许多弯路, 查看源码后发现还是order的问题, WebsocketRoutingFilter在NettyRoutingFilter前面, 所以我们的filter还要往前移.

    @Override
    public int getOrder() {
        // Before NettyRoutingFilter so the URL is
        // changed just before the actual request is made.

        // Before WebsocketRoutingFilter so ws request can be routed right
        return Ordered.LOWEST_PRECEDENCE - 2;
    }

问题3 ws能转发了, 但没完全转发

ws请求能转发了, 正常返回101, 但还是不太正常, ws服务端会报 Error: write ECONNRESET
打开debug日志

logging:
  level:
    org:
      springframework:
        cloud:
          gateway: DEBUG
    reactor:
      netty:
        http: DEBUG
    io:
      netty: DEBUG

输出了大堆日志, 仍给ai, 嗯.. 这玩应还是有点用的, 找出了关键错误信息:

Max frame length of 65536 has been exceeded

从ai和网上的诸多解决方案里试来试去, 最后解决问题的关键配置是:

spring:
  cloud:
    gateway:
      httpclient:
        websocket:
          max-frame-payload-length: 10485760

问题4, 目标无响应时返回500错误

我想要的是404
这个在传递filter链上加上onErrorResume处理就好

// 继续传递请求
        return chain.filter(exchange).onErrorResume(ex -> {
            // 检测异常是否与服务无响应相关
            if (isConnectionError(ex)) {
                return fetchFriendly404Content(exchange);
            }
            // 如果不是服务无响应异常，继续以默认处理
            return Mono.error(ex);
        }).then(Mono.fromRunnable(() ->
                log.debug("Request Complete: {}", originalUri)
        ));

最后,附上主要文件代码

GoServiceFilter

import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.gateway.filter.GatewayFilter;
import org.springframework.cloud.gateway.filter.GatewayFilterChain;
import org.springframework.cloud.gateway.support.ServerWebExchangeUtils;
import org.springframework.core.Ordered;
import org.springframework.core.io.buffer.DataBuffer;
import org.springframework.http.HttpStatus;
import org.springframework.http.server.reactive.ServerHttpRequest;
import org.springframework.stereotype.Component;
import org.springframework.web.reactive.function.client.WebClient;
import org.springframework.web.server.ServerWebExchange;
import reactor.core.publisher.Mono;

import java.net.URI;
import java.util.Objects;

@Slf4j
@Component
public class GoServiceFilter implements GatewayFilter, Ordered {
    @Value("${biz.pty.ns-domain}")
    private String nsDomain;
    @Value("${biz.page404}")
    private String page404;

    private final WebClient webClient;

    public GoServiceFilter(WebClient.Builder webClientBuilder) {
        // 初始化 WebClient，用于代理请求
        this.webClient = webClientBuilder.build();
    }

    @Override
    public Mono<Void> filter(ServerWebExchange exchange, GatewayFilterChain chain) {
        ServerHttpRequest request = exchange.getRequest();
        String host = Objects.requireNonNull(request.getHeaders().getHost()).getHostName();
        String originalUri = request.getURI().toString();
        // 检查合法的 Host (以避免不完整的域名或异常请求)
        String[] hostParts = host.split(".");
        if (hostParts.length < 3) {
            log.warn("Invalid Host: {} -> 404", host);
            exchange.getResponse().setStatusCode(HttpStatus.NOT_FOUND);
            return exchange.getResponse().setComplete();
        }

        // 提取子域名部分
        String subdomain = hostParts[0];

        // 动态构建目标 URI
        String dynamicUri = "http://service-" + subdomain + nsDomain;

        // 完整拼接路径与查询参数
        URI newUri = URI.create(
                dynamicUri + request.getURI().getRawPath() + // 原始路径保留
                        (request.getURI().getQuery() != null ? "?" + request.getURI().getQuery() : "") // 原始查询参数保留
        );

        log.debug("HTTP Routing - Original: {}, Target: {}", originalUri, newUri);
        exchange.getAttributes().put(ServerWebExchangeUtils.GATEWAY_REQUEST_URL_ATTR, newUri);

        // 继续传递请求
        return chain.filter(exchange).onErrorResume(ex -> {
            // 检测异常是否与服务无响应相关
            if (isConnectionError(ex)) {
                return fetchFriendly404Content(exchange);
            }
            // 如果不是服务无响应异常，继续以默认处理
            return Mono.error(ex);
        }).then(Mono.fromRunnable(() ->
                log.debug("Request Complete: {}", originalUri)
        ));
    }
    private boolean isConnectionError(Throwable ex) {
        return ex instanceof java.net.ConnectException || ex instanceof java.net.UnknownHostException;
    }

    // 拉取友善的外部页面内容并返回给客户端
    private Mono<Void> fetchFriendly404Content(ServerWebExchange exchange) {
        exchange.getResponse().setStatusCode(HttpStatus.NOT_FOUND); // 设置 HTTP 状态码为 404

        // 使用 WebClient 拉取外部页面内容
        return webClient.get()
                .uri(page404)
                .retrieve()
                .bodyToMono(String.class) // 获取外部页面的响应体
                .flatMap(body -> {
                    // 将外部页面内容作为响应体返回客户端
                    DataBuffer buffer = exchange.getResponse().bufferFactory().wrap(body.getBytes());
                    exchange.getResponse().getHeaders().add("Content-Type", "text/html; charset=UTF-8");
                    return exchange.getResponse().writeWith(Mono.just(buffer));
                })
                .onErrorResume(ex -> {
                    // 如果外部页面也不可用，返回一个默认的 HTML 提示页面
                    String fallbackContent = "<h1>404 - Service Not Found</h1><p>External page unavailable</p>";
                    DataBuffer buffer = exchange.getResponse().bufferFactory().wrap(fallbackContent.getBytes());
                    exchange.getResponse().getHeaders().add("Content-Type", "text/html; charset=UTF-8");
                    return exchange.getResponse().writeWith(Mono.just(buffer));
                });
    }

    @Override
    public int getOrder() {
        // Before NettyRoutingFilter so the URL is
        // changed just before the actual request is made.

        // Before WebsocketRoutingFilter so ws request can be routed right
        return Ordered.LOWEST_PRECEDENCE - 2;
    }
}

PtyRoute

import com.xxxx.filter.GoServiceFilter;
import lombok.extern.slf4j.Slf4j;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.cloud.gateway.route.RouteLocator;
import org.springframework.cloud.gateway.route.builder.RouteLocatorBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;

/**
 * pty 路由到 service
 */
@Slf4j
@Configuration
public class PtyRoute {
    @Value("${biz.pty.host}")
    private String host;
    @Value("${biz.webview-controller.service}")
    private String webviewControllerService;

    @Autowired
    GoServiceFilter goServiceFilter;

    @Bean
    public RouteLocator innerRoutes(RouteLocatorBuilder builder) {
        return builder.routes()
                .route(p -> p
                        .host(host).and()
                        .path("/webview-controller/")
                        .filters(f -> f
                                .dedupeResponseHeader("Access-Control-Allow-Origin", "RETAIN_FIRST")
                                .dedupeResponseHeader("Access-Control-Allow-Credentials", "RETAIN_FIRST")
                        )
                        .uri(webviewControllerService))
                .route(p -> p
                        .host(host).and()
                        .path("/**")
                        .filters(f -> f
                                .filter(goServiceFilter)
                        )
                        .uri("no://op")) // 这里只是一个占位，实际会被过滤器动态重写
                .build();
    }
}

文章版权归作者所有，未经允许请勿转载。如内容涉嫌侵权，请在本页底部进入<联系我们>进行举报投诉!

THE END