gitcome commented on issue #7973: URL: https://github.com/apache/apisix/issues/7973#issuecomment-1255853598
> 十分抱歉，英文不太好。我重新用中文描述一下，因为线上使用Apisix出了问题急着解决，麻烦帮忙看一下。 调用链是这样的 ``` grpc客户端 <-> Apisix <-> 过滤器apisix-go-plugin-runner <-> grpc服务端 ``` - 搭建grpc客户端和grpc服务端 1. grpc协议文件 https://github.com/grpc/grpc/blob/master/examples/protos/route_guide.proto protoc生成go grpc文件 2. grpc服务端代码 ``` func start(ctx context.Context) { lis, err := net.Listen("tcp", fmt.Sprintf(":%d", 8301)) if err != nil { logs.Error("net.Listen error: ", err) return } var options []grpc.ServerOption grpcServer := grpc.NewServer(options...) defer grpcServer.GracefulStop() svr := &TestSvc{} routeguide.RegisterRouteGuideServer(grpcServer, svr) go grpcServer.Serve(lis) <-ctx.Done() } type TestSvc struct { routeguide.UnimplementedRouteGuideServer } func (s *TestSvc) GetFeature(ctx context.Context, req *routeguide.Point) (*routeguide.Feature, error) { rsp := &routeguide.Feature{} rsp.Name = "tmp" rsp.Location = &routeguide.Point{Latitude: 1, Longitude: 2} return rsp, nil } ``` 3. grpc客户端代码 ``` func start(ctx context.Context) { conn, err := grpc.Dial("192.168.56.19:9080", grpc.WithTransportCredentials(insecure.NewCredentials())) if nil != err { return } defer conn.Close() client := routeguide.NewRouteGuideClient(conn) var header metadata.MD var trailer metadata.MD req := routeguide.Point{Latitude: 99, Longitude: 88} rsp, err := cli.GetFeature(ctx, &req, grpc.Header(&header), grpc.Trailer(&trailer)) if err != nil { logs.Error(err) // 如果在apisix-go-plugin-runner直接返回，这里报closed the stream without sending trailers return } logs.Info("rsp:", rsp.String()) logs.Info("header:", header) logs.Info("trailer:", trailer) } ``` - 搭建Apisix 1. git clone https://github.com/apache/apisix-go-plugin-runner.git 2. vim apisix-go-plugin-runner-master/cmd/go-runner/plugins/say.go ``` // 修改请求过滤处理函数如下，逻辑很简单，随机一个数 // 是奇数的话，设置http200和一些header后返回，该请求直接返回客户端，客户端收到后报错closed the stream without sending trailers // 是偶数的话，RequestFilter直接return，该请求会到上游grpc server处理后返回到客户端，处理正常 func (p *Say) RequestFilter(conf interface{}, w http.ResponseWriter, r pkgHTTP.Request) { w.Header().Set("Content-Type", r.Header().Get("Content-Type")) num:=rand.Int31n(100) if num%2 == 1 { w.WriteHeader(http.StatusOK) w.Header().Set("Grpc-Status", "0") w.Header().Set("Grpc-Message", "0") w.Header().Set("Grpc-Status-Details-Bin", "") w.Header().Set("rspHeader", "value") return // 这里请求返回给客户端 报错 #标记# } // 这里请求会到上游grpc server 正常 } ``` 3. vim apisix-go-plugin-runner-master/ci/apisix/config.yaml 添加http2监听端口 ``` - port: 9080 enable_http2: true ``` 添加ext-plugin插件 ``` ext-plugin: cmd: ["/usr/local/apisix/go-runner", "run"] ``` 4. docker build -f Dockerfile -t apisixtest:v1 --progress=plain --no-cache . 5. docker run -p 9079:9079/tcp -p 9080:9080/tcp -p 9091:9091/tcp -p 9443:9443/tcp -p 9092:9092/tcp -v D:\apisix-go-plugin-runner-master\log:/usr/local/apisix/logs --name=apisixtest1 apisixtest:v1 运行api 6. 打开 http://localhost:9000/routes/list,配置上游服务和路由，并添加ext-plugin-post-req插件 ``` // 上游服务 { "nodes": [ { "host": "192.168.56.19", "port": 8301, "weight": 1 } ], "type": "roundrobin", "scheme": "grpc", } // 路由 { "uri": "/routeguide.RouteGuide/GetFeature", "name": "testa1", "methods": [ "POST" ], "plugins": { "ext-plugin-post-req": { "conf": [ { "name": "say", "value": "{\"body\":\"123\"}" } ], "disable": false } }, "upstream_id": "425744995673178819", "status": 1 } ``` 7. 使用上面的grpc client调用Apisix，发现如果代码来到上面#标记#处，接口返回报错closed the stream without sending trailers。如果请求过了apisix-go-plugin-runner并来到grpc server，则接口返回正常。 apisix-go-plugin-runner的代码也看了，找不出什么原因。猜想是go-plugin过滤请求返回给apisix后，apisix没有将回包grpc协议封装好就返回给客户端了。上面问题查了一周了，实在找不到解决方法，@tzssangglass @bzp2010 @soulbird 恳请帮忙一下，十分感谢！ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
