poorbarcode opened a new issue, #19826:
URL: https://github.com/apache/pulsar/issues/19826
### Motivation
We have two ways to manage pulsar's resources:
- By client API (Can manage some resources, such as `create topic`, `create
subscriber`, and so on)
- By admin API (Can manage all the resources)
The `client API` has no limit on the request length. And the `admin API` has
a limit on the request length(such as HTTP request line and HTTP request
headers), this restriction is done by the built-in web container Jetty.
Almost resources can be created by two APIs, but can only be modified and
deleted by `admin API`. This causes us to be unable to modify or delete
resources created by `client API` with too long a name because it exceeds
Jetty's default HTTP request URI length limit.
### Goal
#### 1. For web servers
Provide a way to modify Jetty's `httpMaxRequestHeaderSize` configuration
(involves two servers: the web server in pulsar and the web server in
pulsar-proxy)
#### 2.For the internal client in pulsar-proxy
Provide a way to modify Jetty-client's `httpClientRequestBufferSize`
configuration.
Since the pulsar-proxy handles HTTP requests like this： `pulsar-admin.sh` ->
`proxy web server` -> `(highlight) internal client in proxy` -> `pulsar web
server`.
When the internal client forwards a request, it forwards the request header
and the request body, and all the data passes through a buffer( we call it Buf
), like this:
- Receive a request
- Put the request line and request headers input to the Buf.
- <strong>(highlight)</strong>Flush the Buf ( If the data in the request
line and request header exceeds the length of the buf, an error is reported )
- Put the request body input to the Buf.
- Flush the Buf if it is full.
So we need a config to set the `buff size` of the Buf:
`pulsar-proxy.conf.httpClientRequestBufferSize` -> `buf size of the internal
client`.
### API Changes
#### ServiceConfiguration.java
```java
@FieldContext(
category = CATEGORY_HTTP,
doc = """
The maximum size in bytes of the request header.
Larger headers will allow for more and/or larger cookies
plus larger form content encoded in a URL.
However, larger headers consume more memory and can make a
server more vulnerable to denial of service
attacks.
"""
)
private int httpMaxRequestHeaderSize = 8 * 1024;
```
#### ProxyConfiguration.java
```java
@FieldContext(
minValue = 1,
category = CATEGORY_HTTP,
doc = """
The maximum size in bytes of the request header.
Larger headers will allow for more and/or larger cookies
plus larger form content encoded in a URL.
However, larger headers consume more memory and can make a
server more vulnerable to denial of service
attacks.
"""
)
private int httpMaxRequestHeaderSize = 8 * 1024;
@FieldContext(
minValue = 1,
category = CATEGORY_HTTP,
doc = """
the size of the buffer used to write requests to Broker.
if "httpMaxRequestHeaderSize" is large than
"httpClientRequestBufferSize", will set
"httpClientRequestBufferSize" to the value of
"httpMaxRequestHeaderSize"
"""
)
private int httpClientRequestBufferSize = httpMaxRequestHeaderSize;
```
### Implementation
-
### Security Considerations
-
### Alternatives
_No response_
### Anything else?
This change should cherry-pick into the previous branches ( includes
`2.9~2.11` )
If the user uses the features `RETRY Topic` or `DLQ`, it is possible that
pulsar will automatically create some topics with names that are too long and
cannot be managed, the [scenario has been discussed in the
email](https://lists.apache.org/thread/q1m23ckyy10wvtzy65v8bwqwnh7r0gc8) before
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
