MonsterChenzhuo commented on code in PR #4372:
URL:
https://github.com/apache/incubator-seatunnel/pull/4372#discussion_r1149948596
##########
docs/en/connector-v2/source/Http.md:
##########
@@ -289,6 +293,32 @@ Http {
}
```
+### page options
+
+## Example
+
+simple:
+
+```hocon
+Http {
+ url = "https://tyrantlucifer.com/api/getDemoData";
+ pageing = {
+ pageNoField = "pageNo",
+ pageNo = "1-2,4,8-10",
Review Comment:
At present, there are two common formats:
<img width="1144" alt="图片"
src="https://user-images.githubusercontent.com/60029759/228106583-f8fbcb64-64b4-412e-9700-a49fb2886240.png";>
<img width="574" alt="图片"
src="https://user-images.githubusercontent.com/60029759/228106749-7f4eba17-2e94-4fa4-8431-0d67be29309e.png";>
If I want to get totalPage, I need to do it:
1.First, request an interface `response = httpClient.execute(httpParameter);`
Get the response body, parse the response body, and get the value of
totalPage.
`ObjectMapper mapper = new ObjectMapper();`
`JsonNode root = mapper.readTree(response.getContent());`
` int totalPage =root.get(“data”).get("totalPageField").asInt();`
`for (int pageNumber = 2; pageNumber <= totalPage; pageNumber++) {`
// Loop through to get paging content
`}`
A sample configuration of seatunnel is as follows:
simple1: Data reading by paging ranges
```hocon
Http {
url = "https://test.com/api/getDemoData";
method = "GET"
format = "json"
pageing = {
page_no_field = "pageNo",
page_no = "1-2,4,8-10",
page_size_field = "pageSize",
page_size = 10
}
schema {
fields {
code = int
message = string
data = string
ok = boolean
}
}
}
```
simple2: Self-incrementing paging until no data is fetched
```hocon
Http {
url = "https://test.com/api/getDemoData";
method = "GET"
format = "json"
pageing = {
page_no_field = "pageNo",
page_size_field = "pageSize",
page_size = 10,
body_field = "data",
total_page_field = "totalPage"
}
schema {
fields {
code = int
message = string
data = string
ok = boolean
}
}
}
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
![https://user-images.githubusercontent.com/60029759/228106583-f8fbcb64-64b4-412e-9700-a49fb2886240.png"](https://user-images.githubusercontent.com/60029759/228106583-f8fbcb64-64b4-412e-9700-a49fb2886240.png"){kind=link}
![https://user-images.githubusercontent.com/60029759/228106749-7f4eba17-2e94-4fa4-8431-0d67be29309e.png"](https://user-images.githubusercontent.com/60029759/228106749-7f4eba17-2e94-4fa4-8431-0d67be29309e.png"){kind=link}