I'm trying to get a few links from website, and i have problem; When im send
request, response give me only half (or less) html content. I suspect there is
something wrong with my request, but i check this in fiddler, and in fiddler i
see complete response This is my code;
proc getlinks(sk: string, ilestr: int): string =
echo "Pobieram linki"
let header = newHttpHeaders()
header["Upgrade-Insecure-Requests"] = "1"
header["Accept"] =
"text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8"
#header["Accept-Encoding"] = "gzip, deflate, br"
header["Accept-Language"] = "pl,en-US;q=0.7,en;q=0.3"
header["Referer"] = "https://www.xxx.pl/";
header["TE"] = "Trailers"
header["User-Agent"] = "nim1"
header["DNT"] = "1"
for i in 0..ilestr:
let gethtml = request(client, sk, HttpGet, "", header)
let html = parseHtml(gethtml.body)
echo gethtml.body
for a in html.findAll("a"):
if a.attrs.hasKey "href":
let sp = a.attrs["href"]
if sp.contains(",") == true:
#echo sp
list.append(sp)
echo "Pobrano linki z " & $i & " stron"
echo "Koniec"
Run
What could be wrong? And second question, what is a best way, to handle with
cookie? Somthing like cookiecontainer?
