Hi Arcadius, Did you determine that that was the problem with CONNECTORS-1113?
The issue with cookies that change the experience of a user is very complex, because it usually means that there's some sequence of pages that get fetched which you DON'T want to index, that are involved in setting the cookies, and then you want all other pages for that site to be blocked until those cookies are set. This is exactly what session login is set up to do. So all you need to do, if you want cookies to work, is figure out what sequence of pages that site is doing to set cookies etc., and supply those as a login sequence for the site. Yes, this is complicated, because every site is different, and unless you want MCF to do something lame like not insure that the proper cookies go to ever page being fetched for indexing, there is no choice. Thanks, Karl On Tue, Nov 25, 2014 at 2:23 AM, Arcadius Ahouansou <[email protected]> wrote: > > Hello. > Many modern website now require cookies for a better user experience. > > Please, how can one setup ManifoldCF web crawler to automatically accept > and remember cookies in a crawling session? > > Thanks. > > -- > Arcadius Ahouansou > Menelic Ltd | Information is Power > M: 07908761999 > W: www.menelic.com > --- >
