Sounds to me like the data needs to be hidden from the web site in a database
with authentication to view records. You could easily allow users to see say
some eg 100-200 records even if not authenticated but not more.
There is a similar restriction in data for instance with electoral roles – you
can look up names but not scrape data – there the reasons are for privacy as
well as to protect their income as they sell data to certain users only.
From: Leigh Wanstead
Sent: Friday, July 4, 2014 10:39 AM
To: NZ Borland Developers Group - Delphi List
Subject: Re: [DUG] Work Wanted in Wellington
Hi David,
It is like amazon. Amazon does not require user name/password just browsing the
data.
Regards
Leigh
On 4 July 2014 10:29, David Brennan <[email protected]> wrote:
Sounds unusual. So the company sells the data but doesn’t have a login system
to control who consumes the data?
David.
From: [email protected]
[mailto:[email protected]] On Behalf Of Leigh Wanstead
Sent: Friday, 4 July 2014 10:16 a.m.
To: NZ Borland Developers Group - Delphi List
Subject: Re: [DUG] Work Wanted in Wellington
Hi Jolyon,
The company I work for is selling data. The data is the income of the company.
Regards
Leigh
On 4 July 2014 09:23, Jolyon Smith <[email protected]> wrote:
I don't understand this determination to make the hacker's life difficult.
Surely the objective is to address the impact on the site for legitimate users ?
Contriving schemes to make the hacker's life difficult is simply extending
the problem domain into an irrelevant area and increasing the complexity by
orders of magnitude in order to protect information that is public already -
there is no mention of any attempt to thwart site security, only scraping of
publicly accessible URL's..
If the intent is to disincentivise the hacker, simply denying them the
ability to scrape the site by detecting and blocking them will cause them
inconvenience enough. Even if it doesn't, as long as their activity is not
impacting on the legitimate operation of the site then the key objective is met
- that of maintaining site response for legit users.
Almost all of these schemes to make the scrapers life miserable do also
impact on the legitimate user experience, loading up the server and the client
browser processing with overhead targeted at the scraper but imposed on ALL
clients.
I can see that the technical challenge of "beating" the hacker could be
attractive, but it seems to me to be an ultimately pointless and resource
sapping "Arms Race" that cannot ever really be won... even if you eventually
drive the scraper to give up entirely, burdensome counter-measures will
themselves have impacted on your site, defeating if not the whole object then
certainly a significant part of it, of getting rid of the scraper activity in
the first place.
Of course, if you can find counter-measures which do not impose any such
burden on legit users then you have the best of both worlds, but the key need
to be met is addressing the scraper by removing the impact on legit users, not
adding to it.
So, bringing it back to the original topic - What makes a good developer ?
Another characteristic would be the ability to remain focused on the key
objective/user need, rather than being drawn into a bottomless honey pot of
technical challenge of limited/no direct relevance to the problem at hand.
:)
On 4 July 2014 09:05, Phil Scadden <[email protected]> wrote:
> Regarding to render the website in Javascript, how are you going to
> stop the browser driven by script? The hacker does not need to
> understand the javascript. All he need is just grab dom element.
That would be true but very unlikely that hacker is using browser. Too
slow. If you load the html with junk data and modify it with js, it may
take the hacker a long time to notice they are using crap. But I would
looking at detecting the hacker without a tip off in first place and
then figure out ways to make life difficult.
--
Phil Scadden, Senior Scientist GNS Science Ltd 764 Cumberland St,
Private Bag 1930, Dunedin, New Zealand Ph +64 3 4799663, fax +64 3 477 5232
Notice: This email and any attachments are confidential.
If received in error please destroy and immediately notify us.
Do not copy or disclose the contents.
_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: [email protected]
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to [email protected] with
Subject: unsubscribe
_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: [email protected]
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to [email protected] with
Subject: unsubscribe
_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: [email protected]
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to [email protected] with
Subject: unsubscribe
--------------------------------------------------------------------------------
_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: [email protected]
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to [email protected] with
Subject: unsubscribe_______________________________________________
NZ Borland Developers Group - Delphi mailing list
Post: [email protected]
Admin: http://delphi.org.nz/mailman/listinfo/delphi
Unsubscribe: send an email to [email protected] with
Subject: unsubscribe
