February 12, 2011

The Dirty Little Secrets of Search
By DAVID SEGAL
NY Times

http://www.nytimes.com/2011/02/13/business/13search.html?ref=technology&pagewanted=print


PRETEND for a moment that you are Google’s search engine.

Someone types the word “dresses” and hits enter. What will be the very 
first result?

There are, of course, a lot of possibilities. Macy’s comes to mind. 
Maybe a specialty chain, like J. Crew or the Gap. Perhaps a Wikipedia 
entry on the history of hemlines.

O.K., how about the word “bedding”? Bed Bath & Beyond seems a candidate. 
Or Wal-Mart, or perhaps the bedding section of Amazon.com.

“Area rugs”? Crate & Barrel is a possibility. Home Depot, too, and 
Sears, Pier 1 or any of those Web sites with “area rug” in the name, 
like arearugs.com.

You could imagine a dozen contenders for each of these searches. But in 
the last several months, one name turned up, with uncanny regularity, in 
the No. 1 spot for each and every term:

J. C. Penney.

The company bested millions of sites — and not just in searches for 
dresses, bedding and area rugs. For months, it was consistently at or 
near the top in searches for “skinny jeans,” “home decor,” “comforter 
sets,” “furniture” and dozens of other words and phrases, from the 
blandly generic (“tablecloths”) to the strangely specific (“grommet top 
curtains”).

This striking performance lasted for months, most crucially through the 
holiday season, when there is a huge spike in online shopping. J. C. 
Penney even beat out the sites of manufacturers in searches for the 
products of those manufacturers. Type in “Samsonite carry on luggage,” 
for instance, and Penney for months was first on the list, ahead of 
Samsonite.com.

With more than 1,100 stores and $17.8 billion in total revenue in 2010, 
Penney is certainly a major player in American retailing. But Google’s 
stated goal is to sift through every corner of the Internet and find the 
most important, relevant Web sites.

Does the collective wisdom of the Web really say that Penney has the 
most essential site when it comes to dresses? And bedding? And area 
rugs? And dozens of other words and phrases?

The New York Times asked an expert in online search, Doug Pierce of Blue 
Fountain Media in New York, to study this question, as well as Penney’s 
astoundingly strong search-term performance in recent months. What he 
found suggests that the digital age’s most mundane act, the Google 
search, often represents layer upon layer of intrigue. And the intrigue 
starts in the sprawling, subterranean world of “black hat” optimization, 
the dark art of raising the profile of a Web site with methods that 
Google considers tantamount to cheating.

Despite the cowboy outlaw connotations, black-hat services are not 
illegal, but trafficking in them risks the wrath of Google. The company 
draws a pretty thick line between techniques it considers deceptive and 
“white hat” approaches, which are offered by hundreds of consulting 
firms and are legitimate ways to increase a site’s visibility. Penney’s 
results were derived from methods on the wrong side of that line, says 
Mr. Pierce. He described the optimization as the most ambitious attempt 
to game Google’s search results that he has ever seen.

“Actually, it’s the most ambitious attempt I’ve ever heard of,” he said. 
“This whole thing just blew me away. Especially for such a major brand. 
You’d think they would have people around them that would know better.”

TO understand the strategy that kept J. C. Penney in the pole position 
for so many searches, you need to know how Web sites rise to the top of 
Google’s results. We’re talking, to be clear, about the “organic” 
results — in other words, the ones that are not paid advertisements. In 
deriving organic results, Google’s algorithm takes into account dozens 
of criteria, many of which the company will not discuss.

But it has described one crucial factor in detail: links from one site 
to another.

If you own a Web site, for instance, about Chinese cooking, your site’s 
Google ranking will improve as other sites link to it. The more links to 
your site, especially those from other Chinese cooking-related sites, 
the higher your ranking. In a way, what Google is measuring is your 
site’s popularity by polling the best-informed online fans of Chinese 
cooking and counting their links to your site as votes of approval.

But even links that have nothing to do with Chinese cooking can bolster 
your profile if your site is barnacled with enough of them. And here’s 
where the strategy that aided Penney comes in. Someone paid to have 
thousands of links placed on hundreds of sites scattered around the Web, 
all of which lead directly to JCPenney.com.

Who is that someone? A spokeswoman for J. C. Penney, Darcie Brossart, 
says it was not Penney.

“J. C. Penney did not authorize, and we were not involved with or aware 
of, the posting of the links that you sent to us, as it is against our 
natural search policies,” Ms. Brossart wrote in an e-mail. She added, 
“We are working to have the links taken down.”

The links do not bear any fingerprints, but nothing else about them was 
particularly subtle. Using an online tool called Open Site Explorer, Mr. 
Pierce found 2,015 pages with phrases like “casual dresses,” “evening 
dresses,” “little black dress” or “cocktail dress.” Click on any of 
these phrases on any of these 2,015 pages, and you are bounced directly 
to the main page for dresses on JCPenney.com.

Some of the 2,015 pages are on sites related, at least nominally, to 
clothing. But most are not. The phrase “black dresses” and a Penney link 
were tacked to the bottom of a site called 
nuclear.engineeringaddict.com. “Evening dresses” appeared on a site 
called casino-focus.com. “Cocktail dresses” showed up on 
bulgariapropertyportal.com. ”Casual dresses” was on a site called 
elistofbanks.com. “Semi-formal dresses” was pasted, rather 
incongruously, on usclettermen.org.

There are links to JCPenney.com’s dresses page on sites about diseases, 
cameras, cars, dogs, aluminum sheets, travel, snoring, diamond drills, 
bathroom tiles, hotel furniture, online games, commodities, fishing, 
Adobe Flash, glass shower doors, jokes and dentists — and the list goes on.

Some of these sites seem all but abandoned, except for the links. The 
greeting at myflhomebuyer.com sounds like the saddest fortune cookie 
ever: “Sorry, but you are looking for something that isn’t here.”

When you read the enormous list of sites with Penney links, the 
landscape of the Internet acquires a whole new topography. It starts to 
seem like a city with a few familiar, well-kept buildings, surrounded by 
millions of hovels kept upright for no purpose other than the ads that 
are painted on their walls.

Exploiting those hovels for links is a Google no-no. The company’s 
guidelines warn against using tricks to improve search engine rankings, 
including what it refers to as “link schemes.” The penalty for getting 
caught is a pair of virtual concrete shoes: the company sinks in 
Google’s results.

Often drastically. In 2006, Google announced that it had caught BMW 
using a black-hat strategy to bolster the company’s German Web site, 
BMW.de. That site was temporarily given what the BBC at the time called 
“the death penalty,” stating that it was “removed from search results.”

BMW acknowledged that it had set up “doorway pages,” which exist just to 
attract search engines and then redirect traffic to a different site. 
The company at the time said it had no intention of deceiving users, 
adding “if Google says all doorway pages are illegal, we have to take 
this into consideration.”

J. C. Penney, it seems, will not suffer the same fate. But starting 
Wednesday, it was the subject of what Google calls “corrective action.”

Last week, The Times sent Google the evidence it had collected about the 
links to JCPenney.com. Google promptly set up an interview with Matt 
Cutts, the head of the Webspam team at Google, and a man whose every 
speech, blog post and Twitter update is parsed like papal encyclicals by 
players in the search engine world.

“I can confirm that this violates our guidelines,” said Mr. Cutts during 
an hourlong interview on Wednesday, after looking at a list of paid 
links to JCPenney.com.

He said Google had detected previous guidelines violations related to 
JCPenney.com on three occasions, most recently last November. Each time, 
steps were taken that reduced Penney’s search results — Mr. Cutts avoids 
the word “punished” — but Google did not later “circle back” to the 
company to see if it was still breaking the rules, he said.

He and his team had missed this recent campaign of paid links, which he 
said had been up and running for the last three to four months.

“Do I wish our system had detected things sooner? I do,” he said. “But 
given the one billion queries that Google handles each day, I think we 
do an amazing job.”

Mr. Cutts sounded remarkably upbeat and unperturbed during this 
conversation, which was a surprise given that we were discussing a 
large, sustained effort to snooker his employer. Asked about his zenlike 
calm, he said the company strives not to act out of anger. You get the 
sense that Mr. Cutts and his colleagues are acutely aware of the 
singular power they wield as judge, jury and appeals panel, and they’re 
eager to project an air of maturity and judiciousness.

That said, he added, “I don’t think I could do my job well if in some 
sense I was not offended by things that were bad for Google users.”

“Am I happy this happened?” he later asked. “Absolutely not. Is Google 
going to take strong corrective action? We absolutely will.”

And the company did. On Wednesday evening, Google began what it calls a 
“manual action” against Penney, essentially demotions specifically aimed 
at the company.

At 7 p.m. Eastern time on Wednesday, J. C. Penney was still the No. 1 
result for “Samsonite carry on luggage.”

Two hours later, it was at No. 71.

At 7 p.m. on Wednesday, Penney was No. 1 in searches for “living room 
furniture.”

By 9 p.m., it had sunk to No. 68.

In other words, one moment Penney was the most visible online 
destination for living room furniture in the country.

The next it was essentially buried.

PENNEY reacted to this instant reversal of fortune by, among other 
things, firing its search engine consulting firm, SearchDex. Executives 
there did not return e-mail or phone calls.

Penney also issued a statement: “We are disappointed that Google has 
reduced our rankings due to this matter,” Ms. Brossart wrote, “but we 
will continue to work actively to retain our high natural search position.”

She added that while the collection of links surely brought in 
additional revenue, it was hardly a bonanza. Just 7 percent of 
JCPenney.com’s traffic comes from clicks on organic search results, she 
wrote. A far bigger source of profits this holiday season, she stated, 
came from partnerships with companies like Yahoo and Time Warner, from 
new mobile applications and from in-store kiosks.

Search experts, however, say Penney likely reaped substantial rewards 
from the paid links. If you think of Google as the entrance to the 
planet’s largest shopping center, the links helped Penney appear as 
though it was the first and most inviting spot in the mall, to millions 
and millions of online shoppers.

How valuable was that? A study last May by Daniel Ruby of Chitika, an 
online advertising network of 100,000 sites, found that, on average, 34 
percent of Google’s traffic went to the No. 1 result, about twice the 
percentage that went to No. 2.

The Keyword Estimator at Google puts the number of searches for 
“dresses” in the United States at 11.1 million a month, an average based 
on 12 months of data. So for “dresses” alone, Penney may have been 
attracting roughly 3.8 million visits every month it showed up as No. 1. 
Exactly how many of those visits translate into sales, and the size of 
each sale, only Penney would know.

But in January, the company was crowing about its online holiday sales. 
Kate Coultas, a company spokeswoman, wrote to a reporter in January, 
“Internet sales through jcp.com posted strong growth in December, with 
significant increases in traffic and orders for the key holiday shopping 
periods of the week after Thanksgiving and the week before Christmas.”

There was considerable pressure from investors for Penney to deliver 
strong holiday results. It has been struggling through one of the more 
trying times of its century of retailing. The $17.8 billion in revenue 
it reported last year is the exact same figure it reported in 2001. It 
announced in January that it would close a handful of underperforming 
stores, as well as two of its five call centers and 19 outlets that sell 
excess catalog merchandise.

Adding to the company’s woes is the demise of its catalog business. 
Penney has phased out what it called its Big Book and poured money into 
its Web site. But so far, the loss of the catalog has not been offset by 
the expansion of the Web site. At its peak, the catalog brought in about 
$4 billion in revenue. In 2009, the site brought in $1.5 billion.

“For the last 35 years, Penney has tried to be accepted as a department 
store, and during unusually good times, it does very well,” said Bernard 
Sosnick, an analyst at Gilford Securities. “But in bad times, it gets 
punished by shoppers who pull back after having spent aspirationally.”

MANY owners of Web sites with Penney links seem to relish their 
unreachability. But there were exceptions, and they included cocaman.ch. 
(“Geekness — closer to the world” is the cryptic header atop the site.) 
It turned out to be owned and run by Corsin Camichel, a chatty 
25-year-old I.T. security analyst in Switzerland.

The word “dresses” appears in a small collection of links in the middle 
of a largely blank Cocaman page. Asked about that link, Mr. Camichel 
said his records show that it turned up on his site last April, though 
he said it might have been earlier than that.

The link came through a Web site, TNX.net, which pays Mr. Camichel with 
TNX points, which he then trades for links that drive traffic to his 
other sites, like cookingutensils.net. He earns money when people visit 
that site and click on the ads. He could also, he said, get cash from 
TNX. Currently, Cocaman is home to 403 links, all of them placed there 
by TNX on behalf of clients.

“You do pretty well,” he wrote, referring to income from his links 
trading. “The thing is, the more you invest (time and money) the better 
results you get. Right now I get enough to buy myself new test devices 
for my Android apps (like $150/month) with zero effort. I have to do 
nothing. Ads just sit there and if people click, I make money.”

Efforts to reach TNX itself last week via e-mail were not successful.

Interviewing a purveyor of black-hat services face-to-face was a 
considerable undertaking. They are a low-profile bunch. But a 
link-selling specialist named Mark Stevens — who says he had nothing to 
do with the Penney link effort — agreed to chat. He did so on the 
condition that his company not be named, a precaution he justified by 
recounting what happened when the company apparently angered Google a 
few months ago.

“It was my fault,” Mr. Stevens said. “I posted a job opening on a 
Stanford Engineering alumni mailing list, and mentioned the name of our 
company and a brief description of what we do. I think some Google 
employees saw it.”

In a matter of days, the company could not be found in a Google search.

“Literally, you typed the name of the company into the search box and we 
did not turn up. Anywhere. You’d find us if you knew our Web address. 
But in terms of search, we just disappeared.”

The company now operates under a new name and with a profile that is low 
even in the building where it claims to have an office. The landlord at 
the building, a gleaming, glassy midrise next to Route 101 in Redwood 
City, Calif., said she had never heard of the company.

Mr. Stevens agreed to meet in mid-January for a dinner paid for by The 
Times. Asked to pick a “fine restaurant” in his neighborhood, he rather 
cheekily selected a modern French bistro in Palo Alto offering an 
eight-course prix fixe meal for $118. Liquid nitrogen and “fairy tale 
pumpkin” were two of the featured ingredients.

Mr. Stevens turned out to be a boyish-looking 31-year-old native of 
Singapore. (Stevens is the name he uses for work; he says he has a 
Chinese last name, which he did not share.) He speaks with a slight 
accent and in an animated hush, like a man worried about eavesdroppers. 
He describes his works with the delighted, mischievous grin of a 
sophomore who just hid a stink bomb.

“The key is to roll the campaign out slowly,” he said as he nibbled at 
seared duck foie gras. “A lot of companies are in a rush. They want as 
many links as we can get them as fast as possible. But Google will spot 
that. It will flag a Web site that goes from zero links to a few hundred 
in a week.”

The hardest part about the link-selling business, he explained, is 
signing up deep-pocketed mainstream clients. Lots of them, it seems, are 
afraid they’ll get caught. Another difficulty is finding quality sites 
to post links. Whoever set up the JCPenney.com campaign, he said, relied 
on some really low-rent, spammy sites — the kind with low PageRanks, as 
Google calls its patented measure of a site’s quality. The higher the 
PageRank, the more “Google juice” a site offers others to which it is 
linked.

“The sites that TNX uses mostly have low PageRanks,” Mr. Stevens said.

Mr. Stevens said that Web site owners, or publishers, as he calls them, 
get a small fee for each link, and the transaction is handled entirely 
over the Web.

Publishers can reject certain keywords and links — Mr. Stevens said some 
balked at a lingerie link — but for the most part the system is on a 
kind of autopilot. A client pays Mr. Stevens and his colleagues for 
links, which are then farmed out to Web sites. Payment to publishers is 
handled via PayPal.

You might expect Mr. Stevens to have a certain amount of contempt for 
Google, given that he spends his professional life finding ways to 
subvert it. But through the evening he mentioned a few times that he’s 
in awe of the company, and the quality of its search engine.

So how does he justify all his efforts to undermine that engine?

“I think we need to make a distinction between two different kinds of 
searches — informational and commercial,” he said. “If you search 
‘cancer,’ that’s an informational search and on those, Google is 
amazing. But in commercial searches, Google’s results are really 
polluted. My own personal experience says that the guy with the biggest 
S.E.O. budget always ranks the highest.”

To Mr. Stevens, S.E.O. is a game, and if you’re not paying black hats, 
you are losing to rivals with fewer compunctions.

WHY did Google fail to catch a campaign that had been under way for 
months? One, no less, that benefited a company that Google had already 
taken action against three times? And one that relied on a collection of 
Web sites that were not exactly hiding their spamminess?

Mr. Cutts emphasized that there are 200 million domain names and a mere 
24,000 employees at Google.

“Spammers never stop,” he said. Battling those spammers is a 
never-ending job, and one that he believes Google keeps getting better 
and better at.

Here’s another hypothesis, this one for the conspiracy-minded. Last 
year, Advertising Age obtained a Google document that listed some of its 
largest advertisers, including AT&T, eBay and yes, J. C. Penney. The 
company, this document said, spent $2.46 million a month on paid Google 
search ads — the kind you see next to organic results.

Is it possible that Google was willing to countenance an extensive 
black-hat campaign because it helped one of its larger advertisers? It’s 
the sort of question that European Union officials are now studying in 
an investigation of possible antitrust abuses by Google.

Investigators have been asking advertisers in Europe questions like 
this: “Please explain whether and, if yes, to what extent your 
advertising spending with Google has ever had an influence on your 
ranking in Google’s natural search.” And: “Has Google ever mentioned to 
you that increasing your advertising spending could improve your ranking 
in Google’s natural search?”

Asked if Penney received any breaks because of the money it has spent on 
ads, Mr. Cutts said, “I’ll give a categorical denial.” He then made an 
impassioned case for Google’s commitment to separating the money side of 
the business from the search side. The former has zero influence on the 
latter, he said.

“If you asked me for the names of five people in advertising 
engineering, I don’t think I could give you the names,” he said. “There 
is a very long history at Google of saying ‘We are not going to worry 
about short-term revenue.’ ” He added: “We rely on the trust of our 
users. We realize the responsibility that we have to our users.”

He noted, too, that before The Times presented evidence of the paid 
links to JCPenney.com, Google had just begun to roll out an algorithm 
change that had a negative effect on Penney’s search results. (The tweak 
affected “how we trust links,” Mr. Cutts said, declining to elaborate.)

True, JCPenney.com’s showing in Google searches had declined slightly by 
Feb. 8, as the algorithm change began to take effect. In “comforter 
sets,” Penney went from No. 1 to No. 7. In “sweater dresses,” from No. 1 
to No. 10.

But the real damage to Penney’s results began when Google started that 
“manual action.” The decline can be charted: On Feb. 1, the average 
Penney position for 59 search terms was 1.3.

On Feb. 8, when the algorithm was changing, it was 4.

By Feb. 10, it was 52.

MR. CUTTS said he did not plan to write about Penney’s situation, as he 
did with BMW in 2006. Rarely, he explained, does he single out a company 
publicly, because Google’s goal is to preserve the integrity of results, 
not to embarrass people.

“But just because we don’t talk about it,” he said, “doesn’t mean we 
won’t take strong action.”

-- 
========================================
George Antunes, Political Science Dept
University of Houston; Houston, TX 77204
Voice: 713-743-3923  Fax: 713-743-3927
Mail: antunes at uh dot edu

_______________________________________________
Medianews mailing list
[email protected]
http://lists.etskywarn.net/mailman/listinfo/medianews

Reply via email to