Consumer Reports' dismissal of HomePod a familiar tale to Apple fans [u]
Testing stalwart Consumer Reports declared this week that Apple's HomePod
falls behind Google's Home Max —and even the Sonos One —in terms of audio
quality. To anyone who knows the publication's controversial history with Apple
products, that conclusion is utterly unsurprising.
In Monday's screed, Consumer Reports, after three days of use and comparison,
declared the Sonos One, and the Google Home Max the champions for sound
quality. That's noteworthy, because virtually every other major test conducted
since the launch of HomePod, including AppleInsider's own comparisons, found
that Apple's product offers superior sound.
Don't just take our word for it (twice), though. Engadget on Tuesday agreed
with the prevailing wisdom on it, as did USA Today, The Verge, What Hi-Fi, a
very particular Redditor, and a pile of other venues.
Consumer Reports is, of course, entitled to its opinion —sound quality
assessments are highly subjective.
But two factors suggest that the report isn't relevant, nor of any particular
value. Primarily, Consumer Reports has a long history of berating Apple
products for issues that don't seem to actually manifest in any real-world
environment. Second, their HomePod comparison has come to a conclusion that
only they seem to hold.
Back in 2010, Consumer Reports used its public-facing webpage to flog issues it
had with the iPhone 4 antenna. But it required a subscription to the site to
view the entire comparative analysis, where it ranked the iPhone 4 above all
other smartphones of the day, saying that the antenna issue was completely
solved by applying piece of transparent tape over the antenna line in the
casing, using a protective case, or not squeezing it with a "death grip" while
making a phone call.
That fired up the internet as a whole, who saw only the public-facing post
decrying the antenna —yet the same complainants were not widely exposed to the
full report stashed behind a paywall that said that the problems were
incredibly minor and the iPhone 4 was at the top of the heap.
As a result of the growing controversy, Apple uncharacteristically made a
public statement, and provided free cases for early adopters, in an attempt to
put the issue to bed.
Gripping any mobile phone will result in some attenuation of its antenna
performance, with certain places being worse than others depending on the
placement of the antennas. This is a fact of life for every wireless phone. If
you ever experience this on your iPhone 4, avoid gripping it in the lower left
corner in a way that covers both sides of the black strip in the metal band, or
simply use one of many available cases.
The laws of physics are inescapable, and every smartphone even now can be
shielded by human flesh when pressed up against an antenna gap
And yet, somehow, this only applied to the iPhone 4, and not the other devices
on the list according to the venue —even though at the same time literally
every other phone on their evaluation was seeing the same thing.
Consumer Reports didn't bother to say anything about it afterward.
The 2016 MacBook Pro
Apple released the MacBook Pro redesign in the fall of 2016, and there was much
to be said about it. As it always does, Consumer Reports got their hands on one
shortly after the machines shipped in late October and early November.
After about a month and a half of testing, in a post on their website, the
publication said that while the new machines earned high marks in display
quality and performance, they were found lacking in terms of battery life.
Specifically, the battery performance of models tested "varied dramatically"
during trials, and was as low as four hours, with the computers that were not
under serious load.
However, it was discovered that, during their testing, Consumer Reports toggled
a hidden developer setting, which triggered an obscure bug. While the bug's
presence was Apple's responsibility, tooling around in the developer's settings
for Safari isn't a "real world" test, contrary to what publication claims.
"This is not a setting used by customers and does not reflect real-world usage.
Their use of this developer setting also triggered an obscure and intermittent
bug reloading icons which created inconsistent results in their lab," Apple
said. "After we asked Consumer Reports to run the same test using normal user
settings, they told us their MacBook Pro systems consistently delivered the
expected battery life."
Following the bug fix, and Consumer Reports not messing around with developer
settings, the product testers reinstated their recommendation for the machine
—an improvement over how they handled the iPhone 4 debacle.
Unlike its declaration about the HomePod Monday just three days after release,
Consumer Reports waited until its month-long testing was fully complete on the
MacBook Pro before it published the results.
The iPhone X
The publication also disclosed its testing on the iPhone X and a bevy of other
smartphones on Dec. 5 of last year, a month after the iPhone X became
available. That report found the publication recommending the Galaxy S8 above
the iPhone 8 Plus, with the iPhone X in a very close third spot.
The testing was dramatic, with the phones taking major screen damage in a steel
tumbling device after 50 rotations. We're not sure how "real world" this is, as
the vast majority of iPhone drops are from approximately three feet to
concrete, and only a few times in a product's life.
The iPhone X tumbler
However, again, Consumer Reports didn't talk about early results until they had
been beating on the phones in question for a month —more than 10 times the
length of time it had before it judged the HomePod.
Consumer Reports has every legal right to share its opinions, the same as
AppleInsider or any other venue. Examining a product in a review is, by
definition, a subjective process, with audio product reviews producing the
widest gap of opinion.
If you're looking for an "objective review," or similar phrasing, you aren't
looking for a review at all. Instead, find a news item dispassionately
discussing the technical specifications of a product. By definition, a review
isn't that, and is chock-full of opinions about a product.
Perhaps Consumer Reports felt like they had collected all the information they
needed after just a few days. That's possible, and with a product evaluated in
a vacuum, and not compared to much else, a review can be done in less than that
timeframe. However, this is contrary to every other consumer electronics
comparison that the organization has ever done.
Damning with faint praise
It would have been better had Consumer Reports waited until their comparative
testing was done to scream loudly that one smart speaker sounds better than the
next. But, instead, this nearly century-old publication decided to say very
little about how they reached their conclusions. Instead, they sensationally
disclosed their findings, some of which flew contrary to everyone else's. And
they did so just days after a product release, and yet they still give
themselves an out if their exhaustive testing proved otherwise.
Why they took this atypical path to publication, we don't know, and can only
guess, since they have yet to respond to questions about the release.
Right now, we don't know anything about the testing methodology. We don't know
why Consumer Reports decided what it did, despite their press release vividly
declaring that Google's offering is better.
I am not here to defend the HomePod, and Apple neither requires nor desires my
assistance —and that's how it should be. I don't have a HomePod, nor do I have
a Google Home Max, so I can't speak to the accuracy of the Consumer Reports
But Monday's report, how it was disclosed, and a lack of a supporting narrative
for the opinion, is not a good look for Consumer Reports, given their history
with Apple products. The release is contrary to their own stated mission of
"rigorously testing products" before advising the public.
Worse yet, the early announcement stands in violation of what I think is their
most profound mission —they have failed to be a publication capable of
"fearlessly investigating where markets have failed," and look to have become
part of the problem instead.
Sent from my iPhone
The following information is important for all members of the Mac Visionaries
If you have any questions or concerns about the running of this list, or if you
feel that a member's post is inappropriate, please contact the owners or
moderators directly rather than posting on the list itself.
Your Mac Visionaries list moderator is Mark Taylor. You can reach mark at:
mk...@ucla.edu and your owner is Cara Quinn - you can reach Cara at
The archives for this list can be searched at:
You received this message because you are subscribed to the Google Groups
To unsubscribe from this group and stop receiving emails from it, send an email
To post to this group, send email to email@example.com.
Visit this group at https://groups.google.com/group/macvisionaries.
For more options, visit https://groups.google.com/d/optout.