Can you explain Consumer Reports to me?

Question

Consumer Reports for July 2007 has a review of several digital cameras. Frankly I'm disappointed. For instance... Their top rated compact camera is the Canon Powershot S2-IS. My daughter owns this camera. It's a great camera, but it was introduced in April 2005. The S-series has gone through the S2-IS and the S3-IS and the S5-IS has just been introduced. You can't even find a copy of their "top rated" camera on the shelf anywhere. The S3-IS is 8th on their list, being marked DOWN for image quality from the S2-IS. The G7 is 12th on the same list, ranking below both the A640 and A630. Somehow I simply don't believe the A630 is better than a G7.

In the subcompact class, their top choice is the Sony DSC-N2 (9 months old), but the second choice is the Canon SD430, which has been on the market for almost two years now.

I know it takes a while to test things properly, but what is the point in ranking digital cameras that have been out for almost two years?

(continued below)

And how did (my beloved) Nikon D200 score the BEST in image quality, battery life and ease of use – the only DSLR tested to score that well on the only three standards listed – come in fourth place??? In fact, it was the ONLY DSLR (out of 8 tested to score "Excellent" for image quality. The other 7 only managed "Very good." But, it's in 4th place...

Clavestone, you KNOW I know about the on-line review sites, but thanks. I was just putting this out there "for discussion" for the gang. You would HATE the article. Of 8 DSLR's, two were by Pentax — the K10D and K100D. I don't have the heart to tell you how they finished.

Actually, the final score was on a scale of 100. There were only 8 points between the top and bottom cameras.
Canon EOS 30D (74)
Nikon D80 (74)
Sony A100K (73)
Nikon D200 (71)
Canon Rebel XTi (70)
Nikon D40 (70)
Pentax K10D (70)
Pentax K100D (68)

They said that the list is "in overall performance order."

I get a kick out of THIS comment, though:

"Other 10 MP alternative ... the Canon Rebel XTi, the latest in the Rebel line and a fine performer, despite its lack of spot metering." They've been reading my answers! (haha)

Answer 1

Consumer Reports, as with any review type medium, is written by humans. As such they all have feelings towards or against things and some have already made their minds up before they even handle the product. I don't read Consumer Reports mainly because if I follow their guidelines I will more than likely spend alot more than I should. I understand that quality costs but come on, do I need something I dont use day to day to be that high quality, usually the middle of the road will do. On that point if I do use something day to day, my hand tools at work for example, I will have my own views that most likely wont match theirs. Your an accomplished photographer right? So you already can tell just by handling a camera at the store how good it should be.

Trust your gut, after all, you've never seen their gut and who knows what that looks like.

Answer 2

So what are the big issues here?
Testing obsolete or obsolescent cameras. Editorial issue - if they keep doing it, they will lose relevance, readers, then advertisers, advertising revenue and go out of business. Likely? Not really, but you can dream that it might.

Rankings don't make sense. Most of these reviews rely on multiple criteria, each separately assessed and evaluated. Having done all this hard work, they simply sum up the individual criteria weightings as if they are all of equal relative value. They haven't done the really hard work of figuring out which of the criteria is more valued by different camera users and properly weighting the results. I believe professionals will have different values in some areas to those of us ordinary consumers, and this ought to be recognised in the weightings that are used.

You can do your own check on this. Develop your own weighting, with those characteristics you value more given a higher weighting, those you don't care about given less. Recalculate the scores and see if the order changes. Do this again from a different perspective. If you did it the first time from a professional's viewpoint, do it again from, say, a young backpacker travelling overseas, or a family with several potential users, adults and children.

If this doesn't sound like fun, then don't do it, but you clearly know that accepting the view of the journo who wrote the original report without question or deeper understanding is a road to bad decisions about equipment purchases.

Answer 3

Hey Dr. Sam, You run into a few issues with print magazines.

Yes 1 it takes time to test all those cameras, and 2 most of those articles must be submitted 2-3 months before the issue they are printing it goes into press. The layout artisis, proofreaders, ad placements etc must be finalized within the last 30-60 days before printing. Consumer reports have always been 5-8 months behind on their models.

Check out DPReview.com (one of the better review sites that is very up to date (usually within 24hrs of an official introduction) but the full reviews (VERY IN-Depth) can take 2-5 months before complete.

Answer 4

well basically what consumer reports does is a lot like kids experimenting in school. THere is a general product. They think of what the product should and should not do. After this, they write up tests to compare the brands based on the what the product should and should not do.

Answer 5

When Consumer Reports recommends a targeted vehicle, they element in now not most effective their scan outcome, however protection rankings, reliability, and fee. A vehicle with excellent scan outcome could now not make it if it has deficient protection or reliability, or prime fee.

Answer 6

i actually dislike consumer reports because seems like everything that they test to be lopsided and by the time they finish their reports and having everything published seems to be old and not up to date with none of the current models after they had started testing everything.