Comparison reviews. Why not?

[steve1]

Car magazines are full of comparison reviews - small SUVs one week, executive saloons the next, etc - hi-if mags review speakers, amps and such, camera mags will compare DSLRs. So why don't model railway mags do comparison reviews of models?

 

Yes, they do minor items, like glue or scenic materials, which, although useful, are really small beer.

 

Will any mag do a comparison review of the 3 Class 25 locos that have all appeared on the market, more or less at once? These are all big ticket models and potential purchasers might like to know which is the best one for their money, just like car buyers get from car mags.

 

What about the various DCC systems? All must have plus and minus features, again buyers would benefit from such knowledge.

 

Individual reviews have their place but when there are several options available for the same prototype, it would be helpful to know the relative merits compared to each other.

 

steve

[mike morley]

I love the idea, but any comparison will inevitably have a winner and a loser, and the loser is likely to get miffed and stop advertising in the magazine responsible, causing loss on income.

[Steamport Southport]

I thought they were all different types of Class 25 though?

 

But I have seen comparison reviews previously. Those Model Rail bookazines are just that, whilst they won't be totally dismissive for obvious reasons, they do point out where the models differ.

 

New one just out. Linked at Kernow for ease. Can't find another link.

 

https://www.kernowmodelrailcentre.com/p/70593/Magazine---Britains-Model-Trains-2021

[dibber25]

It's easy to compare different types of car in the same 'class' - such as SUVs, small hatchbacks etc but how does one do that with models? If one compares, say, mixed traffic 4-6-0s the winning LNER 'B1' is no use to someone who doesn't model the LNER/ER. A Southern modeller isn't going to buy a 'Hall' just because a reviewer says its a better model than a 'King Arthur'. A comparison review is unlikely to convince an early 1960s WR modeller to have a Class 37 in preference to a Hymek. The best way to make comparisons is to read the original reviews of the models that you are interested in and make your own judgement. (CJL)

[D9020 Nimbus]

What surprises me is that where there are competing models of the same class of loco, it generally isn't even mentioned in the reviews. And when it comes to performance, the reviews are so vague as to be all but useless. In some reviews, a lot of the content is cut-and-pasted from reviews of other items—Andrew Burnham's reviews of Fourdees locos in RM are a particular case in point.

 

The reviews in British magazines are a lot less detailed and informative than those in Model Railroader or MIBA, for instance. And before you say that British magazines haven't got enough staff, just check out how many editorial staff RM has now. And MRC did better than any of the current magazines with an editorial staff of just @dibber25…

[Legend]

Yes I always find it curious that in most cases reviews never mention the alternative .  The most recent  one being the Heljan 25 , no mention that there' s an SLW one coming along . Or the Bachmann sealed beam 45, no mention of forthcoming Heljan model.  Its almost as if its bad form to mention competition . While most people will be aware of alternatives it is just possible some don't .   Comparisons would then be useful when both models available . 

[RichardT]

3 hours ago, dibber25 said:

It's easy to compare different types of car in the same 'class' - such as SUVs, small hatchbacks etc but how does one do that with models? If one compares, say, mixed traffic 4-6-0s the winning LNER 'B1' is no use to someone who doesn't model the LNER/ER.

To be fair Chris, the original point was about comparison reviews of models of the *same* prototype from different manufacturers, which is a valid issue for consumers.  ISTR comparison reviews in the magazines back in the 1970s and 1980s between the Lima and Hornby "Westerns" and "Kings".

 

Richard T

 

[steve1]

I agree it wouldn't work trying to compare, say, 4-6-0 steam locos, that's apples v oranges. I was thinking, not only of the 25 but Deltics, Classes 37 and 47, there's more than 1 of each of them already on the market or due very soon.

 

Hi-fi mags would do comparisons of a particular type of equipment, like amps, in a price range of X to Y. Surely the same could be done with DCC systems? There's plenty of them available and they all do the same basic job. Where can anyone go to get impartial advice on which system to choose? A comparison review would at least provide a starting point.

 

steve

[Grovenor]

ISTR at least one mag doing a comparitive review whem both Hornby and bachman produced one of the BR Standard steam locos. Was it the Standard 4 4-6-0?

[dibber25]

4 hours ago, RichardT said:

To be fair Chris, the original point was about comparison reviews of models of the *same* prototype from different manufacturers, which is a valid issue for consumers.  ISTR comparison reviews in the magazines back in the 1970s and 1980s between the Lima and Hornby "Westerns" and "Kings".

 

Richard T

 

That's OK if duplication (which I personally dislike) results in two  models of the same loco appearing around the same time, which did happen with one of the BR 4-6-0s if I recall correctly. However, things move fast in the hobby these days and it would not be fair to compare a 2021 model with one which appeared several years ago. (CJL)

Edited April 20, 2021 by dibber25

[Compound2632]

12 hours ago, dibber25 said:

However, things move fast in the hobby these days and it would not be fair to compare a 2021 model with one which appeared several years ago. (CJL)

 

BBC Radio 3's Record Review frequently compares recent recordings with ones from as long ago as the 1960s. OK, a statement such as "the new Helmann Class 47 is a great advance on the old Triang-Hornby model" may be rather redundant, but over a shorter timescale some indication as to whether the new Hornjan model is significantly better than / as good as / a retrograde step compared to the Bachby version of a few years ago (in specified aspects, in the reviewer's opinion) would be helpful to the reader. 

 

20 hours ago, dibber25 said:

It's easy to compare different types of car in the same 'class' - such as SUVs, small hatchbacks etc but how does one do that with models? If one compares, say, mixed traffic 4-6-0s the winning LNER 'B1' is no use to someone who doesn't model the LNER/ER. A Southern modeller isn't going to buy a 'Hall' just because a reviewer says its a better model than a 'King Arthur'. A comparison review is unlikely to convince an early 1960s WR modeller to have a Class 37 in preference to a Hymek. The best way to make comparisons is to read the original reviews of the models that you are interested in and make your own judgement. (CJL)

 

The hobby is more than just locomotives. A comparative review of currently-available RTR models of wood-bodied PO mineral wagons would, I think, be very useful in helping the would-be purchase sort the wheat from the chaff.

[Steamport Southport]

10 minutes ago, Compound2632 said:

 

BBC Radio 3's Record Review frequently compares recent recordings with ones from as long ago as the 1960s. OK, a statement such as "the new Helmann Class 47 is a great advance on the old Triang-Hornby model" may be rather redundant, but over a shorter timescale some indication as to whether the new Hornjan model is significantly better than / as good as / a retrograde step compared to the Bachby version of a few years ago (in specified aspects, in the reviewer's opinion) would be helpful to the reader. 

 

 

The hobby is more than just locomotives. A comparative review of currently-available RTR models of wood-bodied PO mineral wagons would, I think, be very useful in helping the would-be purchase sort the wheat from the chaff.

 

That's all covered in the Britain's Model Trains 2021 bookazine though.

 

You can get them in places like WH Smith, ASDA and the like. As well as model shops.

 

https://railsofsheffield.com/products/43305/model-rail-bmt2021-n-a-britain-s-model-trains-2021?gclid=EAIaIQobChMInIfTh5aP8AIVFOJ3Ch1SnAohEAQYASABEgKDkPD_BwE

 

 

Whilst it might not go into forensic detail of the individual coaches and wagons, it's a good guide.

 

 

Jason

[Phil Parker]

20 hours ago, D9020 Nimbus said:

The reviews in British magazines are a lot less detailed and informative than those in Model Railroader or MIBA, for instance. And before you say that British magazines haven't got enough staff, just check out how many editorial staff RM has now. And MRC did better than any of the current magazines with an editorial staff of just @dibber25…

 

You've waived this stick before - it's worth bearing in mind that the days of a team producing a single magazine where the only contact with the readers is letters to the editor are long gone.

 

Now, we all have to produce material for several channels - paper, website, forum, Facebook, Twitter, Instagram, YouTube - and respond to people who wish to contact us via those same channels, plus e-mail, letters and telephone calls. Don't think you can skimp on any channel either, the devotees of it will shout and scream. There are also exhibitions to attend, both on stands and simply to gather information and find layouts. Some of us also publish 13 magazines a year. 

 

In addition, more modelling takes place in-house than we used to find in the past. I think ModelRail really started this, but everyone now does the same as the demands for photography etc. are higher. More staff doesn't mean anyone is sitting around twiddling their thumbs!

[Phil Parker]

17 hours ago, steve1 said:

There's plenty of them available and they all do the same basic job. Where can anyone go to get impartial advice on which system to choose? A comparison review would at least provide a starting point.

 

Dozens of them. If you visit someone like Digitrains, then they have at least ten linked to their layout. An in-depth look at all of those would fill and issue and take forever to compile. This is where a forum scores as you can find out from people who actually use the things, rather than perform what has to be a quick test, what each is like.

[woodenhead]

It seems to me that for the people who it really matters to, they tend to have the wherewithal to determine which locomotive, wagon or coach they need.

 

If you need help from a comparison site or someone else as to which model to buy then perhaps you've not done enough of your own research, this isn't buying a car where you have several options to choose from.

 

As to whether a Heljan model is a big step forward on a Bachmann version, it really should be down to the individual modeller to use that thing between their ears and make up their own mind. Too many people these days want others to make the choice for them and then when they are disappointed they can blame someone else for the mistake.   At the end of the day, if @Phil Parker says x model is really good and better than y model, it is his opinion based on what he has seen not a statement of absolute fact.

 

If you want someone to tell you to buy lots of Heljan Class 25s and dump all your Bachmann Class 25s, or tell you whether to buy a Heljan, a Bachmann or an SLW 25, you're not going to get that from BRM or any other mass media magazine, you need to find online zealots for that.

 

Me, if I wanted a class 25 right now (actually I do, but in N haha), I would look at what is available both new and secondhand and decide based on what I, not what I have read.

[Phil Parker]

47 minutes ago, woodenhead said:

if @Phil Parker says x model is really good and better than y model, it is his opinion based on what he has seen not a statement of absolute fact.

 

Since that bloke will be more impressed if the model fires missiles and has a giraffe sticking out of the top, treat his opinions with caution!

[Chris M]

On 15/04/2021 at 19:49, mike morley said:

I love the idea, but any comparison will inevitably have a winner and a loser, and the loser is likely to get miffed and stop advertising in the magazine responsible, causing loss on income.

I've always thought it odd that the cars that do the best in car magazine comparison tests usually come from the company that does the most advertising. Surely this must be a coincidence.

[Chris M]

Cars are all pretty good these days and they will all do their job well. My recommendation is - don't read the comparison test just buy the one you like the shape of or that is the best colour.

 

Likewise model railway stuff. Its nice to read a review but if something fits my era/location and it looks good (and what recent loco doesn't look good) I'll buy it. I've ordered a Kato 800 because I know it will look great and run well; there isn't actually a need for any review let alone a comparison review. There probably isn't a real need for a review of any new item of rolling stock these days but I still enjoy reading them and they are a way of letting us know that a product has arrived..

[woodenhead]

38 minutes ago, Chris M said:

There probably isn't a real need for a review of any new item of rolling stock these days but I still enjoy reading them and they are a way of letting us know that a product has arrived..

A review is a lot cheaper for a manufacturer than an advert   and like you say, it's purpose is to let readers know the product is coming just in case they've been on Mars or don't frequent RMWeb.

[Compound2632]

49 minutes ago, woodenhead said:

A review is a lot cheaper for a manufacturer than an advert   and like you say, it's purpose is to let readers know the product is coming just in case they've been on Mars or don't frequent RMWeb.

 

What you are describing isn't a review, in my book, just a puff. A review should examine the model critically: is it an accurate representation of what it purports to be; does it function correctly; etc.?

[D9020 Nimbus]

3 hours ago, Phil Parker said:

 

You've waived this stick before - it's worth bearing in mind that the days of a team producing a single magazine where the only contact with the readers is letters to the editor are long gone

Sorry. I take your point — and I wasn't thinking of BRM when I said it. But whereas Model Railroader once had a significantly larger staff than Railway Modeller, it's now smaller (6 rather than 7— and one of those has the title "Group Technical Editor" so is presumably shared with other magazines) — and they still produce detailed reviews (but perhaps the effort involved is largely up-front and doesn't increase ongoing effort to any extent).

 

Given that BRM's staff is smaller now than in the past — and that you are also involved with other magazines and running this website and World of Railways — I don't see how you can find the time to do all that you do as it is.

[AY Mod]

4 minutes ago, D9020 Nimbus said:

I don't see how you can find the time to do all that you do as it is.

 

Some times are easier than the others and over the last week or so it's been about getting ahead with layout and product articles as much as possible ahead of the next restriction easing so we can hit the road and build up a library again in case we run into restrictions again later in the year. 

 

Anyway on the subject of comparative reviews I suspect some want to see winners and losers rather than meaningful information. To get the latter the individual reviews are just as relevant. Even duplicate models don't arrive at the same time so in doing a comparison review would either mean a delayed individual review or a duplication of a review of a duplicated model.

[Nearholmer]

IMO, a more useful way forward than comparison between models, which is always invidious, and therefore likely to cause ill-feeling, would be comparison against a fixed set of scales.

 

I have in mind things like:

 

- deviation from known key prototype dimensions (which would, incidentally "catch" that old chestnut of diesel locos with the wrong "face", because that is a function of shape, which is measurable);

 

- deviation from key modelling standards (things like wheel profiles, back-to-back etc);

 

- level of detail incorporated (one would need to invent a scale for this, ranging from "every last rivet and oil line" to "items very clearly visible on the real thing from 100m away are omitted";

 

- fidelity of livery applied;

 

- weight;

 

- tractive effort (measured every time by the same repeatable method);

 

- attainable range of speeds (using a defined controller on straight and level track, with "attainable" being defined to include sustainable without visible fluctuation, which is particularly important at low speeds);

 

- current consumption (at defined speed/tractive-effort combinations);

 

- rolling resistance (the converse of tractive effort, for hauled stock);

 

- finish quality (this would again need a defined scale, covering things like fuzzy boundaries between colours, wonky lining, orange peel etc.);

 

- minimum curve radius (with/without added-on bits if appropriate);

 

- operating performance (a very simple, short-ish duration test over a defined circuit representative of a "bad" layout, containing "obstacles" such as dead-frog points, gradients, and curves, not to check performance across the complete spectrum, but simply to give a broad indication of "layout fitness");

 

- RRP.

 

This is not intended as a thought-through set of parameters, merely the sort of things that spring to mind as being important in varying degrees to varying potential buyers, and it omits anything about DCC, sound etc, because they aren't my bag.

 

The best of the reviews in magazines actually get quite close to this, but they are often arranged as "verbiage", rather than a table of results, which doesn't make for quick extraction of data, and brings-in the style/preferences of the writer, which is sometimes entertaining, but not always objective.

 

One interesting point that all this might bring up is how far shy of prototype dimensions many r-t-r "models" actually are, in order that they can run reliably on "model" track; only P4, S7 etc would really get close to "five stars" on this, yet most of us happily turn a blind eye to that, while, equally happily, quibbling about minor discrepancies in hand-rails etc. Funny old bunch we are.

[woodenhead]

12 minutes ago, Nearholmer said:

IMO, a more useful way forward than comparison between models, which is always invidious, and therefore likely to cause ill-feeling, would be comparison against a fixed set of scales.

 

I have in mind things like:

 

- deviation from known key prototype dimensions (which would, incidentally "catch" that old chestnut of diesel locos with the wrong "face", because that is a function of shape, which is measurable);

 

- deviation from key modelling standards (things like wheel profiles, back-to-back etc);

 

- level of detail incorporated (one would need to invent a scale for this, ranging from "every last rivet and oil line" to "items very clearly visible on the real thing from 100m away are omitted";

 

- fidelity of livery applied;

 

- weight;

 

- tractive effort (measured every time by the same repeatable method);

 

- attainable range of speeds (using a defined controller on straight and level track, with "attainable" being defined to include sustainable without visible fluctuation, which is particularly important at low speeds);

 

- current consumption (at defined speed/tractive-effort combinations);

 

- rolling resistance (the converse of tractive effort, for hauled stock);

 

- finish quality (this would again need a defined scale, covering things like fuzzy boundaries between colours, wonky lining, orange peel etc.);

 

- minimum curve radius (with/without added-on bits if appropriate);

 

- operating performance (a very simple, short-ish duration test over a defined circuit representative of a "bad" layout, containing "obstacles" such as dead-frog points, gradients, and curves, not to check performance across the complete spectrum, but simply to give a broad indication of "layout fitness");

 

- RRP.

 

A lot of these measures can be observed whilst watching a Sam's Trains video, but I don't think he is the benchmark we are looking for here.

 

But this makes it clear how hard it will be to determine such a list and have everyone in agreement that it is a good set of measures.

 

 

[Nearholmer]

It also makes clear that creating a thorough, objective review, against clear measures, would be time-consuming, and need a pretty comprehensive reference library. The job would ideally be done by two people: one an expert on the prototype; and, one a ‘techie’ type to make all the careful measurements and objective assessments.

 

Perhaps what is needed is “Sam’s Trains for Grown-Ups” or maybe. “...... for boring old blokes like me”.