Are reviews of new models independent or not?

[Compound2632]

1 minute ago, AY Mod said:

How much more explicit should that point be?

 

It does raise the question of how a reviewer should handle a matter of factual error. Here Tony has presented it, to my mind, in a rather tentative way, as his opinion. Could he have cited the HMRS livery register in support of his point?

[AY Mod]

5 minutes ago, Compound2632 said:

Could he have cited the HMRS livery register in support of his point

 

Possibly but there are also time and deadline considerations and information may not be immediately to hand; his view was in that case entirely correct. I did have an extended three-way debate with one manufacturer over the shade of blue in a commission; the view being that shade of blue wasn't wrong and that it was subjective. It's just it didn't match any other shade of blue on anyone's else's model (including their own). Some manufacturers will admit to an issue; some won't.

 

So what did I say in print?

 

Quote

Colour can be a subjective issue at times and I find that the shade of BR blue applied to this model is a touch darker than some previous xxx incarnations. We had an extensive three-way discussion about this shade and ended up with three opinions and no conclusion, so I’m sure you'll have your own, too!

 

[Mike_Walker]

The diplomatic corps loss is the hobby's gain!

[Peter Kazmierczak]

Many moons ago I used to write book reviews for the RCTS. 

 

I imagine writing an objective review about a model is harder than one about a book. A book is largely about the content (and maybe a little about the quality of images) whereas for a model there are many facets to consider - all the prototype stuff, plus the how it's engineered, the quality, detailing, performance etc, etc

[MarkSG]

11 minutes ago, Compound2632 said:

 

It does raise the question of how a reviewer should handle a matter of factual error. Here Tony has presented it, to my mind, in a rather tentative way, as his opinion. Could he have cited the HMRS livery register in support of his point?

 

Historical liveries, prior to the advent of Pantone and other definitive colour match systems, will always be at least partly subjective. Photographs don't necessarily render the colour accurately - just look at two different modern photos of the same train - and descriptions are always open to interpretation. Colours printed on paper, or displayed on a website, don't look the same as theoretically identical colours painted on wood or steel. Plus, of course, the way a colour looked fresh from the paintshop isn't the same as it looked after a few months in the sun and rain, and in any case different paintshops - or even the same paintshop at different times - didn't necessarily use precisely the same formula. So there would have been quite a lot of real life variation in livery colours, and accurately modelling those colours is always going to be something of a "best effort" process.

 

To be honest, I don't actually mind that historical colours on models aren't necessarily spot on for that prototype. I don't know what the exact shade was, and nobody else does either, so nobody can say for certain that something is wrong or right. I've got four pristine wagons in BR grey sitting on my layout at the moment, from Oxford, Rapido, Hornby and an older one from Mainline. They're all subtly, but visibly, different shades of grey. Add a kitbuilt and hand-painted van and we're up to five shades of grey. If I keep buying wagons I might eventually get to fifty shades! I don't know which one is more "right" than the others. I suspect none of them are. I have a feeling that the Hornby one is a tad too light, and I'm pretty sure that the hand-painted van is too dark. But the other three are, to my eye, equally plausible variants on the same theme. And I have a feeling that, in real life, those three wagons would have been subtly different as well. If anyone was to be critical of one for being the wrong shade of grey, then that would be purely their own subjective opinion.

 

[Miss Prism]

Two aspects I feel are pertinent in today's landscape:

 

- models are often 'reviewed to death' (sorry for the phrase, but you know what I mean) on, e.g. RMweb, before they become available;

- by the time a model review appears in the printed press, the model is often sold out.

 

[BMacdermott]

4 minutes ago, MarkSG said:

 

Photographs don't necessarily render the colour accurately - just look at two different modern photos of the same train -

 

Hello Mark

 

Exactly. I have a photo (somewhere) of an S&D 7F 2-8-0 in Prussian Blue where the colour on the tender is 'affected' by the coldness of the water. It looks like two different shades of blue between water tank and coal space!

 

Brian

[Compound2632]

10 minutes ago, MarkSG said:

 

Historical liveries, prior to the advent of Pantone and other definitive colour match systems, will always be at least partly subjective.

 

In my opinion, there is considerable scope for nuanced disagreement here...

 

But let's not let the thread drift that way.

[Tony Wright]

What an interesting thread........

 

I can only really comment on my own experiences with regard to reviews.

 

I have had a manufacturer threaten to pull advertising if my review of his product had been printed. It wasn't, but the business went down without 'my help'. Central to this is that he saw the review first. It's something I always do - that is show a review to the manufacturer beforehand, which gives them a right to reply, if not a right to have it altered. It also serves to prevent me from making 'stupid' errors. The manufacturer also gets a complete set of pictures which he/she can use without charge, and the use of any videos I've made of it running, on a test bed or on Little Bytham - apparently the moving footage of Hornby's P2 I shot has been one of the most-watched model railway videos on You Tube, though the workings of You Tube (is that how it's written?) are a total mystery to me. 

 

For many years (less so these days) a loco or rolling stock kit would arrive on my doorstep from all the main manufacturers with a request to please build it, photograph the process, write a review and get it published. Obviously, time-sensitive items like this 'jumped any queue' and my reviews were published in all the mainstream mags (until 2003, when I became full-time on BRM). The models were sent FOC (what use is a Southern 'W' 2-6-4T to me), but in case anyone thinks this smacks of bribery and corruption, the manufacturer got to see the review first, was given a right to reply and also got a complete set of pictures for his/her use. In many cases, I'll have assisted the manufacturer by way of providing prototype material, test-building the first kit and written the instructions. Any input of mine would be known in the review, of course. I even suggested that if anyone couldn't understand my instructions for DJH kits, then the firm give the questioner my phone number. So far, nobody has called! 

 

Regarding my input, if I've helped an RTR manufacturer with a project, I never write a subsequent review of the finished model. This would be unethical.

 

As far as book reviews are concerned, I only now review any which are of use to me. Books are also sent FOC, presumably the publishers feeling that a good review is well worth the cover price. Only on rare occasions have I returned a book, because to review it would have been a virtual 'hatchet job'. I'm delighted by the thanks I get from publishers, though only one publisher now refuses to have my reviewing his books. In the past, any books I've reviewed which I'm not particularly interested in (no matter how good they might be) I've donated to clubs or sold for charity. If I've helped with a book (in a small way), I always express an interest in my review of it. 

 

A problem for potential reviews (and I'm trying to abstract myself here) is the level of 'expertise' needed when assessing a product or a book. For instance, I've just reviewed a new book which features the LNER, and at least three locations and direction of travel have been mis-identified, along with a couple of incorrect dates. What would be case If I'd been asked to review, say, a book featuring the GWR? 

 

For any would-be, say, kit reviewers out there, may I suggest (to really help editors) that three absolutely essential tasks are carried out? These are........

 

1. Build the model in quick time to a 'high standard', and, if it's a locomotive, make sure it works (perfectly) - preferably by subsequently running it at shows for all to see.

 

2. Write an article which (at least) shows a command of English (preferably to further-education standards); great, if you have a degree. 

 

3. Photograph the process to a professional standard. 

 

Of the three, 3. is the most-important, 1. is second and 2. can be sorted out by a decent sub-editor. 

 

 

Edited January 5, 2023 by Tony Wright
typo error

[Talltim]

2 hours ago, melmerby said:

I'm the same.

Wading through a 30 minute YT review to find the relevant bits can be extremely tedious, skipping through the almost inevital 'unboxing' and various other pointless comments to find the 5 mins of the actual model being tested.

 

(It doesn't just apply to model reviews but to most "how to do" videos, where three lines of text would suffice.

I hate videos for this. Lots of IT people seem to like them, but I find I've switched off and missed the bit where they've skipped over the information I needed to know and I then have to rewind 3 or four times to find out thats its not there anyway.

[Peter Kazmierczak]

I occasionally put up a review on Amazon of a book I've bought - no freebies for me, I'm afraid...

What does niggle though, is that even after pointing out clear factual errors in a publication, a subsequent "2nd revised edition" still contains the very same mistakes. Does nobody ever proof read books nowadays?

[Phil Parker]

2 minutes ago, Peter Kazmierczak said:

What does niggle though, is that even after pointing out clear factual errors in a publication, a subsequent "2nd revised edition" still contains the very same mistakes. Does nobody ever proof read books nowadays?

 

Yes, but someone has to check your information is correct, then tweak the book before it heads off to the printers. I doubt any railway book is anything other than a passion project for the author. Very few people make money doing this stuff. Ultimatly, it's down to price, and perfection costs money.

[melmerby]

22 minutes ago, Peter Kazmierczak said:

What does niggle though, is that even after pointing out clear factual errors in a publication, a subsequent "2nd revised edition" still contains the very same mistakes. Does nobody ever proof read books nowadays?

I did that with a music chart book.

There were several glaring errors. Artists missing from the information. Incorrect information, page/data formatting errors etc. (data was in columns and some were in the wrong column)

There was a card in the book asking for reviews and corrections etc.

I duly filled it in and sent it off, with the mistakes I had found.

A second edition came out several years later (With chart entries for more years?) but with none of the previous errors corrected.

[MarkSG]

40 minutes ago, Peter Kazmierczak said:

I occasionally put up a review on Amazon of a book I've bought - no freebies for me, I'm afraid...

What does niggle though, is that even after pointing out clear factual errors in a publication, a subsequent "2nd revised edition" still contains the very same mistakes. Does nobody ever proof read books nowadays?

 

Proof reading and fact checking are two completely unrelated things. The publishers assume, for a factual book, that the author knows what they are talking about. The editor will do their best to ensure that the book is free from obvious spelling and grammar errors, and make suggestions to the author for revisions which will aid readability. But it's not their job to do the author's research work for them.

 

Plus, of course, even facts can be a matter of genuine debate, particularly for historical matters where the documentary evidence is incomplete or ambiguous and where a new author can actually come up with a plausible new interpretation of what evidence there is.

 

Of course, if it's fiction, then facts don't matter anyway.

[Peter Kazmierczak]

When I worked in a library we'd often get authors in doing research. Sometimes you'd get a mention in their acknowledgements or even a free book. A Japanese professor gave me his book; unfortunately not about railways and in Japanese...

Once a chap came in to research some obscure railway fact; he was most surprised when I said I knew of his books (he was/is a respected author on the LMS). I brought in my collection of his works the following day (he said he'd be in the next day too) and he duly signed them for me.

 

But back to the original question.

Yes, I feel that most model reviews are independent, as honesty works both ways. Long gone are the days of the less than impartial reviews of PECO products in the RM, and some of us still recall the halcyon days of those detailed reviews in the MRC.

 

Edited January 5, 2023 by Peter Kazmierczak
Spelling and grammar..

[younGGuns7]

I have my favourite manufacturers and models so I am pretty sure that I would give a bias view but I am not a real professional, when I read some posts and see the knowledge I am gobsmacked so I feel it’s needs to be viewed with an open mind to any review. 

[Willie Whizz]

2 hours ago, Tony Wright said:

…

 

For any would-be, say, kit reviewers out there, may I suggest (to really help editors) that three absolutely essential tasks are carried out. These are........

 

1. Build the model in quick time to a 'high standard', and, if it's a locomotive, make sure it works (perfectly) - preferably by subsequently running it at shows for all to see.

…

 

 

I would agree with almost all your excellent post, Tony, but I believe this point does require an observation. 
 

In my view, a model kit submitted for review should be built substantially in the manner and sequence described in the supplied instructions.  Depending on how "complete" or otherwise the kit is meant to be, clearly some 'intervention' by the builder may be required, but he/she should stay as close as practicable to what the manufacturer provided and intended; and should not simply 'do things how they always do', nor undertake extensive substitution of parts - which is not, then, a test of what is going to be supplied to purchasers.
 

Otherwise it risks becoming, not a review of the product “as supplied”, but of the Reviewer’s own skills as they see them, and a demonstration of how to ‘upgrade’ the product (which, let's be clear, is a perfectly valid thing in its own right as a subsequent matter; but it isn’t then really a ‘review’ of the new product, is it?).  
 

Thankfully that sort of thing is far less common than it used to be, but there are still sometimes echoes. 

Edited January 5, 2023 by Willie Whizz
Brevity

[eldomtom2]

Genuine question, has any reviewer noted that the KR Models Fell has glaring inaccuracies?

[drduncan]

6 hours ago, Phil Parker said:

 

Then you are in the pocket of "Big Transport Publishing", that multi-billion pound industry that controls the world of train books with an iron fist. You probably review them on a yacht while drinking copious G&T's 😆

 

(in truth, anyone who reviews books deserves a medal. It's a lot of work for very little reward)

I’ve done a fair bit of book reviewing within my own professional academic rather than hobby niche and the only reward is getting to keep the book!

 

The one time I wrote a truly damming review the author complained to the book reviews editor of the journal. The editor was kind enough to send me a copy of his reply to the author, where he pointed out that he read all the books before choosing who on his list of wiling volunteers to send it to and that as a result was able to agree with every word I’d written…

Duncan

[PaulRhB]

6 hours ago, AY Mod said:

Occasionally someone at a manufacturer wants my head on a spike;

 

With apologies to Hanna Barbera!

 

 

I think the same applies to reviews as any other research, use multiple sources and make your mind up from that. In that way you get to know which to trust. I like the mag reviews for the better quality pics than most online reviews and what they don’t say is what I focus on. If some aspect is really good then they mention it, if they avoid it then I look for info on that aspect as that’s where I perceive they may be being kind. Still a useful and valid part of researching if a model is worth it and I find many other online reviews are far more likely to focus on one aspect and provide little background. The comments on prototypes here often allow me to know what compromises have been made in advance and decide if that bothers me. 
I don’t have issues with any of the major magazines reviewing and I feel they offer a good general appraisal. 

[Ravenser]

6 hours ago, jjb1970 said:

There are two extremes of the reviewing spectrum - the shill review and the outright hatchet job. Neither is positive and thankfully I think both are rare in model railway magazines (for what it's worth I think it's not that common in YT model reviews either). However, I do think Rail Express crossed the line at the hatchet job end of the spectrum at times, there was one infamous editorial where anyone buying models they didn't approve of was said to have blood on their hands. Which I found full on bonkers.

 

There was a little more to "Blood on their hands". At that point there were two rival OO models of the 37 being announced. One was Bachmann's Mk3 Class 37. It was widely whispered that a member of the REM team had been an advisor for the rival model (though I never heard any hard evidence either way...)

 

6 hours ago, newbryford said:

 

Didn't Bachmann pull advertising from Rail Express (or Rail - can't remember the date) after a review of the early model of the 37?

 

That was the Mk1 Bachmann Class 37 and it would have been REM. Whether Bachmann's action was specifically against the review, or whether it was simply the straw that broke the camel's back is another matter. There was a sense at the time that the review was simply the latest incident in a sustained campaign against Bachmann and its products by a number of people, originally entirely online . However there was a sense that with REM reviewing , one or two of the online critics had now found a print outlet

 

(It wasn't just Bachmann either. I remember one year where there were 6-7 major releases in the couple of months around Warley and leading up to Christmas . And every single one went down in a hail of bullets from the "electricnose" site... These were new models to a standard we had never seen before, recieving a level of hostile criticism and invective never previously seen in the hobby)

 

So by the time of "blood on their hands" there was a strong perception of long term bad blood between at least one manufacturer and several reviewers (online and in print). Potentially, vested interests could give rise to hatchet jobs as well as shill reviews , and there was a suspicion of that hovering over the whole business

 

It was in that context that the whole trope about magazine reviews being in the pocket of the manufacturers, and of magazine reviews glossing over blatent faults to protect their advertising revenues first arose. The online critics did present themselves as being independant and objective reviewers and it was implied online that those who gave more favourable views of models did so because they were compromised. It seems this one has never quite been expunged since then.

 

When people pointed to the reviews in REM as one of the major stengths of the magazine , the implicatuion was that REM offered the only honest rigourous and independent reviews in town. I am not convinced that such a claim was valid.

 

I do feel that some D+E reviewing in that era had lapsed into a kind of "spectator blood-sport" , and I don't think that was a healthy development . Personally when I buy a magazine I want to read about someone's model-making or layout , or deepen my understanding of what the prototype was about. I am not much interested in pages of stuff about last month's new releases , especially when they are just existing models in new paintwork.

 

(And I have to admit that my views about manufacturer bashing , manufacturer X taking down manufacturer Y's model, and the fight for "the definitive Class 37" were forged in those particular fires over about 8-`10 years)

 

It's worth considering the sequel. 

 

- The Bachmann Mk3 Class 37 remained in production until last year. I think that's over a decade of steady sales

- But its rival didn't exactly crash and burn. It remained on sale in quite a few model shops for 5 years or so , until the manufacturer seems to have lost interest

 

Edited January 5, 2023 by Ravenser
The manufacturer may have lost interest, but I lost the last word of this

[The Stationmaster]

5 hours ago, Peter Kazmierczak said:

I occasionally put up a review on Amazon of a book I've bought - no freebies for me, I'm afraid...

What does niggle though, is that even after pointing out clear factual errors in a publication, a subsequent "2nd revised edition" still contains the very same mistakes. Does nobody ever proof read books nowadays?

Going off at a slight tangent but there is something of a parallel with models.  A proof reader can be doing two things - checking spelling, terminology used, grammar,  and so on.  But also potentially checking for historical and subject related accuracy.  To do the first the proof reader needs a good knowledge of english - assuming of course that the book is written in english. Tthat can in some respects be compared with reviewing how a model works - not an exact parallel I would agree but  a sort of similarity

 

 To do the second checking task the proof reader needs a very good knowledge of the subject - here is probably the closest parallel with checking/reviewing the detail realism of a model. Iif you write a book or article  you need a 'technical' proof reader, or editor, who knows as much about the subject, or more, than the you.  I suspect that for many subjects not only are such folk thin on the ground but they - or more likely the production schedule - simply won't allow the time for such checking.  And, especially with time pressures it is easy to fall into a trap when you think you know more than you do and don't bother to check what you have written or are checking - mea culpa.   Potentially, and sometimes actually, no different in process and time pressure from reviewing a model or possibly even something that can't be covered in the amount of space the editor has allowed for the review.

[The Johnster]

There seem to be two separate issues here, the accuracy of reviews and the honesty of reviews.  As far as accuracy goes, a review that it going to go into the public domain be it online or in print will inevitably be fact checked as nobody wants to be made to look an idiot in public, but this does not mean that 100% accuracy is guaranteed; people are human and make inevitable genuine mistakes, and also rely on commonly accepted authoritative material that may itself be less than perfect.  There is always a grey area in which claimed facts may well be assumptions or opinions that have been repeated over long periods. 

 

As an example of the sort of thing I mean, it is often stated that the popularity of 0-6-2T locomotives in South Wales is because the locos run more slowly up the valley and faster coming back down it, and because they run smokebox first up the valley (in order to ensure that the fusible plug was under water), the trailing wheel aided faster bunker first running.  This is, if you like, the accepted received wisdom.  It might have some credence in passenger work, where the speeds were higher coming downhill, but since line speeds were usually lower than 50mph over most of the Valleys network that seems unlikely, and the coal trains were subject to 'incline working' methods, in which handbrake levers were pinned down at the top of gradients so that the loco pulled the train down the bank under power against the brakes at low speed in order to control it (most of the time, anyway).  So the trailing wheel (in reality there to provide support for and distribute the weight of enough coal in the bunker to allow a standard 6-coupled tank engine to manage a full day's work without returning to the shed, perhaps 2 or 3 trips to a colliery, most of it pulling hard at low speeds up hill and down) was of very limited if any use in controlling the ride of the engine coming fast down hills; nobody came fast down hills if they could help it in the Valleys if they wanted to collect their pensions!  The accepted percieved wisdom is, um, what's the word I'm looking for, oh, yes, that's right, wrong.  But it is a generally held fact all the same, and people genuinely and unquestioningly accept it.

 

Honesty is something else.  I am of the opinion that this is by and large a hobby, and a trade, of gentlemen, and that reviews are honest and not biased towards or away from any manufacturer.  But I can understand why some people are suspicious; here are these people, quite big fish in our small pond, whose opinions are respected and well regarded, getting free models from manufacturers in order for their magazines to make money out of by reviewing them, and, assuming the review is favourable, generating sales for the manufacturer.  It looks like something that has the potential to be a cosy, and corrupt, scratch my back I'll scratch yours arrangement.  It isn't, and it would probably be fairly rapidy apparent if it was, but one has to accept that some people had their dummies taken away from them when they were babies and have never trusted anyone since...

 

Now, most people who know me will assert that I'm somewhat of a cynic, but I can't see any evidence for this sort of thing, either in the hobby or in the trade.  We have occasional throwings of toys out of the pram, several of which seem to involve Hornby, and there are sometimes people who hold particularly stringent views of certain other people and their businesses or magazines, though having been praised by a Prime Minister he's flown his private aircraft to China now. 

 

Back in the 70s, I used to regularly buy Hi-Fi magazines, which of course reviewed new equipment as it came on to the market.  The product reviews had to be read by someone who had worked out the coded language; nothing was ever openly or clearly criticised, even mildly (and there was some rubbish about in those days).  You had to interpret stuff like 'anybody in the market for a mid-priced bookshelf speaker would do well to consider this product'.  What does that mean?  Consider it and buy something better for less money?  Consider it and buy it, it's brilliant?  We haven't really listened to it, but it is a mid-priced bookshelf speaker?.  I generally favoured the first option.  Of course, you got reviews that praised products for their performance, or value for money, or both, and these seemd uncoded enough.  But my suspicion was that the magazine reviewers were being circumspect in their negative opinions for fear of upsetting the applecart that both their employers and the manufacturers were pushing.  We are lucky to have the manufacturers, magazines, websites, dealers, and fellow hobbyists that we have, mostly gentlemen!

 

But there were also comparison reviews; '10 budget belt-drive record decks compared' and such.  These were far more informative, pointing out the strengths and compartive weaknesses of each item and estimating a top 10 based on percieved value for money.  Not sure how this could translate to model railways; 'top 5 7-plank minerals compared in this month's issue' (Oxford wins vfm, hands down,).  '37s go head to head' (actually, this might be worth doing!), that sort of thing.  Levels of detail versus RRP is a big consideration in our hobby, and determining which of several very similar models is the best value is subjective, while investigating which is the best detailed is just a matter of inspecting the model.

[Peter Kazmierczak]

And, of course, nowadays everyone has their opinion over something and can easily post it on fora such as this. In ye olden days, there was just the once-a-month magazine fix, to satisfy our needs.

[Willie Whizz]

11 minutes ago, The Stationmaster said:

 

…

 

 To do the second checking task the proof reader needs a very good knowledge of the subject - here is probably the closest parallel with checking/reviewing the detail realism of a model. Iif you write a book or article  you need a 'technical' proof reader, or editor, who knows as much about the subject, or more, than the you.  I suspect that for many subjects not only are such folk thin on the ground but they - or more likely the production schedule - simply won't allow the time for such checking.  
 

…

I agree. And I think, sometimes, people who know little of the world of publishing (but think it must all work the way they would do it “if I were King”) believe it does, or should, work like the world of Academia, where there are always half a dozen other “experts” available to peer-review a new paper fairly and impartially, and time and money and marketability are not going to be an issue. 
 

My own, somewhat limited, experience in the matters suggests that outside Academia you can appeal all you want for those last few nuggets of information you need to complete your masterwork and commonly get no response; yet when you finally say “that’ll just have to do”, and pass it to a publisher, and it gets into print, all of a sudden the ‘experts’ will come out of the woodwork and be all over you like a rash.