GnosticBrian Posted September 18, 2019 Share Posted September 18, 2019 I've just completed the BRM survey. Question: "How many shows do you attend" Answer: "None" Question: "What is your reason for attending shows - four options offered none of which is 'I don't attend shows'" Answer: I had to tick one of the four options; NONE of which apply to me. Utility of surveys where questions don't cover the range of reasonable responses? 1 2 Link to post Share on other sites More sharing options...
RMweb Gold gwrrob Posted September 18, 2019 RMweb Gold Share Posted September 18, 2019 Yes, I said I'd been modelling for over 15 years even though I'm still a teenager.... 5 Link to post Share on other sites More sharing options...
RMweb Gold Enterprisingwestern Posted September 18, 2019 RMweb Gold Share Posted September 18, 2019 When surveyed, 62% said they didn't trust them, 29% said they did and the other 14% weren't bothered one way or the other. Mike. 2 8 Link to post Share on other sites More sharing options...
RMweb Premium rab Posted September 19, 2019 RMweb Premium Share Posted September 19, 2019 I've recently experienced this from the other side. I'm part of a team which is trying to do some community work, and we thought it would be useful to find out what people thought were the strengths and weaknesses of their community. We came up with what we thought was a suitable questionare, but when we went through the responses we found some where people had totally misunderstood the questions. We thought we'd worded it in a way to get the info we wanted, but someone else looking at it read it differently. Having said all that we were a bunch of amateurs 'having a go'. Businesses should have access to the right sources to be able to put together meaningful surveys. 1 Link to post Share on other sites More sharing options...
RMweb Premium TheQ Posted September 19, 2019 RMweb Premium Share Posted September 19, 2019 As far in company surveys are concerned, i've normally found they are rigged to provide the answer they are looking for... 5 Link to post Share on other sites More sharing options...
RMweb Gold ian Posted September 19, 2019 RMweb Gold Share Posted September 19, 2019 5 Link to post Share on other sites More sharing options...
RMweb Gold Oldddudders Posted September 19, 2019 RMweb Gold Share Posted September 19, 2019 I believe there are companies that provide a service compiling questionnaires to help your business. Obviously there is competition among them to win your custom. It is my observation that the winning bid invariably includes the most questions. Which means I seldom complete one. 3 1 Link to post Share on other sites More sharing options...
RMweb Gold Huw Griffiths Posted September 19, 2019 RMweb Gold Share Posted September 19, 2019 20 hours ago, GnosticBrian said: I've just completed the BRM survey. Question: "How many shows do you attend" Answer: "None" Question: "What is your reason for attending shows - four options offered none of which is 'I don't attend shows'" Answer: I had to tick one of the four options; NONE of which apply to me. Utility of surveys where questions don't cover the range of reasonable responses? This reminds me of an exercise a few years back, in which a company were trying to gauge potential viewer reactions to a TV science series. Before and after seeing a pilot show, people were asked some multiple choice questions about their preferences in presenters - some of whom a number hadn't heard of before. Unfortunately, the only answer options available were on a scale from "amazing" to "diabolical" - with no "who's that?" option. 6 hours ago, TheQ said: As far in company surveys are concerned, i've normally found they are rigged to provide the answer they are looking for... I've encountered enough of those - including one in which an employer went through the motions of "consulting" their staff about a change in working conditions, which would prevent staff with families from taking annual leave during school holidays. For some reason, this idea wasn't too popular with the staff - but management were intent on imposing this change, whilst pretending that this might somehow be by popular demand - so they came back with another survey, with no "status quo" option and the other options calculated to be even less popular. Strangely enough, they didn't get many completed surveys back ... . I believe there's a whole discipline devoted to this sort of thing - in which people are invited to choose between an officially approved option (which might not always be too popular with people given a free choice) and some "nuclear option" (which has been calculated and worded to be unpopular). It's called "choice architecture" - and it appears to be favoured by some politicians - with well known examples including a certain referendum a few years back, for which both the referendum itself and what's happened since have acquired some notoriety. This is neither the time nor the place to repeat arguments either way (and I continue to actively avoid expressing my own view in public). However, I think it's fair to say that a number of people on both sides of this divide now seem to view the whole exercise as a textbook example of what can go wrong with surveys and the like, which haven't been adequately thought through beforehand. Huw. 4 Link to post Share on other sites More sharing options...
RMweb Gold Stubby47 Posted September 19, 2019 RMweb Gold Share Posted September 19, 2019 I hope you've also replied direct to BRM to express your dissatisfaction with their survey questions. I often point out such stupidity in surveys at work - eventually it does get through ( not least that I'm now asked to proof read stuff ) 3 1 Link to post Share on other sites More sharing options...
RMweb Premium Keith Addenbrooke Posted September 19, 2019 RMweb Premium Share Posted September 19, 2019 Prize looks nice though. 1 Link to post Share on other sites More sharing options...
RMweb Gold ruggedpeak Posted September 19, 2019 RMweb Gold Share Posted September 19, 2019 On 18/09/2019 at 18:55, GnosticBrian said: I've just completed the BRM survey. Question: "How many shows do you attend" Answer: "None" Question: "What is your reason for attending shows - four options offered none of which is 'I don't attend shows'" Answer: I had to tick one of the four options; NONE of which apply to me. Utility of surveys where questions don't cover the range of reasonable responses? Joined 2015, first post above. 1 Link to post Share on other sites More sharing options...
F-UnitMad Posted September 19, 2019 Share Posted September 19, 2019 What BRM Survey, anyway?? 1 Link to post Share on other sites More sharing options...
Steamport Southport Posted September 19, 2019 Share Posted September 19, 2019 4 minutes ago, F-UnitMad said: What BRM Survey, anyway?? The one we all got sent. Unless you opt out of emails. Check your inbox. Jason Link to post Share on other sites More sharing options...
F-UnitMad Posted September 19, 2019 Share Posted September 19, 2019 14 minutes ago, Steamport Southport said: The one we all got sent. Unless you opt out of emails. Check your inbox. Jason Ah, gotcha. Thanks. Link to post Share on other sites More sharing options...
RMweb Gold Huw Griffiths Posted September 19, 2019 RMweb Gold Share Posted September 19, 2019 4 hours ago, F-UnitMad said: What BRM Survey, anyway?? 4 hours ago, Steamport Southport said: The one we all got sent. Unless you opt out of emails. Check your inbox. 4 hours ago, F-UnitMad said: Ah, gotcha. Thanks. I'd go along with this assessment. In my case, I haven't recently had much time to check stuff like this. Now that I know where to look, I've completed it myself. In my case, the question I take issue with is the one about annual spend on the hobby - with the lowest option being "£1500" - and all the rest significantly more than this. I'm assuming that this one was actually meant to read along the lines of "< £1500" - in which case I can actually answer it. (I've never had that much available to spend on any hobby - and I doubt if I'm alone.) Otherwise, I've got no issues with this survey - and I can't see too many people objecting to the prize for one, very lucky, person who completes the survey. Huw. 1 Link to post Share on other sites More sharing options...
F-UnitMad Posted September 21, 2019 Share Posted September 21, 2019 On 20/09/2019 at 00:35, Huw Griffiths said: I'd go along with this assessment. In my case, I haven't recently had much time to check stuff like this. Now that I know where to look, I've completed it myself. Exactly. The email title was along the lines of "Do you want to win a Western Pullman?". No, I don't, actually, so I hadn't opened the email, so I didn't know there was a survey in it. I do like to mess with surveys though - modelling mostly outside the "4mm British" Mainstream, these things can be quite entertaining, & if by a million-to-one chance I win the Western Pullman, it'll provide useful funds for models I do want. 4 Link to post Share on other sites More sharing options...
RMweb Gold TheSignalEngineer Posted September 21, 2019 RMweb Gold Share Posted September 21, 2019 18 minutes ago, F-UnitMad said: if by a million-to-one chance I win the Western Pullman, it'll provide useful funds for models I do want. My thoughts exactly. However one thing can lead to another I gained a weathered 'Super D' through winning a competition when I was designing a West Country layout. That layout never got built but I've now got a collection of 40 locos which appeared in the West Midlands in the late 1950s. 2 2 Link to post Share on other sites More sharing options...
Darth Vader Posted September 21, 2019 Share Posted September 21, 2019 Typical of survey producers, they don't get the end user to try out the survey to find out if there are bugs or things that don't work. If you put out a survey that is flawed then the results will be meaningless. 2 Link to post Share on other sites More sharing options...
Dungrange Posted September 21, 2019 Share Posted September 21, 2019 The answer to reliability also depends on the sample size. If you get ten completed surveys, it's difficult to draw meaningful conclusions: get ten thousand and it should be easier to draw meaningful conclusions. That said, if questions are flawed, then it doesn't really matter how many people complete the survey. 2 Link to post Share on other sites More sharing options...
RMweb Gold john new Posted September 21, 2019 RMweb Gold Share Posted September 21, 2019 (edited) On 20/09/2019 at 00:35, Huw Griffiths said: I'd go along with this assessment. In my case, I haven't recently had much time to check stuff like this. Now that I know where to look, I've completed it myself. In my case, the question I take issue with is the one about annual spend on the hobby - with the lowest option being "£1500" - and all the rest significantly more than this. I'm assuming that this one was actually meant to read along the lines of "< £1500" - in which case I can actually answer it. (I've never had that much available to spend on any hobby - and I doubt if I'm alone.) Otherwise, I've got no issues with this survey - and I can't see too many people objecting to the prize for one, very lucky, person who completes the survey. Huw. There were several where both, all or none/other would be correct but were missing. DC or DCC was one IIRC, I am currently trialling DCC (a yes tick) but most locomotives are still analogue (a yes tick for that). Also fully agree regarding the less than £1,500 question. A key question I thought was missing, not gender, but do you model solo, with family or in a club/group setting. Edited September 21, 2019 by john new 1 2 Link to post Share on other sites More sharing options...
Steamport Southport Posted September 21, 2019 Share Posted September 21, 2019 1 hour ago, john new said: There were several where both, all or none/other would be correct but were missing. DC or DCC was one IIRC, I am currently trialling DCC (a yes tick) but most locomotives are still analogue (a yes tick for that). Also fully agree regarding the less than £1,500 question. A key question I thought was missing, not gender, but do you model solo, with family or in a club/group setting. "Do you belong to a club?" was in there ISTR. As for the £1500 question. Far more I'm afraid. Apart from drinking and music, it's the only hobby I have, so everything spare goes on things like trains, books, heritage railways. etc. I took it to mean the whole hobby. Jason 1 1 Link to post Share on other sites More sharing options...
RMweb Gold john new Posted September 21, 2019 RMweb Gold Share Posted September 21, 2019 8 minutes ago, Steamport Southport said: "Do you belong to a club?" was in there ISTR. As for the £1500 question. Far more I'm afraid. Apart from drinking and music, it's the only hobby I have, so everything spare goes on things like trains, books, heritage railways. etc. I took it to mean the whole hobby. Jason Club question - I'd forgotten that one then. apologies for the error. Link to post Share on other sites More sharing options...
RMweb Gold The Stationmaster Posted September 22, 2019 RMweb Gold Share Posted September 22, 2019 I shot for the moon in the question about forums when it came to 'other' where I asked for Adult Level discussion on modelling and the prototype. (although I should have said 'consistently' of course ) 1 Link to post Share on other sites More sharing options...
RMweb Premium Alex TM Posted September 22, 2019 RMweb Premium Share Posted September 22, 2019 Hi everyone, I try to avoid surveys where at all possible. My reasons are two-fold: I used to work for an 'organisation' that could make survey results say almost anything that was wanted; and In my current path of life I have been asked to complete surveys that range in function from time-wasting to voyeuristic! Regards, Alex. Link to post Share on other sites More sharing options...
RMweb Premium PeterKem Posted October 5, 2019 RMweb Premium Share Posted October 5, 2019 It would appear that for the Location: Other "Spain" is not allowed. So my views not going in then! Peter Link to post Share on other sites More sharing options...
Recommended Posts
Create an account or sign in to comment
You need to be a member in order to leave a comment
Create an account
Sign up for a new account in our community. It's easy!
Register a new accountSign in
Already have an account? Sign in here.
Sign In Now