Tuesday, June 12, 2007

Surveying subscribers

We publish a monthly e-newsletter for which I am responsible (content, creation, distribution, data management, subscriber satisfaction / evaluation). I have done quite a bit to improve on each of the first four aspects and now need to turn my attention to the last. Apparantly, we conducted a survey almost two years ago now so I think that another one is about due. On top of that, I’m interested to know what subscribers think of the changes or whether they have even noticed them. Some of the changes they are not likely to have noticed (such as the changes to the template which ensure that every issue conforms to XHTML, WCAG and CSS guidelines - all links open in new window), some they won’t know about (such as the new software we use to manage subscriber data and e-newsletter distribution) and others they almost certainly will have noticed (such as the new look our e-newsletter has received).

I haven't done a whole lot of thinking on this one yet but I can immediately see three ways forward:
  1. The first is to put together a few questions in an email to which we ask subscribers to reply. This is probably the cheapest option but:
    1. it’s probably the least professional and slick looking to the subscribers
    2. it’s more onerous on the subscribers
    3. and certainly creates a fair old bit of work for me in compiling the responses for analysis.
  2. The second is to outsource the entire thing to a company specialising in this sort of work. This is likely to yield the most professional-looking result for the subscribers and would be the least onerous for me in terms of data compilation and analysis but is almost certainly going to be the most expensive option.
  3. The third option is to use something like surveymonkey.com (opens in new window) which would enable us to create our own online surveys and does the data complilation and some basic analysis for you. It’s the least flexible of the options (you are working within the confines of an OTS product) but it probably offers the best combination of professional appearance, simplicity for subscribers, simplicity for me, and cost.
I have had a go at putting something together use surveymonkey.com (there is a free version available but I think that we would want to use one of the paid subscription versions as the benefits of doing so are in our favour – e.g. limitations on the number of forms submitted are raised or removed depending on which package you opt for) and am happy enough with it.

Has anyone out there done a survey like this before using a tool like surveymonkey.com? What software did you use? Any ‘top tips’ I might use?


deargreenplace said...

Hi David,
We began using an electronic tool for our user surveys last year. I found Zoomerang really easy to use, and was able to set up our test survey using the free trial. My college later bought an educational license and everyone uses it now.

I suppose it depends on the level of analysis that you'll need for the results. Zoomerang does most of the work for you, but the only drawback we found was that you can't match up who answered what if you have freetext answers.

I started by setting up a test on the free trial, and using my colleagues as guinea pigs. The other advantage of Zoomerang was price - it was relatively cheap for an educational license.

David Bruce said...

Thanks for the URL - I'll try it out on the rest of the team...