Since radio
began researching in the late 70s and early 80s we’ve pretty much stuck with
the same 4 systems:
- Focus Groups – Much of this has
been replaced by the ‘listener advisory boards’ that we often draw from
our databases. Cash payments have
been replaced with Pizza and a few free CDs. The sample is small with only 12 people
in the room, but it can still spark new ideas and eye opening revelations.
- Perceptual Studies – Still
popular, but pretty expensive with many still costing 40K and more. Samples have also shrunk over the last
few years and it’s tougher and tougher to find people who will answer the
land line phone and hang around for 20 minutes. We used to find out a lot here, but it’s
getting harder to get beyond the basic questions in 20 minutes. The expense is also a problem in today’s
budgets.
- Auditorium Music Test – The
samples used top be 100 or more in the 80s and now we are lucky to see 85
and in the
U.S.
that usually forces make up tests. The costs are still around 30K. There has been lots of innovation here from using dials instead of
the scanned test sheets, sending out CDs and doing it at home, having them
show up at a research/recruiting office and taking it on their schedule
and recently doing it on the internet from home.
- Call Out – The costs for 26
tests a year (every other week) is around 40-50K and like any land line
phone based test is getting harder and harder to crank out. Some have moved to automated dialing
which has helped keep costs lower, but it’s still pretty expensive and we
always wonder about the sample – how often is it changing? Many have moved to on-line testing like
Rate the Music and Media Score to supplement and the costs are barter or
just a few thousand a year for the service.
The team
here at McVay Media gets to look at plenty of research. We’re there on the design calls with our
clients and we have seen many a power point on the results. Recently we’ve also seen some new innovations
and experiments that are worth considering as you look at your research
options:
- On Line Library Music tests: It’s been done with as many
as 500 songs with the costs being a lot lower (25 or 30% of what you pay
for most AMTs) and the results were
useable. But, there were pitfalls
including:
- Length of the Test – The 200 song test here seems to work better
with less people falling out ½ way through the
test. That does push up the costs
a little so this model is more likely to cost 40-50% of the typical AMT.
- Weighting the results – Sorting a test with so many
respondents and very little control on the age ranges or other sample
characteristics will require some weighting. If you are looking for
a 25-44 sample you will likely have to throw out a number of under
25 year olds and probably weight down the 25-34s as they will probably
respond in greater numbers. 35-44s
will have to be weighted up. It’s
an exercise that you have to do carefully, in most AMTs we screened the sample before it entered the room, on line there is no way to do that.
There are some big advantages:
- Huge sample sizes of 500 or 600 are pretty common in many
markets with these tests and that really helps when you can break out
small cells and still have plenty of people responding. Some have worried that competitors
could infiltrate the sample and steer your music, but with sample sizes
so large it takes lots of people responding in the same way to move the
needle. You can also track the IP
addresses where they are responding from and see if they are coming from
the same office which means you’d have to get a bunch of interns with
home computers all in the right demos and all responding with similar
traits to really mess up a test.
- Quick Turnaround with only a few weeks from posting the test to
having the results.
- The costs are so low that you can do 5 tests a year with 200 songs
in each and go through 1000 titles a year. With the wider playlist trends in nearly all formats you probably need to test more titles and do
it more often.
- Tests are taken at the respondents pace. They can listen to a hook again if they
didn’t recognize it in the first 5 seconds. They can pause if they are tired or
distracted. All these options are
not possible in the AMT and we often see songs at the end of the session
testing worse than the titles at the start.
- Monthly/Weekly On Line Music Tests: We’ve had these services the longest and
many stations test 40 or so songs 2 or 3 times a month to keep up on
currents/recurrents and often break to test a
few library titles. It’s simple to
set up and usually cost just a few thousand a year or can be done for just
a few barter spots a day.
The same pros and cons that show up in the ON-Line Library
test noted above are there, but the sample here requires even more
caution. Since you built the database
from your web site and through promos on the air it’s 100% P1s and probably
lots Super P1s. While you will see less
UNF music and the burn may show up a little quicker you also don’t see what
songs in your list are ones that could potentially pull in audience from the
other stations you compete with. But, at
least we know what our P1s feel and remember that 80% of your listening
probably comes from 20% of your cume. Still, it’s a big factor to consider when
you scan the results.
The costs are so low here why not at least take a look? So many times we’ve worked with no research
at all because of the costs and while it may take some skill and trial and
error to work with this data at least its data. If you haven’t tried this yet you should.
- On – Line Perceptual tests: The experiment we participated in here
did not use a station database for the sample. Instead a large group of email addresses
that were sorted to the age range and market we wanted to research were
purchased and a mass email went out with some rewards offered to those who
completed the test. This way we had
P1s, P2s and even P3s to the main stations we wanted to research and not
just our own P1s. Some of the pros:
- Lower costs and Big sample: For about 1/3rd less than a
normal 400 person perceptual we had well over 600 people and the sample
balance was close to what we would have shot for in a normal perceptual. In today’s budget world perceptual tests
are harder to get and done less often. Here you have the opportunity to do a 400 person test at around ½
the cost.
- More questions: Since you don’t have to ‘explain’ things like
differential scales (from 1 to 10 with 10 being hard rock and 1 being
soft rock where would you rate WXXX), instead we can just show a scale
put hard rock at one end and soft rock at the other and they drag the
station call letters to the spot where they think it falls. We can also play clips and mixes of
music and they can rewind them and play them back again if they get
distracted or want to hear it again. The quality of the audio is also better on most computers than
over the phone.
- Quick Turnaround: The results were ready in 2 weeks from the
final test of the questionnaire.
There are some disadvantages including:
o Sample Balancing: Again you have no control over who
is responding and you will need to weight the results.
o Email Vs Random Telephone: Many will argue about the validity
of using an on-line vs. telephone sample. Arbitron uses the phone and many say we should
stick to that system – after all who cares what non-diary keepers want – this
is about influencing potential diary keepers and we should work in concert with Arbitron’s methods.
We are also getting to a point with Arbitron’s land line telephone methods where the sample return is a real problem. Are we really seeing what the innovators in
our society want or just those who still rely on what is becoming a quickly
outdated communication device? Are we
relying on just the group who will doesn’t screen calls, rely on cell phones,
or has nothing better to do than hang out with us on the phone for 20 minutes
during dinner? These are all areas where
perhaps we can learn some new truths about what the audience really wants. This disadvantage is really a debate and
could become a big advantage in learning more about the audience for less
money.
These new research techniques use the internet to gather the
data. Lots of other industries are
gathering tons of data on their consumers on-line. From watching what they look at on line, what
they buy on line and also asking them for a lot of feedback. It’s a lot less expensive to research
on-line and you can get a lot more data. In many ways we are behind in this trend, but there are experiments out
there and we should all expect to see lots of research innovation in the near
future.
