Wednesday, November 11, 2009

Feedback on the survey

Of the six people that I contacted to provide feedback, four have responded - lucky me :). I will go through the feedback I received and comment on whether or not I changed the survey, how and why section by section.

Page 1 - demographics
Feedback indicated confusion because the groupings had overlapping numbers (age 16-18 or 18-25 for example). I thought I had watched for this but obviously not so - easy fix :)

I accepted the suggestion to change question 7 to, "What branch do you live in?" I had deliberate phrased it initially as "belong to" because I wanted the participants to identify the branch where they participate (and sometimes that isn't connected to where they live) but I then realized that the information I was looking for will be captured in another section where they indicated their level of activity in their "local branch". Changing this question clarifies it and will give richer information when considered with the level of activity questions.

While none of the test subjects :) commented, I am going to change question 5. the information I am trying to gather is regarding whether or not participants in the society tend to have memberships or not and why. Changing question 5 to, "Do you let your membership lapse?" will provide much more meaningful information, especially if they address it in the open ended question that immediately follows.

Page 2 - activities
I was expecting to have a few additional activies suggested. I was suprised there were no comments.

Page 3 - frequency of activity
One test subject suggested that using the term "weekly" in the first sentence may be confusing if the local branch doesn't have weekly activities and suggested merely dropping it. I have decided to leave it in because my understanding that every branch does have some kind of activity weekly (it isn't always the same activity though) and because I am trying capture whether or not the survey respondants are participating on a weekly basis rather than trying to determine if they are participating with their local group. What i will do to clarify is is change the answers to 'weekly, monthy, quarterly, and annually' then add a couple more questions asking about the frequency of participating in other groups. I think that is more likely to provide the information that I was hoping for - though it does make the survey a little longer, I don't think it will be significant.

There were no comments from my test subjects :), but I am going to add one open ended question here. These questions will capture some of the frequency and distances involved in participation, but will not provide any insight into why people participate the way they do. I will add the questions, "What would entice you to travel to participate in the activities of another branch?" and "What would entice you to travel outside your current comfort zone to attend an event?"

Page 4 - opinions
This section prompted the most feedback and I anticipated this. As I mentioned before, these are hot button items for many people and I deliberate presented them in a closed format to force participants to consider the basic level of answer, without conditions. Based on their feedback, I have decided to alter the wording of question 14 to "Avacal should become it's own kingdom." This removes the indication of immediacy which was, I think the basis of the need for conditions and theneed to comment on that particular question. I have also decided to add a box at the bottom of the page allowing participants to make any comments they wish - not connected to a particular question. Responses might be interesting and will only be made by those who feel the strongest. One of my commentors felt that allowing justifications might increase the response rate).

Page 5 - awards
Based on specific feedback, I will change the structure of the questions to, "Identify the level of recognitions that you have received in the ... Check all that apply. peerage, kingdom, principality, baronial, other, none yet" followed by, "If you wish, please describe the recoginition you received". The first question is clearer, I think. while the second may be a little more abiguous, it may provide richer information.

Based on conversations that ensued when reviewing these questions, I have decided that there are more questions to ask. This information will provide a factual snapshot of where people are at, but there is an underlying emotional issue that I would like to get at.  I will add an opinion statement to the previous page, "The award system in the SCA effectively recognizes individual efforts." I will also add the open ended question, "Please describe if and how recognition affects your enjoyment of participating in the SCA."

Page 6 - recruitment
There was no feedback specific to this section and I can see no reason to make changes :)


I will post the link to the survey when I am finished making all the changes so you can see the final product. I will also be sending the link and an invitation to fill out the survey to as many mailing lists within the principality that I can find. The reading material indicates that I should open the survey for three weeks with reminders sent out at one week intervals - I will do that and see how many responses I get. There are roughly 360 members but I have no idea how many are participating.

1 comment:

  1. Last minute response just came in ... similar comments to others but including a suggestion to add a neutral or unsure answer to the opinion questions. That is a deliberate choice on my part. I would like survey participants to come down on one side or the other - I'm sure some simply won't answer since none of the questions are compulsory ...

    Also the suggestion to add a question to the recruitment section asking if the newly introduced people were still involved and to consider asking about volunteering - this may be a whole new section. I'll have to think about it.

    ReplyDelete