Script (with citations)

Script:

Intro Music---

Harris: Good afternoon this is (Wembley weekly) with me Alex Harris

Richardson: And me Alex Richardson on today's show we will be discussing and critically analysing our research methods.

Harris: That’s right so we had to choose a topic to research and find data to spot trends which supported our hypothesis.

Richardson: Yeah, so our hypothesis was 'The home team in the second leg of a champions league tie is more likely to win and go through to the next round than the home team in the first leg'. So, we began researching around the Champions league.

Harris: We used mixed methods to collect data for our hypothesis. We used 2 methods of quantitative research which were content analysis and a questionnaire. We then did a qualitative research method in the form of a focus group.

Using mixed methods is great for research as you can get both quantitative and qualitative data instead of just one. Other than making the data more comprehensive by gaining more of it, the use of mixed methodology allows the weakness of one method to be overcome by another. Despite using mixed methods being slightly more time consuming, the validity of our evidence for a conclusion is more important and we want the data to be as comprehensive as possible (cirt.gcu.edu, 2018).

Did you want to go into a bit more detail about the content analysis and how it was done?

Content analysis section 1:

Richardson: Of course, so the content analysis was the first thing we did when investigating our hypothesis. A quote from (Stokes, 2003) mentions the importance of content analysis, it says ' One of the advantages of content analysis is that it enables you to conduct your primary research and come up with your own facts and figures to use as evidence in your argument'. That can relate to our hypothesis and how we were using content analysis to try and prove its validity with the data we collected.

Content analysis is one of the best ways of getting quantitative data and is especially helpful with finding the hard data, which is what we needed. Some qualitative methods can sway you off track with what you want to find out and can also take up a lot of time. But content analysis allowed easy access to data without any ethical considerations or people to deal with in interviews and focus groups. 

So, we took a sample from the 2016/17 Champions league knockout stages which included the round of 16, quarter finals and semi-finals as these stages all included 2 legged ties. We then put together a coding manual to help us collect data, (Bryman, 2012) stated that 'the coding manual is a statement of instructions to coders that also includes all the possible categories for each dimension being coded'. This table would help us collect the quantitative data we need to justify the hypothesis.

Harris: However, before that it was important for us to do a pilot coding manual first.  Now a report from the (University of Surrey, 2001) says that 'One of the advantages of conducting a pilot study is that it might give advance warning about where the main research project could fail, where research protocols may not be followed, or whether proposed methods or instruments are inappropriate or too complicated'. So, piloting this first was important, so we could get an initial test of our hypothesis to see if it was worth pursuing.

Our pilot coding manual only consisted of one variable which tested how many ties were won by the home team in the first leg or the home team in the second leg (appendix 1). By doing this we found that 9 out of the 14 ties in the knockout stages were won the by the home team in the second leg.

However, the pilot coding manual was not detailed enough and didn’t give us enough in terms of trends. The categories in a coding table should cover all possible outcomes and all possible questions should be answered.

Richardson: Yeah, so we then began to properly put together the coding manual and add more variables to it. 5 more variables were added to the manual which covered all the outcomes and questions we wanted (appendix 2). This gave it more detail and gave us opportunities to spot trends in the data.

Questionnaire section 1

Harris: That’s right and to gain a further source of quantitative data we wanted to use either a questionnaire or a structured interview. We went for the questionnaire for a number of reasons. Firstly, we wanted to save time in order to complete at least 3 research methods for the task. As (Seale, 2008) explains, self-completion questionnaires have a number of ‘distinct advantages’ over interviews; Questionnaires are cheap to administer, whilst also allowing for a greater geographical coverage without taking the time or cost that travelling to an interview would incur. As well as this it would also reduce the chance of questionnaires being filled in bias as there wouldn’t be an interviewer present meaning it would also provide anonymity for our respondents. By using questionnaires, this is giving us an extra source of quantifiable data to back up our hypothesis.

Richardson: Seale also writes that “response rates in self-completion surveys tend to be maximised when respondents have an interest in the subject of the research and are therefore motivated to complete the questionnaire (Seale, 2008).” As our hypothesis surrounds the UEFA Champions League it was important for us to distribute our questionnaire to areas where our sample would be educated about the subject and more specifically, fans of teams that have contested in the knockout stages of Champions League tournaments in recent years.

Now if you look at Appendix 3 of our blog this shows the questionnaire we designed to send out to as many respondents as possible. Of our 66 respondents, 18% of them answer the first question with ‘other’. This means they don’t support one of the teams already given and therefore means their team has not competed in the tournament in recent years. By ruling out the ‘other’ answer option available, it would considerably reduce our number of respondents.

Harris: By using ‘non-probability sampling’ for our questionnaires we were identifying that only fans of English teams in the champions league was our sample. By using ‘purposive sampling’, meant that going into our research we had a prior purpose. Alvi writes that “we do not include everyone who is available to us rather those available are included who meet the defined criteria (M. H. Alvi, 2016).” Overall I’d say, we were happy with the range of fans that had answered our survey but with a high percentage choosing ‘other’, means that you cannot clarify the validity of some responses. And of course, by using mixed methodology meant that for our focus group we could pick a more ‘convenient sample’ of fans, that we knew were fans of the teams mentioned in question one of the survey. By using convenient sampling, it can be subject to sampling biases (M. H. Alvi, 2016) but we needed to eliminate the issue of the ‘other’ category from the questionnaire.

In terms of our ethical process, we included both anonymity and informed consent in our questionnaire. For our questionnaire, we did not include a consent form as we did not know the exact people that would be taking part in our research, however, by returning the filled in questionnaire, the participant is giving their consent in taking part (Brookes.ac.uk, 2018). We also reassured our participants of anonymity by not requiring your name to respond to the survey. Realistically, the questions asked in our questionnaire didn’t cover a particularly sensitive subject but without that reassurance, respondents can be more likely to hold back their honest feelings.

Focus Group section:

Richardson: With our mixed methods approach for our research we did a pilot focus group to help gather qualitative data. We needed to try and get the right people on our focus group who had some sort of understanding in our topic. (Bryman, 2012) talks about purposive sampling, he says' the goal of purposive sampling is to sample cases/participants in a strategic way so that those sampled are relevant to the questions that are posed'. With this in mind we chose a sample of people that supported teams which had been in the Champions league for the past 3 years. This was done purposefully to try and cover as many teams as possible and get a variety of responses from people who may have different points of view.

So, we were able to gather 8 participants who all supported champions league teams for the pilot focus group. By using fans of English teams that competed in the champions league, we were able to eliminate the weaknesses of our questionnaire by targeting a more informed sample. However, the teams of the fans in our focus groups are rival teams and in some cases at different levels. As a result, you can experience a difference in opinions from your sample that can escalate due to team rivalry. Furthermore, fans may have different opinions through experiencing different situations in the champions league.

Like Man city Monaco last year, because City went out on away goals last year despite scoring 6 over two legs, city fans may be more inclined to show distaste towards the competition than say Leicester fans for example who you could say exceeded their expectations, making it to the quarter finals.

We also prepared questions beforehand (appendix 4) so that we could maintain structure to the discussion and also help bring the discussion back on track if it went off topic. However, it was important for us to try and step back and let the discussion build so we could get people's feelings on certain things.  (appendix 1).

Harris: yeah, so after the pilot focus group was done we realised that the discussion did end up going off topic at some points. The participants were very knowledgeable in the topic and gave some interesting insights on the away goals rule and other things. Here we have a clip from the focus group talking about replacing the away goal rule.

SOUND BITE 26 SECONDS

That was a fair point made and it was something that we had not thought of before and that’s something that’s really great about focus groups in that it can draw up bigger arguments and make further suggestions. However, if you look back to the questionnaire we sent out, both 47 and 56% respectively believe that the away goals rule helps to find a winner in a tie as well as adding drama. Why do you think that is?

Harris: It’s a weird one, obviously we have the point that the 8 people that took part in our focus group is a bit of a low number to act as a representative of English champions league fans as a whole. As a result, it is hard to draw a conclusion due to the differing opinion of whether the away goal rule is good for the champions league. But, due to the stronger response on the questionnaire of fans approving of the away goal rule and the fact that it has in a way been proven to work, that fans on the whole don’t mind it.

And if you look at question 2 of our questionnaire it's clear most of our respondents ages range between 18 and 64. This was important for us as those between the ages of 18 and 64 are more likely to be educated on the champions league and the process of the knockout stages. And looking at question 3, from a fan perspective, shows that fans do prefer to play at home second. I think there really is a stigma against playing at home in the first leg, with 67% saying they do.

Yeah it is very interesting, I should mention that we did discuss the possibility of carrying out unstructured interviews however we thought that focus groups would be much more beneficial for us. One of the reasons is that it's a much faster method of collecting qualitative data as we were able to hear from 8 people in under 20 minutes. Compare this to a 1 on 1 interview which could take much longer if you have to ask them the same questions individually. There is also the added possibility of one person bringing up a point to which someone else in the group, with further knowledge, can maybe expand upon to give a deeper insight. And this wouldn't be possible with a single person in a unstructured interview.

Content analysis section 2:

Richardson: So moving on now, from what we gathered with our content analysis we could see that the home side in the second leg of a tie did indeed have an advantage over the other side.

Looking at appendix 2a we found that 64% of teams that were at home in the second leg went on to win the tie.

Appendix 2c showed that 42% of home teams in the second leg won the tie after scoring 2 or more away goals.

Also, appendix 2d saw 28% of teams at home in the first leg lose the tie on aggregate to the team at home in the second leg.

Well in a way that proves it. When you go away in the first leg of the champions league and you come away with a result of any kind, win lose or draw, you know then don’t you what roughly will need to be done to qualify and how hard that will be. Spurs for example drawing 2-2 in Juventus knew that if they could just nick a 1-0 or 1-1 that they would go through.

Exactly so these variables did support our hypothesis, however one problem with our coding manual was that some of the variables tested did not support it. Appendix 2b, 2e and 2f did not give us any clear trends for the two categories.

Also, it could be argued that one of our variables were subjective. So, the variable was 'teams of similar ability that won the tie' (appendix 2e) now this was down to us to determine whether the teams facing each other were of similar ability. And at the end of the day it is our opinion, but some people may disagree. Now (Bryman, 2016) said that 'coders must draw upon their everyday knowledge as participants in a common culture in order to be able to code the material with which they are confronted'. This means that depending on the person coding you may get different sets data and it is impossible to know which is more reliable as it is just a matter of opinion.

And again, with the ethical factors of this method. The ESRC state that informed consent should be provided in a ‘comprehensible form’ (ESRC Framework for research ethics, 2015). Whilst time was allowed for participants to decide whether they wanted to proceed, we were also verbally open about why they were chosen for our sample and a rough line of questioning. As opposed to telling them what we were asking, we felt it was better to tell them what we wouldn’t ask so they were aware we wouldn’t be touching on any sensitive subjects. As well as this, you can hear from the focus group that at not one point are any names mentioned. Ethical considerations for focus groups are the same as for most other methods of social research (Homan 1991). All our participants were given full anonymity and confidentiality that no one except the researchers and moderators would know who or what was conducted in this study.

Content analysis conclusion:

To conclude our research methods we just want to talk about what went well, the bad things that happened and what we would do next time to improve.

Harris: Yeah, so let's start with what went well first, so the pilot coding manual was very important to the research we done. This was because we found that our hypothesis was valid as the coding showed that 9 out of the 14 teams who were at home second in the UEFA Champions League won their tie. Finding this out meant that we could begin to add more variables to the table to try and spot trends that could further back up our hypothesis.

On to the negatives now, after reflecting on our coding manual we realised that we could have added more variables or added different variations of one variable. One example is appendix 2d which was 'Won the first game but lost the tie on aggregate score' we could have adapted it into other variables such as 'Lost the first game but won the tie on aggregate score' or 'Drew the first game but lost the tie on aggregate'. Without us exhausting every possibility we may have missed out on data and trends we could use to prove our hypothesis. When doing this in the future we will have to make sure we cover every possible angle on our variables to ensure we get the best quality data possible.

Harris: As mentioned earlier in the show one of our variables was seen to be subjective. Just to remind you the variable was 'teams of similar ability that won the tie' (appendix 2c) and this really came to down to a matter of opinion from the coder. This means that different coders can input different data based on their opinion, so it can't be seen as too reliable. When doing this again in the future we will try to avoid subjective variables so that our data can be more reliable.

Questionnaire conclusion:

Richardson: In terms of our questionnaire, we believe that our questionnaire had a number of positives and negatives. There is no doubt we stayed on topic with our questioning, and whilst trying to recover data that is easy to quantify, we also asked questions that would easily prove our hypothesis. The average time for respondents to complete our questionnaire was 1 min. Our only questionnaire which was a pilot, received 66 responses in the first three hours. However, although it was quick and easy to complete, I think had we have had more time or repeated the task in the future, we could ask deeper questions in order for us to notice further trends. Whilst it’s important to remember a long questionnaire will lose the respondents interest (Bryman, 2012) we felt that after our pilot focus group that there could’ve been more questions to ask, had we completed the focus group before the questionnaire.

Harris: Yeah, the reason for this being that as a focus group is qualitative and open-ended, our sample for our pilot focus group brought forward further deeper questions that could’ve been asked in the questionnaire. For example, we could’ve added a question suggesting other methods of finding a winner other than the away goals rule by seeing how respondents feel with the choice of away goals, extra time and golden goal.

Harris: Also, a main advantage of using a pilot study is the fact that It can greatly reduce the number of unanticipated problems because you have an opportunity to redesign parts of your study (Woken, 2013). The sample we chose for the questionnaire was distributed on our personal social media accounts meaning we were unintentionally limiting the variety of respondents as people on our social media accounts tend to support the same football teams as ourselves.

Richardson: Despite this however, what you can conclude from our questionnaire is that it was definitely a broader and therefore more representative sample of English champions league fans. And based on results in the questionnaire, fans believe that the away goals rule is a good form of sudden death if needed in the champions leagues knockout stage and there is a clear deduction that there is a preference of playing at home second over two legs.

Focus group conclusion:

Richardson: Moving onto the focus group, one thing I think that was good is the range of fans we had. Our 8-man sample made up of man utd, Liverpool, Chelsea and arsenal fans meant it was likely to receive a range of opinions. However, one issue with focus groups is that when you have a short amount of time, they become time consuming. After conducting the focus group, we felt it would be more suitable in future to use a smaller group of perhaps 6 people and do two focus groups.

Harris: Absolutely, and not only would that evidently increase our sample for the focus group but it would make it easier for everyone to have a say. What we found in our focus group is that because there were so many people for a small room, there were cases of people being spoken over when trying to have their opinion heard. Here’s a short clip from our focus group so you can hear for yourself.

SOUND BITE 23 SECONDS

Richardson: Now, whilst it is our job as the conductors of the research to keep everyone calm we feel that next time we could use a smaller group and also conducting the group with more strangers. With participants being from the same university degree it meant that our sample were already familiar with each other and felt comfortable to shout over their friends. But at the same time you want your sample to feel comfortable while the research is undertaken, and there is clear evidence of this. Next time, by using people that have never met would give more people an opportunity to speak individually, with no interruption.

Additionally, our sample lacked a variety of ages and gender. By including more women but also older football fans can offer more valid and educated responses than our sample of an average age of 19 years old. Women may view men’s football in a different way to men themselves and fans that are older would have more experience than someone of a younger age. We felt one positive that we utilised in the focus group was that we didn’t inform our sample the specific questions surround the champions league.

Harris: Yeah, so by not giving the sample pre-prepared questions we felt that their responses were more likely to be honest as they had no time to prepare an answer and in turn, eliminate any possibility of bias by giving them time to calculate an answer. In addition to that, although the last 5 minutes of our focus group goes slightly off topic, we felt that this was an important feature of qualitative research as you want to delve into the why’s of something and by being as open as possible with your questioning can lead to other arguments and suggestions being brought to the floor.

Overall process conclusion:

Throughout the process we have used mixed methods research, combining both quantitative and qualitative data. We have used both methods of data to gain completeness by building more comprehensive research and creating an offset in which the weaknesses of one method will be strengthened by another, as seen in our focus group by using a more ‘convenient’ sample (Bryman, 2006).

Richardson: To end the show we want to talk about how our research has proven our hypothesis to be true. Just to remind you our hypothesis was 'The home team in the second leg of a champions league tie is more likely to win and go through to the next round than the home team in the first leg.'

Our findings with content analysis and the coding manual helped us to see that 64% of teams that were at home in the second leg went on to win the tie. This data kind of speaks for itself really, it gives solid evidence for our hypothesis.

Harris: Exactly, there is also something else we found in relation to our questionnaire and focus group that could bring up further discussion.

48% of respondents on our questionnaire were aged 25 to 64 compared to the 46% aged 18 to 24. This meant we had a majority of older respondents and results from the questionnaire told us that people enjoyed the current system used in the Champions League.

Richardson: Yeah, so looking at the participants in the focus group it was clear that a lot of them wanted a change in the system used. Things like golden goal and replacing away goals with the standard extra time and penalties were bought up. Also, our participants were aged between 19 and 20 so it was clear to see a difference in opinion.

Harris: With this in mind it could suggest that there is a generational difference with what people think about the current Champions League system. What this has shown though is that through combining the use of the quantitative questionnaire and qualitative focus group we spotted a trend that we didn’t earlier suspect. And it wouldn’t have arose had we had not used both methods.

Richardson: Exactly, I think that we have to do some more research on this in the future to see if there are bigger questions that need answering.

Harris: Yeah for sure, anyway we have come to the end of the show, thanks for joining us, we will be back next week.

Reference List:

Bryman, A. (2012). Social Research Methods. 4th ed. New York: Oxford, p.299.

Bryman, A. (2016). Social Research Methods. 5th ed. New York: Oxford, p.408.

Stokes, J. (2003). How to do Media & Cultural Studies. New Delhi: Sage Publications, p.56.

University of Surrey (2001). Social Research Update. Guildford: Department of Sociology, p.1.

Cirt.gcu.edu. (2018). When to Use Mixed Methods - Center for Innovation in Research and Teaching. [online] Available at: https://cirt.gcu.edu/research/developmentresources/research_ready/mixed_methods/when_to_use [Accessed 18 Mar. 2018].

Alvi, M. (2016). A Manual for Selecting Sampling Techniques in Research. Munich Personal RePEc Archive, [online] 70218. Available at: https://mpra.ub.uni-muenchen.de/70218/1/MPRA_paper_70218.pdf [Accessed 18 Mar. 2018].

Seale, C. (2018). Researching society and culture. London: SAGE, pp.182-183.

Bryman, A. (2012). Social Research Methods. 4th ed. New York: Oxford, p.235.

ESRC Framework for research ethics. (2015). [online] p.29. Available at: http://www.esrc.ac.uk/files/funding/guidance-for-applicants/esrc-framework-for-research-ethics-2015/ [Accessed 22 Mar. 2018].

Brookes.ac.uk. (2018). Guidelines for informed consent - Oxford Brookes University. [online] Available at: https://www.brookes.ac.uk/Research/Research-ethics/Review/Guidelines-for-informed-consent/ [Accessed 22 Mar. 2018].

Woken, M. (2013). Advantages of a pilot study. Center for Teaching and Learning, University of Illinois, [online] 7. Available at: https://www.uis.edu/ctl/wp-content/uploads/sites/76/2013/03/ctlths7.pdf [Accessed 19 Mar. 2018].

Bryman, A. (2012). Social Research Methods. 4th ed. New York: Oxford, p.633.

Homan R (1991) Ethics in Social Research. Harlow: Longman.