‘On the inside’: Research in partnership with a client reference group

This article presents the participation of a client reference group in a qualitative research study which explored clients’ experiences of counselling and natural therapies services in a women’s health centre. The article focuses on the development of working relationships between the reference group and researchers using a capacity building approach which facilitated a two-way exchange of skills, knowledge and experience. This ensured that the views of clients and community members were represented in the research design, thereby increasing its rigour and accountability and fostering social inclusion. Members’ reflections on their journey and the changes they experienced as outcomes of the research process are presented. Ethical issues in working with the reference group are explored. Some members experienced vicarious traumatisation as a result of their exposure to data on domestic violence. This analysis of the use of a reference group and its beneficial impact on research outcomes will be of interest to other researchers looking to work with a client reference or advisory group.

There is a growing expectation by funding bodies that human services organisations will draw on the experience of consumers to inform planning and service delivery through service consultations, evaluations and the use of advisory committees (Commonwealth of Australia, 2010; Lewis & Harkin, 2004;Lane, 2001). In research, this expectation is not yet universally established, even though it can be seen as a logical extension of organisational practice in the welfare and health sectors (Lammers & Happell, 2004). A reference group is "the group to which services and providers … must refer if they are to identify accurately what the group's needs are and what are the best solutions … the group who must finally … decide … whether the services … 'got it right' and their needs are met" (Wadsworth 1991, cited in Lammers & Happell, 2004). White also advocates for consumer participation in research as crucial to increasing the accountability of researchers to their target populations and in ensuring research has "valued outcomes" from the consumer perspective (2002, p. 439). Berends and Johnston, however, point to the complexity of adding people from diverse backgrounds and levels of experience to the research team (2005). Gregory (2008) argues that consumer representation in the health sector is not well understood. She identifies a lack of literature on consumer participation in the Australian context, particularly at organisational and policy levels (Gregory 2008(Gregory , 2007a(Gregory , 2007b. The role of a consumer representative is to provide a client perspective, which will differ from the views of service providers, academics or professionals (Consumers Health Forum of Australia, 2004). "Consumer" can be understood as "anyone who is accessing or has accessed a particular health service or programme, or who may use [that service] in the future" (Lee, Thompson & Amorin-Woods, 2009, p. 64). This definition is seen as interchangeable with the term "client". Members described themselves as clients and community members.
The reference group formed at the Centre for Women's Health as the Feedback and Research Group (FARG) using clients who were already volunteers. The aim was to achieve greater client participation in service planning, delivery and evaluation. Individual services at the Centre include the Domestic Violence Counselling Service and the Mid to Older Women's Health Service which provides counselling and natural therapies. The involvement of clients as members of a reference group requires sharing decision making and power between the organisation, researchers and clients. While it is a given that researchers have greater formal power through professional knowledge and employment, clients are recognised as having power through their local knowledge and unique experience as service users (Ife, 1997). Client engagement in research allows for significant influence as advisors over a period of time (Bastian cited in Gregory, 2007b;Barber, Boote & Cooper, 2007), and involves a transformation from being the subjects of research to partners and co-researchers valued for their expertise (McKevitt, Fudge and Wolfe, 2009). The FARG engaged in research for a two year period.
At the time the reference group was formed in 2005, there was not a body of literature available to inform the process of developing a research reference group (Lammers and Happell, 2004). There are now many publications to be found on the use of consumer reference groups to guide mental health research (for example McCann, Baird, Clark & Lu, 2008, MacDonald et al., 2007Lammers et al., 2004), on their use in drug treatment service consultations (for example Brener, Resnick, Ellard, Treloar & Bryant, 2009;Bryant, Saxton, Madden, Bath & Robinson, 2008), and their use in health research (Barber, Boote & Cooper, 2007;Ward et al., 2009). Some literature in the form of reports on consumer engagement and guidelines for consumer representatives is to be found on the websites of government departments and non government organisations (for example Victorian Department of Health Services; Consumers' Health Forum of Australia).
There is still little published on the use of reference groups to advise on research in the community services sector and in particular women's health services. The specific focus of the reference group on research differs from the purpose and characteristics of most health consumer groups (Baggott, Allsop & Jones, 2005). For example, undertaking research was seen as a low priority by the majority of consumer organisations surveyed in the UK (Baggott et al., 2005).
This article provides a case study of developing and working with a feedback and reference group (FARG) for a research study which examined client experiences of individual counselling and natural therapies services provided by the Benevolent Society's Centre for Women's Health, located in the South West of Sydney. McKevitt et al. (2009) have argued the need for empirical accounts of consumer involvement in research in order to establish an evidence base, which should include how research is influenced by consumer partnerships. The study was conducted in partnership with a researcher from the University of Canberra for the two year period from 2005 to 2006. The inclusion of a reference group was a natural extension of the philosophy of empowerment and community participation in decision making which guides the service in which this study was undertaken, and the researcher's professional identity as a social worker. Clients' right to participate in decisions that affect their lives is an essential philosophy of the Centre for Women's Health, and ensures that services are effective and relevant to the needs of clients. The role of participants in the reference group was not to represent all the clients who accessed the Centre, but rather to contribute to the research from their own experience as clients and community members.
In this article, data gathered to evaluate the reference group experience and outcomes will be presented, including reference group members' views of the difference participation in the project made in their lives. Some of the challenges of working in partnership with a reference group will be illustrated. Gregory (2007a) notes that the ability of organisations to evaluate consumer engagement develops with experience.
The research study had a strong capacity building component and philosophy. The reference group was trained in research processes and skills such as focus group facilitation and qualitative data analysis during the study. Gregory (2007), in a consultation study on consumer engagement, found that appropriate training and support is vital for consumer representatives to be effective in their role. The reference group not only learnt new skills but also developed a knowledge framework about research that could be applied in other contexts. Ward et al. (2007, p. 64) describe the benefits of consumer participation in research as increased "relevance, credibility, dissemination and transferability" of findings through the contribution of lay knowledge. Berends and Johnston (2005) view the use of a diverse team to analyse data, including consumer representatives, as an important mechanism for increasing the rigour of qualitative research processes (2005). The value of including a reference group in the design was found to be substantial for the organisation, researchers and reference group members.

Method
A client and community reference group has much to offer research, particularly research focusing on consumers' experiences and perspectives. The Feedback and Reference Group was made up of women who had previously accessed the Mid to Older Women's Health service and the Domestic Violence service. Members of the reference group had first hand knowledge of the services being evaluated in the research and mid to older women's health and domestic violence. Most members had previously taken up various volunteer roles at the Centre, including co-facilitating groups, producing the Centre newsletter Egeria, fundraising and membership of committees. All but one of the members were over forty-five years of age, and their contribution to the study was enhanced by their considerable life experience, their motivation to learn and contribute, and their non judgmental attitudes, insight and understanding. The group demonstrated a strong desire to assist in service improvement to benefit future clients.
Approval was obtained from the University of Canberra Human Ethics Committee to conduct a service evaluation study at the Centre in 2005, and included the use and evaluation of a consumer reference group. The reference group participated in the design of the data collection methods for the service evaluation. Survey questions focusing on change were developed in collaboration with the reference group and the Centre's practitioners: "Has coming to counselling made any difference to you?"; "Can you describe this difference?" and "What do you think happened to make this difference?" The reference group designed posters to advertise the survey and advised on placing survey return boxes strategically in different places around the Centre so that clients could feel confident the return of surveys was anonymous. The reference group then collaborated on developing focus group topics based on the findings of the survey data analysis.
The focus group and interviews were co-facilitated by the researcher employed by the Centre and one co-researcher from the reference group. Clients were able to explain their experiences of accessing individual services at the Centre to a peer as well as the researcher. The power imbalances between research participants and the research investigators were reduced by the co-facilitation of the focus groups by a member of the reference group and a researcher (McKevitt et al., 2009). For example, the co-researcher described her role as "staying on track, keeping the group together, valuing their input and letting them know we valued it." The use of a cofacilitator from the reference group enhanced the data collection process, as previous research has found that research participants feel more at ease in talking about their experiences of services with a peer who has had similar experiences (McKevitt et al., 2009). The co-researcher reflected at the end of the study on how "stuff that came up [in the focus group] was upsetting especially domestic violence … I realized the value of it though the heaviness which gave it real impact." NVivo, a software program which facilitates the identification and coding of themes, emergent categories, and relationships between codes, was used to analyse the data. NVivo is based on grounded theory methodology in which explanations and theories emerge from the data analysis rather than the researchers' theoretical assumptions (Barbour, 2001). NVivo allows for the systematic use of qualitative data analysis techniques such as coding and data retrieval (Bazeley, 2007;Richards & Richards, 1991). Once the researchers had completed the initial coding of the data using NVivo to identify categories of clients' service experiences and relationships between codes, this was presented to the reference group who contributed their views to the coding and interpretation.
The reference group then participated in the development of evaluation forms for the individual services which were based on the categories identified in the data analysis of what the participants valued in individual services, and found effective and useful.

Participatory Action Research
Throughout the research study, participatory action research (PAR) methods (Chambers & Spriggs, 2006;Ottmann, Laragy & Damonze, 2009) were used to partner with members of the reference group, build transferable research capacity, record the reference group's participation in decision making and capture their experience of reference group membership. PAR methods included the researcher recording and distributing public minutes of reference group meetings and documenting group reflections on the process and outcomes (Chambers & Springs, 2006). Towards the end of the study, the experiences of the FARG group members were documented in an evaluation session conducted by the external researcher. A final evaluation in which members submitted written feedback was also conducted to elicit their views on the difference their participation in the reference group had made in their lives. Group members understood writing about their experiences as contributing to the writing of this article, not only as data.
Dialogue which shared knowledge about research and the client experience of the Centre helped to equalise the working relationships between reference group members and researchers. Members developed a greater understanding of the research process and expressed their own views about the research. Researchers developed an in depth understanding of clients' experience of the Centre. Methods of participatory democracy were used in reference group meetings, including building consensus through group discussion, group decision making, and "wordsmithing" (Merry, 2003) about the language used for data collection, analysis, writing up of the findings and the development of new evaluation tools for the Centre. The perspectives of reference group members provided an important counterbalance to the professional assumptions that the researchers brought to the research design and implementation. The researchers and the reference group developed shared understandings of the contribution made by group members through the collaborative approach used during the study and a reflective process on the completion of the study. How this unfolded will be described in the next section on the process of collaboration between the researchers and the reference group.

Partnership with the reference group
This section will present on how the reference group contributed to the research process and outcomes. Members' sensibilities towards other women's experiences, including the trauma of domestic violence, was a major asset the reference group brought to the research study. As members had previously participated in group programs at the Centre, they had already honed their skills in sensitivity and confidentiality. Past experiences of "not having a voice" helped to reinforce the importance of research which listens to women who have been disempowered.
In the initial stage of forming the reference group, its precise role and tasks were unclear and in development. While the research and ethics proposals included the concept of working collaboratively with a consumer reference group to inform the research process, there was no prescribed plan or list of tasks that the group should undertake. Some members conceived of the group as an "opportunity to bring up things needed throughout the Centre, such as a screen door for safety purposes" and others "didn't have much idea what was required of us" (FARG evaluation). On reflection, beginning with some level of uncertainty around roles and grappling with what the reference group's role should be ultimately resulted in the consumer contribution being more meaningful than was originally conceived.
While good preparation was crucial for successful group meetings, flexibility and immediate adaptability were also essential. The main element that had been planned, meeting times of two hours once a month, proved unworkable, with extra meetings needed as the study progressed to allow meaningful consultation about the data analysis. Flexibility with resources, time and ways of consulting with members proved necessary for more meaningful participation of the reference group as the study progressed.
There was an awareness of the need to shift the group's role from being the subjects of research to active facilitators of research and meaningful decision makers. This approach represents a feminist view of research where equalising power relations is central to the research process (Stanley & Wise, 1993). To develop the group's skills in research, a capacity building stage was built into the process. The areas in which members undertook training included focus group facilitation, ethics including confidentiality and sensitivity in research, the Centre's aims and outcomes, evaluation and data analysis, including a demonstration of using NVivo to analyse qualitative data. Skills in designing survey and evaluation questions and interviewing were also developed.
However, in some ways, the most significant 'training' occurred when members recognised their own skills and contributions to the research. One member said: "I lacked confidence in myself and didn't think I would have much to contribute" (FARG evaluation). Creating an informal environment, asking many questions about members' own experiences and listening and recording discussions in meeting minutes all helped to build confidence and were appreciated by members. One member wrote: With patience and support from [the group facilitator] we realised that we did indeed have some worthy information to contribute.
The training in focus group interviewing also helped group members to recognise and consolidate the experience they already possessed as participants or volunteer facilitators at the Centre. The need for facilitators to create a comfortable and confidential environment and manage group dynamics was recognised. These skills were utilised in the co-facilitation of the research focus group and interviews.
An important staging post in the process was the attendance of a reference group member at the 2005 Australian Women's Health conference in Melbourne to present a segment of the research paper. The reference group experience was presented: Our group has been busy designing posters that we display throughout the Centre… and preparing survey forms …. This project has a real purpose and we feel useful and important …. We are learning to facilitate focus groups, something that is quite BIG for us volunteers. We are used to having a paid staff worker facilitate groups and for us 'little' people to have this opportunity is … quite a big step. It makes me feel important and able and is quite a confidence builder. (Lewis, Lopez and Beattie, 2005) Some tasks such as analysing qualitative data proved to be unrealistic or impracticable for the group, given the time consuming nature of coding and analysing text. The reference group's role in regards to data analysis developed into understanding the process and commenting on the coding schema, codes and categories as they emerged. A difference emerged between the researchers' use of professional language for codes, and the reference group members' preference for in vivo codes which maintained the use of research participants' language and descriptions. A discussion about this difference led to a greater sensitivity to the language and descriptions used by research participants.
In reading, discussing and analysing the data, great sensitivity was demonstrated by the group. Sometimes members could draw comparisons from their own lives and where they could not they showed empathic understanding of other women's experiences.
A final outcome of this research was the development of evaluation forms for counselling and natural therapies based on what clients valued about the service as identified in the qualitative data analysis. Members drew on their first hand experience of the services and previous evaluation forms used by the Centre, to help design and refine user-friendly evaluation questions. As far as possible, the language and descriptions of clients participating in the research was used in developing questions. Gregory has found an outcome of successful consumer engagement is that organisations adopt the "language and definitions of consumers and draw on consumers' ways of categorising the world" (2007, p. 6). Consumer ownership of this process is demonstrated by one member communicating the reference group's experience and the research study's outcomes to a wider range of consumers through the Centre's newsletter, Egeria, in 2006. While plain English research reports could be sent to the focus group and interview participants, survey participants remained anonymous. The reference group decided to publish on the research outcomes in the newsletter to reach clients and community members who might have been survey respondents or would be interested in the findings. Broad dissemination of research outcomes kept the researchers more accountable to their target population and is recommended by the National Health and Medical Research Council (Commonwealth of Australia, 2004).

Ethical issues for research reference groups
The final evaluation with the reference group demonstrated the deep impact on reference group members of listening to and reading the traumatic stories provided by domestic violence counselling clients. Research participants located the changes in their lives which came about through participation in counselling within their trauma histories. These had a deep impact on members during the data collection and data analysis stages. The co-researcher who co-facilitated the focus groups spoke about this effect: "Upsetting and deep stuff came up -I thought I'd be prepared but I wasn't. It's still difficult for me." Other members agreed that even reading the data was disturbing. One member said "I have had a good life but there were things as a child I hadn't thought about." For some members it was a learning process that violence goes on; for others it was being able to relate to client stories through their own experience:

I could relate to parts of it in my life, it has made me who I am and to face it, I realized the value of it though the heaviness, it gave it real impact and I saw it was very important. I talked about this in the group, it helped to talk about it.
Other members saw the importance of talking about the impact because:

We came from the generation 'you made your bed and now you lie in it', there was nothing for women going through the change of life and having nervous breakdowns.
The impact of trauma stories on practitioners is conceptualised as vicarious traumatisation, which develops from empathically listening to or "witnessing" traumatic stories and symptoms (Pearlman & Caringi, 2009;Herman, 1992;McCann & Pearlman, 1990). Practitioners can suffer from secondary trauma symptoms such as intrusive thoughts of clients' stories, negative feelings of distress and outrage, raised anxiety and startle responses. Trauma results in changes to people's cognitive schemas as they adapt to the knowledge that bad things can happen to oneself or others who are known and cared about. Cognitive changes affect people's beliefs about the safety of self and others, trust in others, and meaning of life. Ways of thinking and even the content of thoughts change as they struggle to adapt to new beliefs, for example reflecting on the nature of good and evil (Straker, 2000).
It is clear that research into women's stories of domestic violence also carries the potential for vicarious traumatisation. Community members and clients may be even more vulnerable to this phenomenon due to the lack of support structures such as supervision and professional boundaries which delineate [somewhat artificially] the practitioner's self from clients' lives. Client reference groups therefore require debriefing following any exposure to traumatic material gathered as research data.
Debriefing had been offered to the group member/co-facilitator immediately after the focus group, however this was refused. At the final evaluation stage, this group member explained why she had not accepted the researcher's offer: "[Facilitator] asked me did I want debriefing and I declined -I thought 'I'm tough, I don't need it.' " After the reference group evaluation session, this group member was able to participate in debriefing. In a following meeting, the group and facilitator developed further strategies for the remainder of the study and some suggestions for other researchers working with client reference groups. These recommendations include formally structuring debriefing sessions into the research design immediately following exposure to any sensitive data. Members described debriefing as "unload it before you leave." Preparation sessions before exposure to traumatic material were also suggested for future studies.
There were also positive outcomes from hearing trauma stories, similar to the effect of post-traumatic growth described by Calhoun and Tedeschi (2006). These positive outcomes included the realisation that more help is now available for women who are victims of violence than in the past; being more compassionate; and "feeling honoured women trusted me enough to share with me, they were very open and honest. I acknowledge their strength." Although these challenges are significant and need to be anticipated in planning research studies using a client reference group, the benefits outweighed the risks identified in this section.

Reference group outcomes
In an evaluation session conducted by the external researcher at the completion of the study, members of the reference group spoke of what they had gained through their participation in the research process. The benefits included a different perspective, being more informed and aware of what goes on "inside" the Centre, and having a sense of achievement. Group members felt that their opinions had been valued and that they had been encouraged to contribute in a positive way. Members also felt that they had helped others through their involvement, something the group had stated as an initial aim. The sense that the reference group had achieved its goals was expressed particularly in relation to the newly developed evaluation tools: I think the new forms are better -they are more relevant, with yes/no questions and options to know what people are thinking, they have more depth and are shorter. Time will tell when you get it back. The questions are more understandable, and can't be taken two or three ways.
The difference in perspective was demonstrated in a stronger commitment to the process of evaluating programs and services:

Now we see it [evaluation forms] differently: we say to group members 'we have to do it…paperwork is not a waste of time.'
There was also evidence of empowerment as consumers through participation in FARG: "my experience of giving feedback on groups here has been changed by this project"; "it's great that the Centre allows volunteers to participate in important things." Group members felt empowered to suggest other organisational changes and proposed that the Centre should "rewrite the group [evaluation] tool in the light of this experience." Members also identified personal changes such as being "comfortable in expressing yourself" and a growth in confidence. One member said:

I was unsure and thought 'who would want to listen to me talk?' [but] I found as the two years went by I was talking and being listened to so my confidence built up.
Another member said her experience with the reference group had encouraged her to further her studies. An increase in self confidence was particularly identified by group members who took on more public roles. The group member who volunteered to present at the conference on behalf of the reference group described the gains from taking this role: I can go to Melbourne and back in a day, I have confidence to prepare a speech, going to a crowded venue with people from all walks of life and professions. I got up and spoke to people and managed on a dais with two microphones. I was on a roll and then saw the director sitting there and thought 'if I get the sack I get the sack'. It was a fantastic experience.
The co-facilitator of one of the focus groups described the growth in self confidence: "I can handle this." There were also gains related to feeling part of a group and making a visible contribution: I used to feel anonymous as a member of a group whereas what we do here, people wouldn't know but we are more aware and can bring what others say to us over a cup of tea.
Another member said, "We all feel we are giving something back to the Centre and we know we are taken seriously." Members said that they enjoyed coming to meetings, and valued the companionship which had grown between other members and being able to "speak about things I've never spoken about", especially in relation to experiences of domestic violence.
Reference group members defined their role as being client advocates. This role was seen as providing more honest feedback for the Centre because volunteers hear things that professionals don't otherwise know. This was jokingly referred to as being "double agents"; "our role is to hear things but not to look like we're listening -like the Japanese with paper walls." Therefore the extent to which members represented the experience of other clients at the Centre developed over the life of the reference group.

Group facilitation skills
The group was asked about significant learning points through their involvement in the research project. One member responded that the outcome was not new learning, rather it was reinforcement of what was already known, for example that one group does not suit everybody. Others learnt that the evaluation process used by the Centre was "not a waste of time". Listening to other points of view was also seen as important learning. The group valued what they had learned about focus group interviewing: to stay on track with the discussion, to keep the group together, to value participants' input and to let them know it was valued.
The group was also able to identify what members had contributed to the project. This included tapping into their ideas and experiences of evaluation processes and forms. There was an awareness of the importance of confidentiality: "don't reveal private things about people's lives -guard as well as your own life." This awareness of confidentiality comes from members' prior employment experiences, and from participation in groups at the Centre which used group agreements about confidentiality. This demonstrates the relevance of clients' life experience to their role in the reference group.
The group was asked to identify the skills of the researchers that were helpful in facilitating the reference group. They identified picking out the relevant points of discussion and keeping the group focused so that meetings went smoothly. The warmth and personal attributes of the researcher who facilitated the reference group were highly valued. Members described the facilitator's qualities as enthusiastic, pleasant, easy, patient, professional, task-focussed and "having time for us." The recording of the discussion and sending out minutes were also seen as very important and allowed absent members to feel up to date with the project. As one member said, "sometimes I thought we had talked about nothing much" [until receiving the minutes]. The minutes gave members the message that the reference group was contributing to the project and achieving something constructive.
Handouts on focus group facilitation and data analysis were also seen as valuable resources to take away for reflection. However, there was a sense of progress on the research study being "slow". Members felt there was a need to keep being reminded of the big picture of the research to maintain interest, as it seemed that things moved so slowly, especially in the data analysis stage. There was also recognition that "it takes time to do things properly."

Discussion
The identified outcomes demonstrate that elements of social inclusion were facilitated by the reference group's involvement in research. Members voluntarily participated in research training and acquired skills which could be applied in other contexts. They engaged meaningfully with each other, the researchers and clients at the Centre through the research process. They influenced decision making about the study and the Centre's new evaluation tools. These elements are congruent with the Australian government's social inclusion agenda (Commonwealth of Australia, 2010). The shared engagement in collective learning which took place between the researchers and the reference group can be understood as a community of practice formed through the activity of the research study. The characteristics of a community of practice are the relationships which develop over time around shared goals and the generation of social capital through knowledge sharing (Lave and Wenger, 1991;Wenger, 1998).
The reference group members saw themselves, at the end of the research study, as being "on the inside" because they held inside knowledge about clients' views that professionals don't usually have access to, and because their participation in the Centre's research and evaluation processes was meaningful and was seen to shape the process of conducting research, interpreting data and developing new evaluation tools. This description demonstrates the power sharing achieved through the recognition of clients' unique knowledge and contribution to the study. McKevitt et al. (2009, p. 87) found that power balancing is achieved by the "effort on the part of researchers to promote equality in the face of a relationship characterized by an imbalance of power". Lee at al. (2009) similarly found more equitable relationships developed between clients and service providers through the development of an active reference group by a primary health service to overcome barriers to access. The balancing of power indicates successful consumer engagement has occurred through meaningful input into organisational processes (Gregory, 2008).
This case study of a research reference group has demonstrated the two way learning which occurs when clients and researchers work together and draw on each other's knowledge (Commonwealth of Australia, 2004), and the benefits for research participants, reference group members and researchers when a reference group is structured into the research design. Research participants benefit through the inclusion of appropriately skilled peers in data collection methods such as focus groups and interviews, as power differentials between researchers and participants are decreased. Clients and community members also benefit through their participation in a reference group, as a result of the capacity building components which enhance their research skills. Additionally, the benefits of group membership such as increased ownership of the services to which they contribute and improved self confidence have been identified in the reference group evaluation. Other recent research drawing on the expertise of reference groups has found improvements in the wellbeing and mastery of group members (Lee at al., 2009).
The researchers developed increased sensitivity to the views of clients, as well as improving the quality of data analysis through checking coding schemas and interpretations with the reference group. Methods used to access participants and collection of quality data were enhanced through tapping into the expertise of consumers. The language used for coding the data and writing up research outcomes became more consistent with the language used by research participants as a result of the input of the reference group. The partnership with the reference group therefore increased the quality of the research study.
However, not all studies have found that the involvement of consumers as coresearchers automatically improves the quality of research. Research involving service users recovering from stroke found their recall of interviews was affected by reduced memory (McKevitt et al., 2009). A study using chronically ill adolescents as co-researchers found that recruitment of participants and quality of interview data was poor as a result of unskilled interviewing, and that overall the quality of the research had not been improved (van Staa, Jedeloo, Latoo & Trappenburg, 2009). There are also ethical challenges in the inclusion of client reference groups in research design (McKevitt et al., 2009). This study faced the challenge of protecting members from harm caused by exposure to data about traumatic experiences, and the possible retriggering of members' own trauma.
This article has demonstrated the importance of research undertaken using clients' perspectives, and the benefit of client participation in ensuring research design and service evaluations are relevant and responsive to consumer needs. The data analysis and interpretation of results remained close to the experience of clients through the involvement of the reference group.

Biographical Note
Jessica Lopez is an experienced social researcher in women's health and with Indigenous and CALD communities. She worked for the Benevolent Society at the time this research was conducted and facilitated the Feedback and Reference Group. She is currently coordinating volunteers for a Telephone helpline at the Schizophrenia Fellowship of NSW.
Ione Lewis is an Associate Professor at the Faculty of Education, University of Canberra conducting research in the community sector and Papua New Guinea. She is committed to involving clients and communities in research that affects their lives, using participatory action research and capacity building approaches, in Australia and Papua New Guinea.
The Feedback and Reference Group members were Jan Abello, Marie Beattie, Deborah Mapp, Pat Prior and Glenda Wood. They worked closely with Jessica and Ione to ensure that the research study was inclusive of the views of participants. They volunteered their time to improve the service experience of other clients.