How can I design a recovery-oriented e-learning website for people with mental health difficulties?
This paper presents my living theory, developed as I sought to improve my practice as a mental health professional, and as I answered my research question: â€śHow can I design a recovery-oriented e-learning website for people with mental health difficulties?â€ť
Information technology has the potential to increase learning opportunities, promote inclusion and improve the quality of life for people with mental health difficulties. However, this group currently experience significant inequalities in accessing and maximising the potential of online learning interventions, due to lack of consideration by designers of their specific learning, usability and accessibility needs.
This paper presents my research as I attempted to use technology to enhance and support the learning of people with mental health difficulties in a day service in south Dublin, and to encourage a recovery-oriented mental health service delivery, which has hope, inclusion, learning by doing and group support as its guiding principles. This enquiry involved the design and evaluation of an e-learning website for this group of service users.
I look forward to any comments or suggestions you may have about this paper.
I particularly invite the comments of my reviewers Pip Bruce-Ferguson and Sarah Porter.
This is a timely and sensitive paper in which you seek to show how you have worked with people who have mental health difficulties, in seeking to improve internet access and usage with this community of users.
You set the context well by summarising your approach to working with individuals at the outset, reinforcing this with strong statements about your own values in section 3.2. I am not an expert in your particular area of mental health, so cannot comment comprehensively about your literature review or the web design issues that you raise in section 2.
In section 3, I found some inconsistencies with voice a little disruptive at times. You sometimes address the reader as â€śyouâ€ť (as in â€śYou may state what you believe in, or valueâ€ť but later write in the third person (â€śThe researcher then gathers evidence to show how they have developed their own learningâ€ť. And you speak in the first person, which is appropriate given the Living Educational Theory approach youâ€™re using (â€śI feel that not only must I be involved in my research, but â€Iâ€™ must be at the centre of my researchâ€ť). Ronan, Iâ€™d just go back and check the consistency of usage throughout the document.
In section 3.3, you indicate that false names have been used if people refused permission to use their real names, but not indicated whether you had to delete â€imagesâ€™. Your previous statement said that â€śMost agreed that their image and real names could be usedâ€ť so this raises questions in the readerâ€™s mind about whether ethics meant you had to delete or obscure peopleâ€™s faces, though I didnâ€™t notice this in any of the video clips.
There are lots of places where I put ticks in the margin (my shorthand for â€Yes, I agree!â€™) I have done this in 3.2 where you talked about ethical clearance; in 3.3 where you talked about your validation group; in the numbered points at the end of this section where you stated your desire to be judged on whether your educational values are clearly revealed and justified; in 4.3 where you clearly indicate that youâ€™ve genuinely involved your service users and state the ethical grounds for doing so; in 4.6 where Jack has commended your work; later in that section where you commend the service users for their time and effort, and that they have influenced your learning. At the end of 4.3.4 I have written â€śgreat structuringâ€ť as you lead the reader from Cycle One to Cycle Two.
Clearly coming through your description of the methodology, implementation and evaluation is your desire to really make these technologies work for the users, to help them to access what they believe they need from the internet. I did have some concerns when watching the videoclips, as some of your questions appear to â€leadâ€™ respondents in a way that is at variance with the way I have been taught to interview. However, it is possible that this degree of leading may be necessary given your client group. This is a question that should be answered by someone more familiar with the needs and requirements of this group than myself.
Certainly, your service users were very positive about what they had been able to achieve with your improvements. It is hard to see how, ethically, one could carry out work such as you have done without the informed, active involvement of these people. If you had just implemented design changes without their input, making assumptions about what you thought they needed, you would have violated your own values; from what you have said of good practice in your field, also current norms; and been at risk of inflicting changes that might not have worked at all well for your service users.
Ronan, I do believe that this is a strong paper, about which I would recommend few changes besides the relatively minor ones I have raised â€“ the voice issue, clarification about the â€imageâ€™ obliteration (if used), and perhaps a response to my comment about leading your respondents. Otherwise, I think the paper reads very well and that your values and the standards against which you wish to be judged, are clearly and appropriately expressed and met.
There are a few minor typos that I can alert you to if you wish to contact me by email â€“ not worth mentioning in this analysis.
Firstly thank you very much for your comments and suggestions. They clearly come from a very considered reading of my paper, which is very much appreciated.
I have revisited the entire paper in relation to the voice consistency issues you mentioned, in particular section 3, and have amended those sections, attempting where possible to use the first person. I believe it reads much better now!
Regarding the consent issue in section 3.3 I have added the following to the text:
All research participants signed consent forms and most agreed that their image and real names could be used; where this consent was not given I have used false names in any interview or focus group transcripts and have not recorded their image in any way.
I only used cameras or video where all participants had given full consent; where this was not given I used a dictaphone and any transcripts were written using codes (such as p1, p2 etc) which I later changed to false names. Guidelines for people with disabilities are quite strict in this regard. Any video interviews I posted on You Tube for EJOLTS purposes gave further consent (in my original dissertation these were only on an accompanying DVD).
Your point about 'leading questions' is a very valid one and had come up in previous conversations with my supervisor. I had considered including a paragraph or two in my original dissertation on this topic, but did not due to space constraints. I feel it may be part of my own learning as part of this research, and that my asking 'leading' questions, so to speak, resulted partly from my deep involvement with the website itself and my familiarity with the users in my research group. There could also be a discussion in here around the needs of the users themselves to be guided or directed more than others... but I'm not sure if that's a discussion best left for another day!
I think that I will include a short paragraph on this, as it is something that I have learned in the course of my research. Do you feel this should be in the reflection section of my implementation chapter or should I mention it earlier in my methodology?
I will email you directly re: typos. Any further suggestions would be welcomed. Regards,
I have been looking over my article again, particularly with reference to your concern about the 'leading' questions posed in the interviews, which I believe is an important issue to tackle. I also discussed this with Margaret recently and I feel I should clarify one or two points in the article for readers.
The initial focus group sessions I held with a larger number of participants also included those who later took part in the one-to-one video sessions. I am attaching the transcripts of these (a number of the group did not give consent for their image to be used so I only used a dictaphone in this case - the focus group sessions were not videotaped) - these were included as attachments to my initial sumission to EJOLTS so I'm not sure if you have seen them yet.
The focus group sessions helped me to identify common themes in relation to recovery, accessibility and usability, as well as to further analyse the results of the initial questionnaire. I also used the principles of the recovery model (including hope, empowerment, and self-directed learning) as areas or themes around which to structure the later interviews. I recall discussing this with Margaret in the course of my research and she seemed happy that the initial questionnaire and focus group data helped to give focus to the individual interviews which were held later.
I take your point that someone viewing the video interviews in isolation may view my questions as leading but if I add an explanation in the article as to how I came to ask those particular questions (in addition to ensuring that the focus group and video transcripts are available as links within the article), it may clarify the issue somewhat. What do you think?
I could also mention that I was trying to help the interviewees by providing example answers rather than giving a wide open question (which can sometimes be a difficulty for people with mental health difficulties to answer - trying to get blood out of a stone springs to mind!).
Greetings and an apology for my late reply. This is my first review. My background is mental health and my dissertation advisor was David Morgan PhD one of the early writers about focus groups,in the 1980's Sage publication.
I am positively impressed with the organization and flow of ideas in your manuscript. Like Pip, I experience the passion of your work - it seeps through, nicely done there. I appreciate your use of focus groups to add to your understanding of participates' experience. It is an interesting way to both develop more structured questions as well as validate interviews. Dr Morgan always stressed that when interviewing go for the topic until it feels saturated and then ask, 'does anyone have something different to add'. I did look for questioning about what would you do to improve the system, or was there something you wanted that you didn't have. Maybe I just missed that part.
I really did like the consistency of your philosophical approach your values, the method and implementation of the method - I think it hangs together very well. You recognize the potential for the research method itself to be empowering and have connected it with current research and theory.
Another thing to think about is whether or not your status as a mental health worker and the implicit power, that you might have, in any way influenced the responses of the participants. That coupled with the exposure on camera or tape could have a stultifying effect on communication (theoretically speaking of course) where they would 'give you what you want to hear'. This would be something to ponder in relation to the trustworthiness of data. You sought and received quality supervision in this research. It is possible this was discussed but not explicated in your manuscript.
I have a few minor suggestions regarding writing and they are just suggestions for you to consider:
Regarding the Abstract â€“ as a reviewer somewhat new to the writing of Living Theorists â€“ I am tempted to put the last paragraph first. First it lets me know immediately what the writer is attempting and second it seems to switch the gestalt of living theory bringing it to the forefront. For me this has more â€grabâ€™.
DIY ~ perhaps when you first use this acronym you could specify "do it yourself (DIY). It took me a minute to catch on.
Page 10 Love the paragraph outlining ethical considerations. Sometimes researchers include a statement that they can withdraw from the study anytime for any reason without prejudice. Was that also a consideration? Perhaps your more general statements imply that option.
You have a very decent review of the literature. You are breaking new ground.
Thank you for all your hard work and creativity. You should feel very good about your accomplishments, I certainly do.
Dear Sarah and Pip,
I am attaching the latest draft of my paper in which I have attempted to address some of the issues (and typos!) you have both suggested.
- I believe I have resolved the issue with voice (i.e. first and third person conflicts).Â I you can identify any further issues you can let me know.
- I have also clarified the use of participant images
- I have attempted to explain the 'leading' question issue by adding some explanation on p14 and 17 in relation to the structure and content of focus group and interviews
- I have also tried to clarify this issue through the addition of a couple of paragraphs in section 4.6 rigour and validity
- I have changed the paragraphs around in the abstract as suggested
- I have removed the DIY reference on p9
- I have added your suggestion in my ethical considerations p10 (it was actually in my original dissertation but I had edited it out for the article!)
- As mentioned above for Pip, I have tried to clarify the potential for my influence in the focus groups and interviews
Once again thank you for your suggestions. I hope any changes I make have improved the article and it still reads well! Please let me know what you think of the latest incarnation.
P.S. I intend the focus group and video interview transcripts I attached earlierÂ to be included as links in the final article
I am attaching your paper which I enjoyed very much, with a few suggested changes for syntax and style. Not much, though, because it was very clear and readable all through. My changes are in red through track changes, and some comments and thoughts in yellow.
I've changed semi-colons to full stops just once or twice where I found myself stopping the sense to ask "now why is that semi-colon there?"
I also was changing Chat to chat facility, but realised you were capitalising all through. So I imagine then it is the actual title of a named site, rather than just the general 'chat facility' ? Anyway, just a thought.
I hope this helps with the final push towards publication,
all good wishes,
Thanks Jane for your feedback. I have made any relevant changes you suggested (see attached).Â I will leave Chat in capitals as it refers to a specific element added to my website, rather than chat functions in general.
I do appreciate the additional work that you have done to respond to feedback. I think the voice issue is now completely resolved. Your "I" is clearly at the centre of the work, and you have removed third person comments where necessary. You have also clearly clarified the use of images and confidentiality.
I still find the issue of leading respondents somewhat problematic, despite the comments that you have inserted. I noted Sarah's response, "Another thing to think about is whether or not your status as a mental health worker and the implicit power, that you might have, in any way influenced the responses of the participants." Given that you say Margaret had also raised concerns about possible leading of respondents, I think it would be appropriate for you to add a brief additional paragraph in your reflection on the work, possibly on page 19, indicating that your supervisor and two outside reviewers had raised queries about the way that the interviews/focus groups proceeded. You could then insert your comments about why you felt it important to lead in the way you did (you'd referred above to 'trying to get blood out of a stone' with some mental health users, and this may have led you to fish for answers more than you might with non-mental health users.) You might also indicate that now you have been alerted to the way this kind of leading can be perceived, you would be careful next time either not to lead so much, or to advance a sound rationale for why you, knowing your client group and the norms of working with these folk, have conducted your work in the way that you have done.
However, I also note that you have cited feedback from other 'outsiders' such as Jack Whitehead and your validation group, so you may feel that an additional paragraph is not necessary. I'd suggest that you check this with Margaret, if she's available for that conversation.
I hope that the paper will be through the reviewing process shortly Ronan, as I have a couple of people here in New Zealand whom I think will be interested to read it! So, overall, very well done, and I hope that these reviewing comments have helped you as you reflect on your practice.
Pip Bruce Ferguson
I appreciate your comments on this issue.Â It certainly can be an area of concern, particularly in disability research. It is difficult at times to move a conversation forward, given the particular needs of some of the people I work with, without leading or bringing my influence into the process.
I have altered and added a couple of paragraphs to section 4.6 rigour and validity which I hope explains my own reflections on this topic and how I intend to approach similar situations in the future (see attached and below):
"My main concern in facilitating the focus group sessions and conducting the interviews myself was that I not be seen as influencing the scope and direction of the conversations.Â This is always a possibility as I have worked closely with all participants in the past as part of my role as a mental health professional; also participants were aware that I had designed and developed the site and this could influence their responses.
"My supervisor and outside reviewers did raise minor concerns around leading respondents, particularly in the one-to-one interviews.Â I believe that, given the particular needs of some of the participants, I, as interviewer felt it necessary to provide direction within the conversation at times, including framing possible responses to move the process along. Â I accept the comments raised as to my potential influence, as this can be an area of concern in disability research (NDA 2002), and will certainly consider this issue in any future research.
"I believe, however, that by structuring my evaluation as I have done by firstly conducting a usability and accessibility questionnaire, analysing the results in a focus group setting, and later in individual interviews based around themes emerging from the focus groups, and building my recovery evaluation around the principles of the recovery model has made a considerable contribution towards improving the validity of my research.Â I also acknowledge my potential influence at all stages with participants."
Once again, thanks for your feedback and hope this clarifies this issue within the article for the reader.
Well done, and I think it's fine at this point. From my perspective, it should proceed to publication.
If it's any consolation, when I visited the UK in 1994 (three month teacher exchange paid for by the British Council) I was introduced to videotaping practice. So I went back to my polytechnic, set up a peer observation action research exercise with a colleague, and we both videotaped our own practice. We wrote a self-critique, then sent that with the tape for each other to view. When I looked at my video, I saw a rather more directive teacher at the front of the class than I actually thought I was. My conception of myself (Schon's espoused theory) was that I was practising Freirean liberatory education, but what I saw (Schon's theory in use) was rather different! I wrote quite a scathing self-critique along those lines. However my colleague, when she saw the tape, put another slant on it. She said that what she had experienced of my teaching (I'd previously taught her) and what she saw on the tape, was not that far out of line with my Freirean orientation. She pointed out that the videotape had focussed only on me, not capturing the students (for ethical reasons) and that most likely I was picking up visual cues that students were ready to move on, and responding to these, rather than imposing my own rate of progress. Well, that might have been a very kind perspective, but I certainly found it helpful.
So I guess all I was trying to do in my feedback was to alert you to a possible discrepancy between your espoused theory and values, which you articulate clearly, and your theory in use, if you were being too directive with your respondents. It is over to you to make that call. I think now that you are alerted to the possibility, you will reflect carefully when you conduct such interviews in future, to ensure that any direction you give your respondents will be appropriate to their needs and wants. Critical friends can be supportive as well as critical, as Kate was to me!
All the best Ronan.
Thanks again for your feedback.Â I have learned a lot from this reviewing process and I certainly think my article has been improved upon through you suggestions and the feedback from others.Â Once again very much appreciated.
Margaret suggested a couple of minor edits clarifying my validation group meeting on page 19 that I had uploaded to YouTube so this may be the final draft!Â I have linked all images to their YouTube links.
I would ask that the editorial board link the files I added to my initial upload, namely:
1. The usability and accessibility questionnaire referenced and highlighted on P13, P15 and P16
2. The focus group transcript referenced and highlighted on P15 and P18
Once again thanks for all your help and feedback throughout this process.