Students’ Reflections about Medical Writing Presentation Program: A Qualitative Analysis
Kevin Yulianto*1, Elisabeth Rukmini1
*Corresponding author: Kevin Yulianto
1 Medical Education Unit, School of Medicine, Atma Jaya Catholic University of Indonesia, Jalan Pluit Raya 2, Jakarta 14440, Indonesia
Download PDF:Journal Article Kevin Final
Background: To determine whether Medical Writing Presentation program is useful in preparing medical student for their learning.
Methods: This research was a descriptive-explorative study using a qualitative approach. Subjects comprised 207 first semester medical students of Atma Jaya Catholic University of Indonesia batch 2012. Qualitative data were collected from students’ written reflections about their learning experiences in MWP program. Qualitative analysis utilized content analysis included identification, coding, categorization, and synthesis.
Results: Students reflected positive views about using references in their writing, learning with internet as a media, and searching literature with EBM principles. (1) Students have a positive view of learning EBM, (2) Students are confident that the skills they learned in the MWP program will be useful for learning in medical school, (3) Students have positive perceptions using Zotero as a reference manager and peers who helped them to learn Zotero, (4) Students have positive views of using e-learning as a media for retrieving and collecting task, (5) Students also learned life skills that are indirectly related to the MWP program. While learning in MWP program, students had their own difficulties and challenges, but in the end they felt the program was effective and necessary for medical students.
Conclusions: Students had positive views of the program and its implementation. Students understood the importance of learning the MWP and felt that they could implement their studies directly in their academic careers.
Keywords: early EBM; perception; reflection; medical writing; presentation
Medical students will face various challenges and difficulties while learning in medical school. One of the many critical skills in learning medical theories and practice is the ability to solve medical question faced in learning.3,4 As a future researcher, a doctor also has to be able to understand research paper and to write it in a scientific way.
In learning medicine, medical students need to learn techniques of learning by utilizing various skills such as Evidence Based Medicine, ability to write research papers, and ability to search literatures related to their problems. Because of the importance of these skills, it is important for medical faculty to teach students skills about learning in medical school.
EBM was introduced in 1992 and considered as “one of the 15 greatest medical milestones since 1840” by British Medical Journal.5,6 David Sackett (1996) defined EBM as “the conscientious, explicit and judicious use of the current best evidence in making decisions about the care of individual patients”.7 There are 5 critical steps in EBM process8, among them are (i) creating answerable question from the clinical problem, and (ii) effectively and efficiently search the medical literature to identify the best available evidence to answer the question.
Research shows EBM courses in pre-clinical phase are satisfying, students understand the importance of searching answers for clinical questions, and students’ score in medical school are inreased.1,9 In searching answer to their question, student need to effectively search and evaluate information in higher education and health care settings. It mean that student need skills to choose literature that could help answer their questions, and understand the result of the research.
Designing the program
Series of tasks was given to 207 first semester students in a program called Medical Writing Presentation (MWP). The purpose of the program for medical students was to equip them with skills that would be needed in medical school, among them were the ability to learn based on Evidence-Based Medicine (EBM) principle, to use Zotero as a reference manager software to make bibliography, and to utilize MoodleTM as a media for e-learning. While learning to use Zotero, students were helped by their senior to learn the technical aspects of utilizing Zotero. These seniors are called Zotero Volunteer Troops (ZVT). The purpose of MWP program was to educate students to assess medical literatures and to present their researches in writing and verbally in front of their friends. The tasks were divided into 8 parts utilizing MoodleTM and EBM to complete. In the beginning, there was a survey for students asking whether MMR vaccine was related to autism. 22% students answered that MMR was related to autism. On the following assignment, students were assigned to search for literatures about autism. With the assignment, the instructor hoped that students would be enriched with knowledge about the problem they faced. In the second task students were assigned to search literature about MMR. Then students were assigned to search for literatures about autism and MMR. With the first three tasks, researcher targeted students to have enough knowledge about MMR and autism. Next, students were assigned to write whether MMR was related with autism. Following the fourth assignment, there was another survey, questioning the same problem whether MMR was related to autism. From the second voting, there were only 2% students that answered “MMR is related with autism.” From the second voting, the researcher wanted to know whether EBM helped students in learning medicine. In the fifth task, students were asked to write their conclusion whether MMR was related to autism based on the three medical literatures10-12 related to the problem. Among those three literatures, one of them was retracted by Lancet10. Students’ last assignment was to write a reflection about their experiences while learning in MWP program and their future plan regarding their skills in searching literature and using Zotero as a reference manager.
Data collection and analysis
The researcher (KY) retrieved 207 students’ reflections from the program contributor and erase the identifier such as name and students’ number. Then KY made representative codes based on the 30 students’ reflections. KY analyzed the reflections utilizing content analysis to categorize students’ reflections. Inter-rater meeting was completed with other researcher (ER) and two external inter-raters. From the inter-rater meetings, researcher agreed on categories from 30 students’ reflections. Then these categories were inserted into a table, combined with students’ reflections. All 177 reflections were analyzed using the method above. Triangulation data was performed by comparing the codes with the data from students’ evaluation of MWP program and the ZVT members’ self-evaluation. The evaluation included students’ perspectives on their senior who teach them Zotero.. To guarantee dependability, an audit trail was performed by an external expert to ensure the validity of the research.
Qualitative Research Results
Zotero helps student in creating bibliography
By using Zotero as reference manager, students did not need to write bibliography manually, Zotero would create the bibliography based on the selected styles.
“Other than that, I could understand more in using Zotero, so if I want to make a bibliography based on medical journal, I didn’t need to write it manually”
“ My plan for using this program is to help me work so I only have to edit some of the bibliography, so that I don’t have to rewrite and search for the source of my bibliography which takes more time”
PVZ helped students to learn Zotero
PVZ helped students to understand techniques in using Zotero
“To be honest, at first I didn’t understand how to write bibliography using Zotero, but when I redid my faulty task, I asked a lot of questions to my friends who understood, I was also helped by peers from PVZ, so that now I understand how to write bibliography using Zotero”
“I still didn’t know how to use Zotero. But after asking questions to my friend and PVZ, I started to learn how to use Zotero and tried to do my tasks in the best possible way.”
EBM helped in searching medical literature
EBM technique helped students searching valid and reliable literature.
Early EBM taught in MWP program had helped students to search and select literatures which were valid in medical world.
“I need to remember that not every source can be trusted for it’s validity, so I have to be selective in choosing my sources, and with these tasks, I am helped in choosing valid sources.”
“Doing tasks about MMR and autism made me understand how to search for valid sources. As a doctor, we should search for reliable sources and to not easily trust any source. I have to select which sources are valid and cannot search only from one source and trust it immediately, but it needs to be from various sources.
EBM introduced students to medical search engines
From learning wihin MWP program, students knew various medical search engines such as Ebscohost, Proquest, Pubmed, etc.
“Apart from Zotero, I also learned that in making a research paper there were sources categorized to be valid and not valid, there were search engines for searching valid sources too. These search engines included Google Scholar, Pubmed, Ebscohost, Proquest, etc. And search engine such as Google, Yahoo, and Wikipedia couldn’t be used as primary tools in searching literatures, these are different from my high school tasks which allowed me to use any source.”
“I also learned how to search valid literature from websites such as Proquest, Ebscohost, Pubmed, etc.”
“From these tasks, I could know various search engines and internet sources that were valid and reliable, especially in medical world, because in the past, I thought that all sources (including Wikipedia, blog) could be used as valid primary sources.
EBM technique helped students understand and synthetizing literature
“Other than that, I also learn to choose, read, understand, and also synthetize journals.”
“Other than that, I also learned to read and synthetize articles into one new article with paraphrasing technique”
“I also had to be able to synthetize research journal I’d read and conclude it. Other than that I also had to be able to create bibliography correctly, and I was helped by Zotero.”
MWP prepared students for their research paper
Zotero and Vancouver style helped student to create bibliography correctly in their research paper.
“Materially, I learned to write bibliography with Zotero and Vancouver style. Learning Zotero and Vancouver style will be helpful in my studies as medical student, especially in writing research paper and creating bibliography in the given tasks.”
“Like Zotero tasks, this assignment has a purpose so that in the future we will not be confused how to create bibliography using Vancouver style. We also learn how to search for a valid literature such as NEJM and Ebscohost, we are prepared (in writing research paper) by Zotero which is very helpful.”
“In the future, when making a paper or other tasks which need bibliography I will use Zotero and Vancouver style, that helped me in writing research paper. I will also learn how to write bibliography in other styles if I already mastered Vancouver style.”
E-learning helped student in their learning
MoodleTM facilitated student in doing and submitting their tasks online
MoodleTM became a media where student could download tasks given by the lecturer, fulfill the task, and submit the results online. These features helped student because students did not necessarily need to meet the lecturer face to face and as a result students could finish their task more effectively.
“After knowing [about Moodle™] further, I realized that MoodleTM is interesting and efficient. Teacher’s instruction to students could be easily accessed, and submitting an assignment could be done online without using print-out. There’s also a forum, although I rarely use it, but I think it is effective as a media for communication.”
“It helped me as Atma Jaya medical student to see assignment from teacher and submitting it from MoodleTM through my laptop.”
MoodleTM facilitate communication between teacher and student
MoodleTM provided a communication forum between teacher and student in doing their tasks without the need of face to face communication.
“From all tasks given, now I realize how to use MoodleTM and Zotero. MoodleTM act as a media that helped me to interact with colleague or teacher, and also as a media for teacher to assign tasks and for student to upload the result. 
At the beginning, students panicked and were stressful because of the given assignments.
Students were panicked and stressed because the various tasks were given without understanding how to do the tasks in the limited time.
“In my early study period at Atma Jaya School of Medicine, I was shocked by the various assignments given by teachers, among them were Medical Writing assignments. At first, I was so stressed and confused due to many tasks with limited amount of time. But, as a medical student, I realized that the given tasks had good purposes, to help us understand materials being learned. That’s how I felt after doing these tasks, it was meaningful at the end.”
“In the beginning I felt stressed and was in a hard situation due to many assignments. I was tired and was complaining. I felt confused about how to learn things like this and I felt down at certain times. But as time went by, learning in a group became motivating for me because I didn’t feel alone, there’s someone helping me. I am more knowledgeable in creating bibliography although I felt “lost” in the beginning.”
In the end students felt the benefits of the given assignments.
Although students felt burdened by the tasks at the beginning, they could feel the benefit of the given assignments.
“Although the tasks given is quite numerous, but I felt that the tasks would be useful in the future. Sometimes I felt very tired by the continuously given tasks, but I realized that it was for my benefit.”
“Although in the beginning I felt confused and tired by the given assignments and didn’t understand anything at all, in the end I knew that the assignments were given for my benefit.”
Students learned life skills
Students were trained to work in a team
With the given group assignments, students learned to socialize and work with their teammates.
“All of these tasks made me learn values different from what I learned from lectures, like working in a team and helping each other.”
“While learning [from the assignments] I learned one valuable thing, we, as medical students need friends to discuss and exchange ideas. When we are facing difficulties or trouble in learning, we can share our difficulties with them, hoping that they will sympathize. With friendship we can support and strengthen each other. It was the same with the assignments given, we should help each other in a group, sharing our knowledge with others who didn’t understand, so that this assignment could be finished.”
Students learned about time management
With the amount of assignments and time given, students learned to manage their time to do the tasks.
“Other than that, a doctor needs to manage their time. That is what I learned from the deadline given for the tasks.”
“From doing the assignments, I learned to manage my time, I was an unorganized person that at the beginning [of the program] I panicked. I realized when I entered medical faculty it wouldn’t be easy, these amount of tasks made me realize that time is worthy, expensive, and should be use properly due to the dense schedule and other time for socializing.”
Students learned about values of life
From various experience in doing their tasks, students learned values of life such as persistence, humbleness, gratitude, and learning from mistake.
“I learned to be more alert and detailed in doing assignments, more patient, persistence on things and looking for solution, not easily judge the difficulties of given tasks , because I know I could do it. From these tasks I now have a principle in learning: “Everything isn’t difficult. If someone can do it, you can also do it and even better” and “Life must go on” so I didn’t focus on my mistakes and realized that achieving something takes process, so I have to be patient while pursuing my achievement.”
“From this [MWP program] I realized that I shouldn’t repeat mistakes I had done before. We should learn from our mistake to be a better person. To me, this [MWP program] is important in medical school, because experience is the best teacher. Other than learning from mistakes, as a medical student we need to do and complete assignments on time. Although the given tasks were numerous, but the purpose of the tasks was to train our responsibilities on doing assignments and increase our knowledge, not merely to burden us but to prepare ourselves for the future.”
Students were trained to read medical journals
While doing tasks in MWP program, students needed to search literatures for doing their assignments. From doing these works student became fluent in reading medical journals.
“I learned a lot of things from these tasks. First, I learned to create a habit for myself to read and understand scientific journals in English.” 
“Indirectly, I also make a habit to read medical journals.” 
Students had a chance to learn independently in MWP program. Students thought that independent learning was needed by medical students, because the knowledge they gained was much greater than learning from lectures alone.
“The way of learning applied [in MWP program] was based on e-learning and independent learning. The application of independent learning is needed in Atma Jaya Faculty of Medicine where teachers act as facilitators. To gain more knowledge students need to actively participate in searching for materials for independent learning. So I will deepen my learning and learning independently to facilitate my future learning.”
“This way of learning is quite good to be applied because it makes me more active and learning independently in searching information that I need to do my task or to extend my knowledge, by asking senior or searching for literatures from library or the internet.”
Students had positive perceptions about Zotero, mainly because they don’t need to write bibliography manually. Moreover, Zotero saved a lot of time in creating bibliography according to necessary styles , in this case Vancouver style. But, without giving a brief explanation about Zotero before doing their tasks, students were confused about the use of Zotero. For new medical students or even new users, Zotero seemed complicated. But after understanding why and how to use Zotero, students realized that the function of Zotero was extremely useful and they planned to use it in their future assignments. While learning to use Zotero, students were introduced to literature searching in medical search engines such as PubMed, Proquest, and EBSCO. Students also learned to write bibliography in Vancouver style. Students found that in using Zotero for their bibliographic writing, they needed to adjust manually within the word documents. Some adjustments they needed to do were the journal abbreviation and date format.
While learning to use Zotero as a reference manager, students were helped by ZVT to understand the technical aspects of Zotero. There were various ways for ZVT to help students, such as using online forum and face-to-face meetings. Apart from learning Zotero itself students were also introduced to their peers. In their reflections, students said that learning from their peers were more comfortable and less intimidating. These results were similar to a research in Washington University13, students felt that learning from their peers was more comfortable and should be integrated to the curriculum.
There were various EBM techniques learned by the students, among them were the differences of primary and secondary literatures, searching for medical literatures, understanding and synthetizing literatures. Research in Isfahan University showed that EBM workshop in pre-clinical phase could increase student’ ability to make clinical questions and searching for related literatures. It also induced student to use EBM in their further learning.14-17 Students felt that EBM was important in learning medicine; they planned to use it in Problem Based Learning and in writing research papers. Research in Monash University2 shows that skills in EBM are useful in solving problems in PBL.
Students concluded that MWP program prepared them before learning in medical schools with skills they would use in their future academic careers. Although students complained because of many tasks and limited time to do their assignments, in the end they felt that the skills they have had learned were useful. Reflection can be used to help students face their confusion and is able to make their learning meaningful18. Interactions between students also increased their satisfaction in learning meaningfully19. These experiences match with the principle of learning meaningfully. According to Mayer20, “meaningful learning is reflected in the ability to apply what was taught to new situations, so we measure learning outcome by using problem-solving transfer tests.”
The usage of MoodleTM as e-learning media helped students to retrieve and submit their assignments effectively. Lau and Bates conclude that e-learning is more effective and can increase students’ score compared to the traditional method. E-learning also helps to equalize students’ perception15-17. One of the drawbacks in Moodle was the utilization of ‘forum’. Although there is a forum to facilitate interaction between students and teachers, it was rarely utilized due to more accessible media such as Blackberry MessengerTM.
While learning in MWP program, students also internalized various values of life and life skills. They learned to work in a group while doing their tasks and they supported each other when facing difficulties. Learning in a small group could induce student to think critically22. They also learned to manage their time to do the assignments and various other activities. Research in Midwestern University23 shows that student with the ability to manage time effectively have a lower stress level. In searching MMR and autism literatures, the students became used to read medical literatures in English (English is not their native language).
Learning in MWP program allowed student to learn actively instead of passively receiving material from lectures. They understood that they could not rely only on lectures in their learning. They realized that they needed to actively search for information, both online and from their peers. In active learning, students could understand their learning purpose and plan their studies24.
In the beginning of the program, students panicked, were stressed, and confused due to many assignments, their inability to use Zotero, and they did not know how to do their tasks. The situation was made worse by the obligation to use the Vancouver citation style, which they were not familiar with. However, along the program they became more familiar in doing their assignments correctly. When the program ended, students were thankful because the assignments and experiences in the program had taught them skills they would need in learning medicine.
The research purpose was to know the first semester students’ perceptions of MWP program at the School of Medicine Atma Jaya Catholic University of Indonesia. Researcher analyzed 207 students’ reflection using qualitative approach. Results showed that: (1) Students had positive perceptions about learning EBM principles, because EBM helped students in searching for valid literatures and finding important information within the literatures, (2) Students thought that skills they learned in MWP program would be useful in their future studies at the school of medicine, (3) Students had positive perceptions about using Zotero as a reference manager in doing their assignments, and students had positive perceptions about ZVT who helped them to use Zotero, (4) Students had positive perceptions about Moodle as a media for doing their assignments, (5) Students also learned values of life and life skills such as time management, teamwork, and reading medical literatures. While learning in MWP program, students faced various challenges and difficulties, but in the end students felt that MWP program was effective and was needed for medical students.
- Jarmila P, Vladimir M, Dagmar K: Teaching Evidence-Based Medicine to undergraduate medical students: Information Specialists as Multi-Professional Team Members. European Social Fund, 2007;3 (4):49-51.
- Ilic D, Forbes K: Undergraduate medical student perceptions and use of Evidence Based Medicine: A qualitative study. BMC Medical Education, 2010;10(1):58.
- Kingsley KV, Kingsley K: A Case Study for Teaching Information Literacy Skills. BMC Medical Education 2009, 9:
- Pinto M, Cordón J, Gomez Diaz R: Thirty years of information literacy (1977-2007). Journal of Librarianship and Information Science 2010, 42:3-19.
- group E-bmw: Evidence-based medicine – a new approach to teaching the practice of medicine. JAMA 1992, 268(17):2420-2425.
- Montori VM, Guyatt GH: Progress in evidence-based medicine. JAMA 2008, 300(15):1814-1816
- Sackett DL, Rosenberg WM, Gray JAM, Haynes RB, Richardson WS: Evidence-based medicine: what it is and what it isn’t. Br Med J 1996, 312:71.
- Straus S, Glasziou P, Richardson W, Haynes B: Evidence-based medicine: how to practice and teach it. Edinburgh: Churchill Livingstone: Elsevier; 2011.
- Taheri H, Mirmohamadsadeghi M, Adibi I, Ashorion V, Sadeghizade A, Adibi P: Evidence-based medicine (EBM) for undergraduate medical students. Ann. Acad Med. Singap, 2008;37(9):764–8.
10.Wakefield A, Murch S, Anthony A, Linnell J, Casson D, Malik M, et al. Retracted: Ileal-lymphoid-nodular hyperplasia, non-specific colitis, and pervasive developmental disorder in children. Lancet, 1998;351(9103):637–41.
11.Allan GM, Ivers N: The autism-vaccine story: fiction and deception? Can Fam Physician, 2010;56(10):1013–1013.
12.Madsen KM, Hviid A, Vestergaard M, Schendel D, Wohlfahrt J, Thorsen P, et al. A population-based study of measles, mumps, and rubella vaccination and autism. New Engl J Med, 2002;347(19):1477–82.
13.Josephson SA, Whelan AJ: A new first-year course designed and taught by a senior medical student. Acad Med. 2002;77(12 Pt 1):1207–11.
14.Ilic D, Forbes K: Undergraduate medical student perceptions and use of Evidence Based Medicine: A qualitative study. BMC Med Educ. 2010;10(1):58.
15.Lau F, Bates J: A review of e-learning practices for undergraduate medical education. J Med Syst, 2004;28(1):71–87.
16.Abdelhai R, Yassin S, Ahmad MF, Fors UG: An e-learning reproductive health module to support improved student learning and interaction: a prospective interventional study at a medical school in Egypt. BMC Med Educ, 2012;12(1):11.
17.Jamero DJ, Borghol A, Mihm L: comparison of computer-mediated learning and lecture-mediated learning for teaching pain management to pharmacy students. Am J Pharm Educ, 2009;73(1):5.
18.Hajar T, Maede M, Iman A, Vahid A, Atefe S, Peiman A: Evidence-based medicine (EBM) for undergraduate medical students. Ann Acad Med SG, 2008;37.
19.Cheng HM, Guo FR, Hsu TF, Chuang SY: Two strategies to intensify evidence-based medicine education of undergraduate students: a randomised control trial. Ann Acad Med SG, 2012;41:4–11.
20.Mayer RE, Moreno R: Nine ways to reduce cognitive load in multimedia learning. Educ Psyc, 2003;38(1):43–52.
21.Varghese J, Faith M, Jacob M: Impact of e-resources on learning in biochemistry: first-year medical students’ perceptions. BMC Med Educ, 2012;12(1):21.
22.Gokhale AA: Collaborative Learning enhances critical thinking. J Technol Educ, 1995;7(1).
23.Misra R, Mckean M: College students’ academic stress and its relation to their anxiety, time management, and leisure satisfaction. Am J Health Studies, 2000;16(1).
24.R.L.B. N, D.A. P: Toward independent learning: curricular design for assisting students to learn how to learn. J Med Educ, 1981;56:886–93.