Evaluation of the Satisfaction and Usefulness of a Web-Based Educational Program for Breast Cancer Patients
Myungsun Yi1, 2, Jeongeun Kim*, 1, 2, Dong-Young Noh3, Jung-Lim Lee4, Keun-Young Yoo3, Ki-Tae Hwang5, He-Doo Chung6
Identifiers and Pagination:Year: 2008
First Page: 129
Last Page: 137
Publisher Id: TOMINFOJ-2-129
Article History:Received Date: 24/3/2008
Revision Received Date: 14/4/2008
Acceptance Date: 15/7/2008
Electronic publication date: 26/8/2008
Collection year: 2008
open-access license: This is an open access article licensed under the terms of the Creative Commons Attribution Non-Commercial License (http://creativecommons.org/licenses/by-nc/3.0/) which permits unrestricted, non-commercial use, distribution and reproduction in any medium, provided the work is properly cited.
The purpose of this study was to evaluate the effectiveness of a web-based breast cancer educational program which consists of special features such as flash animations and online counseling as well as 7 different categories of information on breast cancer. The effectiveness of the program was analyzed in terms of its function and content. A total of 147 women with breast cancer who visited the website for at least 30 minutes and a minimum of 3 visits, participated in the survey.
In the satisfaction evaluation of web-based educational program, usefulness of information, system efficiency, adequacy of information and convenience of use all received positive evaluation and showed even distribution of 49.14 (±6.05) points out of 64 points total. In the usefulness evaluation, the subcategories had following scores from the highest to the lowest; understanding of breast cancer was 3.34 (±0.51), life after treatment (3.21±0.58), early detection and examination (3.20±0.60), chemotherapy and hormonal therapy (3.18±0.55), related factors and prevention (3.16±0.59), treatments (3.13±0.53), and diagnosis (3.02±0.56). Factors affecting the satisfaction of the program were age, religion, income, stage of disease at diagnosis, source of health information, duration of Internet usage, and whether the patient performs breast self-examination. Factors that affect the usefulness of the program were religion, period since diagnosis, source of health information, frequency of Internet usage, recurrence of breast cancer, and family history.
Although the program was evaluated as somewhat useful and satisfactory, it should be improved upon by providing in-depth and cutting edge breast health information especially for women from a higher educational and income background.
The number of breast cancer cases in Korea has been increasing significantly, and it has become one of the most common types of cancer in women, making up 16.1% of all cancer incidence in 2001. It is reported that most Women with Breast Cancer (WBC) felt abandoned by health professionals when the treatment was over, and they needed assistance in the transition from being a cancer patient to a cancer survivor . This phenomenon is supported by many Korean research studies [2-4] which indicate that WBC are eager to obtain breast health information, primarily in the areas of prevention of recurrence, etiology, diagnostic tests, and prognosis, as well as symptom management, diets, exercises, and alternative therapy [2, 3]. This finding is consistent with the study from 102 WBC investigating the key educational and counseling needs of WBC in Seoul area . The highest, educational need was to know about the method of recurrence prevention. In the pre-admission period, etiology diagnostic tests, treatment and prevention of the breast cancer were needed the most. During the admission period, symptom management after the operation had the highest score. After discharge, prevention of recurrence had the highest score. The results of this research indicate that an educational program should be focused on the medical and symptom management information based on their treatment stages.
Recently, information and education provided through Internet has been actively used to provide emotional support for the patients and their families, as well as helping them in their decision of choosing a health care provider . The most helpful aspect of the Internet is that the user can seek information at the most convenient time and place for them. Through education via Internet, patients can gain significant medical knowledge, self-care and bio-clinical goals can change positively, and the quality of life improved .
Although some studies [7, 8] indicate a negative impact of Internet use, such as decreased rates of family communication, decreased social contact, and greater loneliness, other studies demonstrate that Internet use offers greater social support. Its positive impact was demonstrated in the studies focused on the need of specific types of individuals, for example, parents of children with developmental disabilities , individuals with isability , caregivers of Alzheimer's disease patients , widows experiencing bereavement , and elderly individuals residing in assisted and independent living facilities .
The effect of Internet use was also demonstrated in cancer patients with various diagnosis [14, 15], and breast cancer patients in particular [16, 17] Fogel et al.  indicated that 42% of breast cancer patients use the Internet as a source of information in the United States.
A recent survey in Korea reported the number of Internet user as almost 2.7 million, accounting for 64.1% of the population in 2003 . Among them the number of women was about 1.3 million. The age distribution was highest in their 30s and 40s, and most of them (71.6%) used the Internet to search for information. The mean number of hours spent on the Internet, per week was 12.2 hours. However, well organized Internet sites about health care, breast health in particular, have not been developed to a great extent in Korea. Thus we strongly believe that web-based educational program would be very effective and efficient in supporting WBC in Korea.
The primary goal of this study is to develop and test a web-based educational program for WBC in Korea. A web-based program was chosen not only because Korean women are reluctant to discuss breast healthcare publicly  but also because Koreans have relatively high access to high-speed Internet at home.
- Assess the educational needs of WBC in Korea.
- Construct and develop a web-based educational program based on these needs. The program will include a self-learning program using decision tree model.
- Evaluate the satisfaction and usefulness of the web-based educational program.
1) Development of the Website
To achieve the objective number one, the analysis of the necessity and need of breast cancer related web contents geared towards breast cancer patient was administered from May 2005 to July 2005. Based on this study of general needs, a web-based educational program was developed to achieve the objective number two. The breast cancer related contents of the website were decided to consist of understanding breast cancer, etiology and prevention, early detection and prognosis, symptom management, chemoradiotherapy and hormone treatment, as well as life after full recovery. In order to have distinguishing features compared to already existing websites, and effectively achieve the educational goals, the website developed Flash animation as means of displaying its contents. Prior to uploading them to the website, the rough drafts of the Flash animation were submitted to breast cancer Internet patient community and edited and improved according to their critique. The finished menu of the website consisted of Introduction, Cancer Summary, Breast Cancer Information, Expert Consultations, Patient’s Community, Family’s Community, FAQ, Recent Research Directions, Cancer Survivor’s Stories, and Links (Fig. 1). For the Expert Consultations, medical experts from various University hospitals provided their insights. The website premiered on March 19th, 2006 and reopened after edit and improvement on May 1st, 2006, providing information to the patients who signed up through the administration process. Finally on March 19th, 2007, the content became accessible to general public without the login process (http://www. kbccc.org/).
Screenshot of the flash animation of “Breast Cancer Information”.
2) Measurement Tools
(1) Items Measuring Satisfaction of the Website
A questionnaire consisting of total 16 questions was developed to evaluate the functions of website, with the highest possible score being 4.0 per item. The subcategories consisted of 2 items of System Efficiency, 3 items of System Convenience, 1 item of Design, 7 item of Adequacy of Information and 3 items of Usefulness of Information. The lowest score possible for each item is 1.0, and highest being 4.0 that makes the total sum of the questionnaire from the lowest score of 16 points to the maximum of 64 points. The reliability score of the measurement tool was Cronbach’s alpha 0.92.
(2) Items Measuring Usefulness of the Website
There are total of 7 questions evaluating the contents of the website and the maximum of 4.0 points indicate the highest satisfaction. The subcategories consisted of cancer chemotherapy and hormonal therapy, and the lowest possible score being 1.0 and the maximum of 4.0 points. The lowest possible score for the evaluation is 7 points with the highest possible score being 28. The reliability score of the measurement tool was Cronbach’s alpha 0.83.
3) Data Collection
Among the people who visited Korea Breast Cancer Cyber Center (KBCCC), the subjects were those who logged in more than 3 times with total visiting period of 30 minutes or longer, and those who consented to participate in the study. It was decided to include the participants who are eligible for the criteria, because the volume of the contents could be understood only by the sufficient frequency and time, namely over 30 minutes through no less than 3 times. There were 497 users according to the access log, however, those who were unmet the criteria were excluded for the study. As it was not sufficient to recruit participants only by contacting e-mails and it was necessary to recruit as many participants as possible, telephone and mail contact were chosen. The subjects were contacted through phone call and gave verbal consent before they were given written consent form along with the questionnaire via postal mail. The total number of people contacted via phone call was 286 and out of them 49 did not answer their phone calls, 74 declined to participate in the study. 163 people agreed to participate in the studies on the phone and out of them 147 people successfully returned their finished questionnaire form.
4) Data Analysis
The collected data were statistically analyzed using the SAS System for Windows Version 8.0 by following methods:
- The analysis on the subjects’ specific demographics, satisfaction and usefulness of educational program were done by t-test and ANOVA.
- The variables affecting the evaluation of web based educational program was analyzed through multiple regression analysis (Stepwise Regression).
- The reliability of instrument was examined by obtaining Cronbach’s alpha value.
1) Demographics and Characteristics Related to Breast Cancer and Internet Utilization
The demographics and characteristics related to breast cancer and Internet utilization are summarized in Table 1. The average age of subjects is 46.6 years old ranging from 29 to 65. Forty three (29.5%) of the test group replied that they do monthly breast self-examination, and 17.1% percent replied that they never do breast self-examination. Forty one point two percent reported that they participate in breast cancer self-help group and 28.1% replied that they participate at least once every one or two months. Eighty one point nine percent of them replied that they get information of breast cancer through Internet, and 16.0% answered ‘through Internet patient community’ and 6.3% replied that they obtain information through their peers. Also magazines, books and newspaper each served as source of information for 3.5% of the group. Ninety nine point three percent of the patients answered that they use Internet, 65.5% answered that they have been using Internet for more than 3 years, and 22.1% had Internet experience of 1 to 3 years. Sixty percent of them answered that they use Internet more than 5 days a week, and their average time spent on the net was 86.4 minutes ranging from minimum of 12 minutes to maximum of 480 minutes. Eighty six point nine percent answered that they use Internet at home and 95.4% replied that they reference health information available online.
Demographics and Characteristics Related to Breast Cancer and Internet Utilization (N=146)
|Residence||Seoul and big city||88||(60.3)|
|Small and medium-sized city||51||(34.9)|
|Menarche||Before 12 yrs||9||(6.3)||14.4||1.4||11||19|
|After 12 yrs||135||(93.8)|
|Breast self examination||Once a month||43||(29.5)|
|Methods of breast cancer diagnosis||Routine exam||24||(16.4)|
|Period since Dx (yrs)||<1||40||(27.4)||2.1||2.3||0||13|
|Stage of disease at Dx||0||9||(6.2)|
|Type of treatment|
|Participation to self-help group||1/1-2mon.||41||(28.1)|
|Source of information|
|Duration of Internet use||<6 mon||7||(4.8)|
|Frequency of Internet use||Daily||87||(60.0)|
|Duration of Internet use/day (min)||86.4||81.7||12||480|
|Place of Internet use||Home||126||(86.9)|
|Experience of use of health information online||Yes||137||(94.5)|
|Recurrence of breast cancer||Yes||7||(4.8)|
|Metastasis of breast cancer||Yes||10||(6.9)|
|Family History||Mother or mother’s/ father’s sister||5||(3.4)|
2) Satisfaction of Web-Based Educational Program
‘Usefulness of information’ scored average of 3.17 (±0.41) out of 4.0 scoring the highest, and then came the ‘system efficiency’ scoring at 3.15 (±0.53), ‘adequacy of information’ 3.07 (±0.46), ‘system convenience’ 2.99 (±0.45) and ‘design’ was scored the lowest with 2.90 (±0.52). Total score showed a relative high and even distribution of 49.14 (±6.05) out of 64. The lowest scoring subcategory was ‘adequacy of information’ (2.75±0.67) and the highest scoring subcategory was ‘useful information in decision making related to breast cancer patients’ (3.33±0.51) (Table 2).
Satisfaction of Web-Based Educational Program
|System efficiency||Easy to access||146||3.24||0.57||1||4|
|Fast to search||145||3.06||0.60||1||4|
|System convenience||Have contact number||142||2.93||0.57||1||4|
|Easy to use the construct of program||143||3.02||0.54||1||4|
|Friendly to use||144||3.05||0.57||2||4|
|System design||Good design||145||2.90||0.52||1||4|
|Adequacy of information||Correctness||144||3.06||0.56||2||4|
|Usefulness of information||Have lots of breast cancer-related information||144||3.26||0.54||2||4|
|Useful information in decision making related to breast cancer patients||143||3.10||0.49||2||4|
|Easy to understand information given||144||3.17||0.47||2||4|
3) Usefulness of Web-Based Educational Program
In the evaluation of usefulness of web-based educational program, out of 4.0 scale, understanding of breast cancer scored the highest with average of 3.34 (±0.51), then came life after treatment (3.21±0.58), early detection and examination (3.20±0.60), chemotherapy and hormonal therapy (3.18±0.55), related factor and prevention (3.16±0.59), treatment (3.13±0.53), and diagnosis (3.02±0.56) (Table 3). Almost each of the categories of the satisfaction and usefulness scores are correlated from 0.163 to 0.699.
Usefulness of Web-Based Educational Program
|Understanding of breast cancer||145||3.34||0.51||2||4|
|Related factor and prevention||144||3.16||0.59||1||4|
|Early detection and examination||144||3.20||0.60||1||4|
|Chemotherapy and hormonal therapy||144||3.18||0.55||1||4|
|Life after treatment||145||3.21||0.58||2||4|
4) Influencing Factors on Satisfaction of Web-Based Educational Program
By implementing stepwise regression, the factors that affect the satisfaction of web-based educational program are age, religion, income, stage of disease at diagnosis, source of information, duration of Internet usage and whether the patient performs breast self-examination. It showed that subjects in their 40s, Christian, having income of 4 million won and those who used Internet for 3 years or longer showed positive evaluation. On the other hand, if the source of information about breast cancer was magazine, books or newspaper, the evaluation returned negative. The R2 value of this model was 0.296 (F=6.15, P<0.0001) (Table 4).
The Influencing Factors on Satisfaction of Web-Based Educational Program (N=146)
|Age group: 40-49yrs||2.67899||9.89||0.0021*||0.87318||4.4848|
|Breast self-exam: sometimes||-2.2156||7.32||0.0077*||-3.99116||-0.44004|
|Income: 400,000-499,000 W||2.88675||3.96||0.0487*||0.15204||5.62147|
|Stage of disease at Dx: others||-7.67685||10.16||0.0018*||-12.63088||-2.72282|
|Type of treatment: radiotherapy||-2.12141||6.21||0.0140*||-4.02414||-0.21869|
|Source of information: broadcasting||-3.48934||4.62||0.0334*||-6.9034||-0.07527|
|Duration of Internet usage ≥3yrs||2.42706||4.42||0.0374*||0.43095||4.42316|
* P < 0.0001.
5) Influencing Factors on Usefulness of Web-Based Educational Program
By implementing stepwise regression, the factors that affect the usefulness of web-based educational program are religion, period since diagnosis, source of information, frequency of Internet usage, recurrence of breast cancer and family history (R2 = 0.270, F=5.45, P<0.0001). It showed that if the subject is Christian or has other religion, was diagnosed more than one year but less than 3 years ago, rely on Internet as their primary source of information, uses Internet daily, has family history of breast cancer in mother or aunts, then the benefits of web-based educational program were positive. However, in the case of subjects of recurrent breast cancer, the effect was negative (Table 5).
The Influencing Factors on Usefulness of Web-Based Educational Program (N=146)
|Period since diagnosis: longer than 1 year and shorter than 3 years||1.14509||6.89||0.0097*||0.30356||1.98661|
|Source of information: Internet||1.8514||11.21||0.0011*||0.77777||2.92504|
|Frequency of Internet usage: daily||1.03529||5.92||0.0163*||0.16074||1.90984|
|Recurrence of breast cancer: yes||-2.3431||5||0.0271*||-4.40404||-0.28216|
|Family history: mother or parent’s sister||2.51074||4.78||0.0305*||0.23668||4.7848|
* P < 0.0001.
The influencing factors are somewhat different between satisfaction and usefulness, besides the religion and the source of information.
The purpose of this study was to analyze the acceptance level of an Internet website designed to help those with breast cancer. Recently, through the increased in popularity of Internet websites and its information sharing network and emotional support, there have been many new Internet patient communities with lively activity. Along with the increase in usage of blogs, mini-homepage, online clubs and communities, there have been an increased need for consumer’s participation and common sharing. This sheds light on how important it is to take the health information consumer’s needs into account when developing Internet health information website.
When Internet first became popular it was widely thought that majority of Internet users were male demographic who work in offices or have professional careers. However, data shows that women, especially in their 30s and 40s, use Internet more frequently specifically in terms of searching for Internet health related information . To extrapolate this trend, we can predict that the usage of Internet website for breast cancer patient will only increase with time. In this study we showed that women in their 40s, who have higher income and have been using the Internet for more than 3 years, were the most satisfied by the website. This is because highly educated, high income demographics tend to rely more highly on Internet and use it more frequently, and also this group has higher expectation of quality regarding Internet health information. The recent development of Internet including the more user-friendly web 2.0’s debut has added to the previous functions, resulting in higher reflection of health information consumer’s needs and consideration of functional upgrades.
The study showed that the subjects evaluated the websites beneficially, but agrees that there can be more done in terms of the amount of content. Online patient communities that focus on a specific disease tend to demand the most in-depth information available. There are many websites that have general health information on the web, however in terms of the depth, they do not satisfy the demands of health information consumer. Hence, even those online patient communities that are well received need to keep up their quality control and provide contents that are rich in both quantity and quality.
According to one study , there has been frequent usage of Internet health information abroad. However due to language barrier and socio-economical reasons, Koreans living abroad have harder time approaching the medical systems in their residential countries and also if the health information website is in English, it makes it even harder for the Korean speakers which resulted in higher usage of health information website in Korean. Hence, since Internet allows information sharing between all countries, this also allows the Korean immigrants overseas to access the Internet health information websites that are developed within Korea. This shows the usefulness of Korean version of foreign websites and also the websites developed for this study specifically. As the issues around web-based education are not unique to Korean patients, the findings and clinical implication of developing and utilizing the health information websites will be applicable to the users globally. As this website is a sister site to the well-known Breast Cancer website provided in English, it could be extended to the global users by developing the English version of the flash educational materials and posting and linking them to the site.
To conclude, it is highly imperative for the development of educational websites that are sensitive to the needs of health information consumers and especially the online patient communities that are specific to certain conditions.
This study aimed to evaluate the effectiveness of the web based educational program by developing a website for breast cancer patients and analyzing the function of educational program and contents by surveying the women with breast cancer and 147 people replied.
- Each category of Usefulness of information section showed high scores. The highest scoring subcategory was ‘Useful information in decision making related to breast cancer patients’ (3.33±0.51). Out of total 64 points, it showed a relative even distribution of 49.14 (±6.05) points.
- Whether the demographic characteristics influence the educational effectiveness of Web-based educational program was verified. The result showed that all other factors except for the frequency of Internet usage have no relevance, and higher the frequency of Internet usage, the higher the effectiveness of the system scored.
- The factors that influence the satisfaction of web-based educational program are age, religion, whether the subjects perform breast self-examination, income, stage of disease at diagnosis, source of health information, and the duration of Internet usage. The R2 value of this model was 0.296 (F=6.15, P<0.0001).
- The factors that influence the usefulness of web-based educational program are religion, period since diagnosis, source of information, frequency of Internet usage, recurrence of breast cancer, and family history (R2 = 0.270, F=5.45, P<0.0001).
Through this study, there will be many useful improvements in terms of richness of resources and useful functions in the web-based educational program geared towards breast cancer patient in the future. For example, to improve the satisfaction of the users could be accomplished by developing customized contents by age groups and adopting features that are used more conveniently by age specific cognitive and motor skills. The authors would like to have the follow-up research to test whether women who have used the website are better off by measuring whether they are better informed long-term, or assessing whether they check themselves more effectively and so on. The research design would be post-treatment and long-term outcomes evaluation.
This research was supported by a grant from Susan G. Komen for the Cure (#POP0504408).
|||Pelusi J. The lived experience of surviving breast cancer Oncol Nurs Forum 1997; 24: 1343-53.|
|||Hur HK. Information needs of women with breast cancer Korean J Acad Adult Nurs 2000; 12: 286-95.|
|||Kim GY, Hur HK. Information needs of patients with cancer in Korea Korean J Acad Adult Nurs 2002; 14: 135-43.|
|||Yi M, Lee EO, Park YO, Noh DY, Choe KJ. A descriptive study on educational and counseling needs of breast cancer patients based on the treatment stages Korean J Oncol Nurs 2003; 3: 5-14.|
|||Krishna S, Balas EA, Spencer DC, Griffin JZ, Boren SA. Clinical trials of interactive computerized patient education: implications for family practice J Fam Pract 1997; 45: 25-33.|
|||Jung YY. The development of a Internet-based computer assisted instruction program and analysis of its educational effects on patients with coronary artery disease, Doctoral dissertation Seoul,Korea: College of Nursing Seoul National University 2004.|
|||Kraut R, Patterson M, Lundmark V, Kiesler S, Mukopadhyay T, Scherlis W. Internet paradox: A social technology that reduces social involvement and psychological well-being? Am Psychol 1998; 53: 1017-31.|
|||Morahan-Martin J, Schumacher P. Incidence and correlates of pathological Internet use among college students Comput Hum Behav 2000; 16: 13-29.|
|||Mickelson KD. Seeking social support: Parents in electronic support groups In: Kiesler S, Ed. Culture of the Internet. Mahwah, NJ: Erlbaum 1997; pp. 157-78.|
|||Finn J. An exploration of helping processes in an online self-help group focusing on issues of disability Health Soc Work 1999; 24: 220-31.|
|||White MH, Dorman SM. Online support for caregivers: Analysis of an Internet Alzheimer mailgroup Comput Nurs 2000; 18: 168-76.|
|||Bacon ES, Condon EH, Fernsler JI. Young widows' experience with an Internet self-help group J Psychosoc Nurs Ment Health Serv 1992; 38: 24-33.|
|||Cody MJ, Dunn D, Hoppin S, Wendt P. Silver surfers: Training and evaluating Internet use among older adult learners Health Commun 1999; 48: 269-86.|
|||Fernsler JI, Manchester LJ. Evaluation of a computer-based cancer support network Cancer Pract 1997; 5: 46-51.|
|||Klemm P, Hurst M, Dearholt SL, Trone SR. Cyber solace: Gender differences on Internet cancer support groups Comput Nurs 1999; 17: 65-72.|
|||Fogel J, Albert SM, Schnabel F, Ditkoff BA, Neugut AI. Internet use and social support in women with breast cancer Health Psychol 2002; 21: 398-404.|
|||Sharf BF. Communicating breast cancer on-line: Support and empowerment on the Internet Women Health 1997; 26: 65-84.|
|||Fogel J, Albert SM, Schnabel F, Ditkoff BA, Neugut AI. Use of the Internet by women with breast cancer J Med Internet Res 2002; 4: e9.|
|||Ministry of Information and Communication, Republic of Korea. Available at http://www.mic.go.kr/index.jsp 2003 [Accessed January 2,2007];|
|||Ryeu SW, Ha YJ. Usage of health information on the Internet Health and Wellness Policy Forum 2004; 11: 71-87.|