If you don't remember your password, you can reset it by entering your email address and clicking the Reset Password button. You will then receive an email that contains a secure link for resetting your password
If the address matches a valid account an email will be sent to __email__ with instructions for resetting your password
The COVID-19 pandemic caused a shift from in-person care to telemedicine, providing a unique opportunity to evaluate trends and efficiency of telemedicine usage within surgical subspecialties in a large, integrated health care system before and after shelter in place mandates.
This retrospective cohort study included all of the Kaiser Permanente Northern California members referred to surgical services from January 1, 2019 to June 13, 2020 and receiving a surgical procedure. We compared the patient referrals (categorized as benign, urgent, or cancer) before and after shelter in place mandates, and we examined rates of telemedicine (video or telephone) usage for preoperative consultations, postoperative visits, time from referral to first surgical encounter, and to surgery or procedure. In multivariate analyses, we assessed the patient and provider characteristics associated with telemedicine usage.
There was a total of 34,875 surgical referrals resulting in a procedure, with a significant decline in referral after shelter in place mandates. Preoperative encounter types shifted from 89.8% in-person before shelter in place mandates to 70.2% telemedicine after shelter in place mandates (P < .0001). The median time from referral to first encounter decreased after shelter in place mandates, as did median time to procedure. After shelter in place mandates, postoperative encounters were mainly telemedicine (65.8%) compared with before shelter in place mandates (41.7%) (P < .0001). Overall, there was a comparable uptake of telemedicine usage in almost all evaluated categories of patient characteristics after shelter in place mandates.
Within a health care system with prior telemedicine capability, surgical specialties were able to shift to telemedicine rapidly, equitably, and efficiently in the preoperative and postoperative encounters of benign, urgent, and cancer diagnosis during mandated COVID-19 restrictions.
Telemedicine has the potential to provide high-quality, rapid, and convenient communication between providers and patients.
However, significant obstacles have hindered widespread adoption of telemedicine, including access and comfort with online technology, cost of developing new infrastructure, Health Insurance Portability and Accountability Act–compliant platforms, and reimbursement for telemedicine services.
On March 11, 2020, the World Health Organization declared COVID-19 a global pandemic. Shortly after, on March 19, shelter in place (SIP) orders were implemented in California, leading to a shift in health care practices. The SIP orders became a catalyst for the accelerated implementation of telemedicine. Our integrated health care delivery system already had an infrastructure in place supporting telemedicine, so all service lines moved to a “video first” strategy for patient care.
Because surgeons uniquely rely on physical intervention for treatment, an in-person exam is traditionally viewed as a key component of patient evaluation and treatment planning.
Therefore, the change in practice offered a singular opportunity to evaluate telemedicine usage in surgical subspecialties. Previous publications describing the use of telemedicine in surgical specialties showed that telemedicine is an acceptable alternative to outpatient care in select settings, such as preoperative or postoperative visits or in a specific subspecialty.
Only one study evaluated large-scale trends of telemedicine conversion during COVID-19 pandemic for surgical visits, and although this study found an increase in telemedicine usage, the overall uptake rate of telemedicine visits remained low, suggesting unidentified barriers to widespread telemedicine adoption in surgical care.
Our study evaluated the implementation and use of telemedicine in a large integrated health care delivery system, before and after SIP, and included a variety of surgical services.
Kaiser Permanente Northern California (KPNC) is an integrated health care delivery system serving >4.7 million members, with 262 medical offices and 21 medical centers. It covers >30% of the population in the counties where it has a physical presence. Membership is racially and ethnically diverse and represents the demographics of the northern California population, except at the extremes of income.
However, in the surgical departments only telephone visits were commonly scheduled, and these appointments were mainly for routine uncomplicated postoperative patients. This practice aligned with recognized patient preferences.
When the pandemic SIP mandate went into effect, all of the service lines moved to a “Video first” strategy for evaluating patients. This meant all referrals to surgery, including new consultations, established patients, and postoperative appointments, were scheduled as a video visit. Clinic visits were still available with physician recommendation.
Given KPNC’s longstanding institutional commitment to telemedicine, our organization had all eight recommended essential components for successful telemedicine adoption: an existing electronic health record (EHR), training sessions for providers and staff, patient education on accessing the portal, available hardware such as smartphones and video-capable computers, integrated billing and coding functions, information technology support, a Health Insurance Portability and Accountability Act–compliant audiovisual platform, and patient and caregiver buy-in.
As with in-person visits, language translation services were available for all of the telephone and video visits.
This retrospective cohort study included all of the KPNC members referred to a surgical service from January 1, 2019, through June 13, 2020. An extended period before the pandemic was chosen to illustrate the steady state of prepandemic surgical practice. Our aim was to assess patient and provider characteristics associated with telemedicine uptake, including variables such as surgical subspecialty and surgical acuity (benign, urgent, and cancer), and to determine the predictors for and barriers to telemedicine usage.
We defined telemedicine to include both telephone and video-based outpatient clinical encounters. All of the video-based encounters were in real time with audio and visual connection.
For the patients who physically presented to the office for an evaluation, we termed this as in-person encounter. We used the data from the study organization’s comprehensive EHR (Epic, Verona, WI) to examine surgeries before and during the initial pandemic period and to assess the patient and provider characteristics, rates of telemedicine use for preoperative consultations and postoperative visits, time from referral to first surgical encounter, and time from referral to surgery or procedure.
The pre-SIP and post-SIP cohorts were defined based on the dates of referral initiation and preoperative and postoperative encounters. If all of the encounters occurred before March 19, 2020, then the incident was defined as pre-SIP. If all encounters occurred after March 19, 2020, then the incident was defined as post-SIP.
The patients who were not enrolled in the health plan for 12 months before referral or 6 months after referral were excluded to ensure the complete capture of patient history. Only surgical referrals that resulted in a procedure were included. The surgeries were confirmed using the presence of an operative note or surgical pathology report. We categorized the patients’ surgical diagnoses into 3 categories (benign, urgent, and cancer) based on the documented referral problem reason. Ten surgical specialties were included: bariatric, colorectal, general, OB-GYN, OB-GYN oncology, oncologic, plastic, thoracic, trauma, and vascular surgery. Because not all of the patients referred had proceeded to surgery, fixed study follow-up periods were designated to consistently capture >90% of eligible patients, allowing for 60 days from referral to surgeon consultation (preoperative encounter), 90 days from preoperative encounter to surgery, and 30 days for capture of postoperative encounters.
The encounter and procedure information and data on cohort characteristics were obtained from the KPNC HealthConnect EHR databases. The patient characteristics studied included their age at time of preoperative encounter, sex, race/ethnicity, primary language, patient portal access, residential address as of their encounter dates using United States 2010 census block group, and socioeconomic status; categorized as above or below 200% poverty level and above or below 250% education level.
The provider characteristics included surgeon age, sex, and specialty.
Descriptive statistics were used to examine the distribution of the demographics, clinical characteristics, and provider characteristic between pre-SIP and post-SIP referral cohorts, using median and IQR for the continuous variables and frequencies, and percentages for the categorical variables. The median time (in days) were compared between the pre-SIP and post-SIP referrals by using nonparametric tests. The main study outcome was defined as a telemedicine preoperative encounter versus in-person preoperative encounter. Stratifying by SIP status, we used multivariate logistic regression to examine patient and provider characteristic and diagnosis category associated with telemedicine encounters (compared with in-person office visits) while controlling for physicians clustering among service areas and patients among physicians. We also used the models to calculate average marginal effects to quantify the adjusted impact of each dependent variable on rates of telemedicine. The SAS statistical software version 9.4 (SAS Institute, Cary, NC) was used for all statistical analyses. All of the statistical tests were 2-sided.
During the study period, there were a total of 34,875 surgical referrals resulting in a procedure. The number of surgical referrals decreased from 508 per week pre-SIP to 186 per week post-SIP. As shown in Figure 1, office visits declined sharply after SIP, and then gradually rose but remained significantly lower than the pre-SIP levels.
Preoperative telemedicine encounter types increased after SIP for all 3 categories of surgical acuity: benign, urgent, and cancer (Table I). The benign diagnoses had the largest increase of telemedicine encounters, rising from 8.8% pre-SIP to 75.3% post-SIP (P < .0001), followed by cancer diagnoses, going from 1.2 8% to 64.9% (P < .0001) and urgent diagnoses 11.7% to 62.9% (P < .0001).
Table IEncounter type pre-SIP and post-SIP: Preoperative and postoperative
Preoperative telemedicine visits shifted from mainly telephone to significantly higher use of video visits after SIP. Pre-SIP, video visits accounted for <1% of all telemedicine visits, whereas after SIP video visits made up 37.5% of all preoperative encounters (P < .0001). The video visits were more commonly used for benign diagnoses (47.3%) than for cancer (26.5%) and urgent (24.7%) diagnoses.
Median time from referral to first encounter decreased post-SIP for all of the encounter types and acuity (P < .05) (Table II). For both telephone and video visits, this interval decreased by several days (5 to 2 days, 5 to 3 days, respectively (P < .0001, P < .0001), compared to only a 1-day improvement in access for in-person encounters (6 to 5 days, P < .0001).
Table IIDays from referral to first presurgical encounter and to surgery/procedure
Median days (IQR)
Median time from referral to first encounter
5 (2, 9)
5 (1, 11)
4 (1, 10)
7 (5, 13)
5 (3, 8)
6 (1, 11)
5 (1, 8)
2 (1, 5)
5 (2, 10)
2 (0, 5)
3 (0, 8)
3 (1, 6)
4 (1, 7)
3 (1, 6)
3 (1, 6)
6 (3, 10.5)
2 (1, 5)
4 (1, 6)
0 (0, 0)
5 (3, 7)
Median time from referral to surgery
30 (15, 52)
21 (9, 40)
35 (15, 58)
28 (10, 53)
23 (15, 36)
15 (7, 27)
33 (17, 53)
27 (13, 45)
33 (20, 52)
27 (14, 50)
35 (19, 57)
38 (15, 58)
28 (19, 41)
20 (13, 35)
37 (22, 57)
27 (16, 54)
42.5 (23, 64)
27 (14, 49)
45.5 (24, 64)
34 (15, 58)
21 (21, 29)
18 (12, 28)
35 (35, 35)
27 (19, 47)
Shaded boxes represent cells with ≤5 cases in which estimates are unstable.
Nonparametric tests have been performed for pre-SIP and post-SIP.
The median time from referral to surgery or procedure decreased post-SIP for all of the encounter types and acuity. Overall, the in-person visits had the shortest time from referral to surgery, both pre-SIP and post-SIP (30 days and 21 days, P < .0001). Telephone and video visits had a pre-SIP median time from referral to surgery of 33 days and 42 days, but this time interval decreased in both categories to 27 days post-SIP (P < .0001; P < .0001).
The percentage of patients with documented postoperative encounters after a surgery or procedure was not statistically different (60% pre-SIP and 66% post-SIP, P = .465).
Use of telemedicine visit for postoperative encounters was relatively frequent pre-SIP, accounting for 41.7% of all visits, and rose to 68.5% post-SIP (P < .0001) (Table I). Within the pre-SIP postoperative encounters, telemedicine was the predominant visit type for benign diagnoses (54%), whereas cancer and urgent cases more frequently used in-person postoperative visits (74% and 72%). Post-SIP, telemedicine became the most common postoperative encounter type (P < .0001), again more frequent for benign diagnoses (74.5%) compared to cancer (65.5%) or urgent (58.7%) diagnoses.
Within postoperative telemedicine visits, video encounters showed an increase going from 2.4% pre-SIP to 22.2% post-SIP. Video visits were more common for postoperative benign diagnoses (24%), compared with cancer (22.2%) or urgent encounters (18%) post-SIP.
The patients pre-SIP and post-SIP were of similar age, sex, Charlson comorbidity index, race/ethnicity, language preference, and SES (Table III), and the majority of the patients had access to the patient portal both pre-SIP (89.6%) and post-SIP (92.4%).
Table IIIPatient characteristics, pre-SIP and post-SIP
Total (n = 34,467)
Benign (n = 20,843, 62% of total)
Cancer (n = 8,636, 25% of total)
Urgent (n = 4,296, 13% of total)
Age, median, y (IQR)
58 (45, 69)
59 (45, 70)
53 (40, 65)
54 (40, 66)
66 (56, 73)
65 (54, 73)
61 (47, 71)
62 (48, 72)
Low neighborhood SES
Patient portal access
Prior telemedicine experience
Other Race/Ethnicity includes patients in multiple races, Native Hawaiian/Pacific Islander, American Indian/Alaskan Native, or unknown. Low neighborhood socioeconomic status is based on the patient’s neighborhood socioeconomic status, defined as high poverty and low education residential neighborhood based on 2010 census data in the census block group of the patient’s residence. Prior telemedicine experience means any telephone or video visit experience 12 months before e-consult initiation date. This analysis excluded critical care surgery specialty.
The pre-SIP patient characteristics statistically significantly associated with telemedicine usage included age <40 (P < .0274), female sex (P < .0009), English language preference (P < .0051), low SES (P < .0001), and prior telemedicine usage (P < .0001) (Table IV). Post-SIP, only race/ethnicity entered as “other” was statistically significantly associated with increased telemedicine usage (P < .0048). Overall, there was good uptake of telemedicine usage in almost all of the evaluated categories of patient characteristics after SIP.
Table IVSurgery category and patient characteristics associated with presurgical telemedicine visit type: adjusted analyses stratified by pre-SIP and post-SIP time periods
The pre-SIP provider characteristics statistically significantly associated with telemedicine usage included age >60 (P < .001) and male sex (P < .0004) (Table V). Trauma and GYN oncology providers had the lowest telemedicine usage. Colorectal surgeons and vascular surgeons also tended to use telemedicine less frequently than other specialties. Post-SIP, age >60 remained a significant predictor of telemedicine usage by a surgical provider (P < .0146), but telemedicine usage was similar for male and female providers. There was increased telemedicine usage in all subspecialty categories, with GYN oncology and colorectal surgery continuing to be the least likely to use telemedicine.
Table VProvider characteristics associated with presurgical telemedicine visit type: adjusted analyses stratified by pre-SIP and post-SIP time periods
OR (95% CI)
Adjusted percent telemedicine
OR (95% CI)
Adjusted percent telemedicine
OB/GYN, obstetrics and gynecology; SIP, shelter in place.
To our knowledge, our study was the largest and most detailed assessments of telemedicine usage in surgical subspecialties during the COVID-19 pandemic. We found a rapid shift to telemedicine across a spectrum of surgical diagnoses and acuity, without significant identifiable provider or patient barriers.
evaluating surgical subspecialty implementation of telehealth in Michigan during the COVID-19 pandemic, our study found prepandemic use of telemedicine was rare in surgical practices, but telemedicine usage grew significantly after COVID restrictions, followed by a decline. Unlike the Michigan study, which found that in-person visits contributed to the majority of new surgical visits, telemedicine became the predominant visit type for our patient population. Before COVID-19, Nandra et al
published a single-institution successful implementation of telemedicine attributing their success to having a dedicated telehealth coordinator, a smartphone app, and limiting the option to specific visit types. Most likely a pre-existing telemedicine infrastructure in our institution, as well as training to onboard patients, facilitated the transition from mainly in-person visits to telemedicine visits.
Examining surgical acuity, we found benign diagnoses consistently had the highest telemedicine usage, both for preoperative and postoperative visits, and cancer diagnoses the lowest telemedicine usage (P < .0001). It is most likely that prepandemic routine use of postoperative telephone visits for uncomplicated surgeries had normalized telemedicine for the benign diagnoses subset, which generally has had low complication rates and standard recovery patterns. Other studies also found that for uncomplicated patients, telemedicine follow-up as a replacement for an in-office visit provided a high level of patient satisfaction with no differences in health outcomes.
For more complex diagnoses, such as cancer, both the need for good communication before surgery to establish rapport and trust and physical exams to guide surgical planning likely lowered the adoption of telemedicine in this population.
Pre-SIP, the patients who used telemedicine were statistically more likely to be <40 years old, female sex, English language preference, low SES, and have prior telemedicine usage. In contrast, with expanded use of telemedicine post-SIP, only race/ethnicity listed as “other” was statistically significantly associated with increased telemedicine usage (P < .0048). Other studies showed that there continued to be disparate usage of telemedicine by patients after COVID-19; specifically, the patients who were male, older, and lived in a rural or below-median income ZIP code were less likely to use telemedicine.
Some of the differences in telemedicine adoption may be owing to a relatively limited “digital divide” in technology-oriented northern California and a high rate of patient portal access. Interestingly, the patients’ prior experience with telemedicine was associated with ongoing telemedicine usage, which suggests that once exposed to telemedicine, the benefits and convenience are significant from a patient’s perspective.
Within providers, certain subspecialties (GYN oncology, colorectal, and trauma) showed significantly lower use of telemedicine compared with other subspecialties (P < .001, P < .006, P, 0.001), whereas surgical oncology and thoracic surgery showed significantly higher usage (P < .001). Both GYN oncology and colorectal surgeries require a physical exam that cannot be replicated over telephone or video. Two Italian studies published a consensus among expert colorectal surgeons arguing against telemedicine for surgical decision making and recommending an in-person exam to avoid a cancer misdiagnosis.
the plastic surgeons in our organization have since developed patient education tools outlining how to upload standard photos of sensitive body areas (ie, breast, abdomen) to facilitate a remote consultation. In contrast, thoracic surgery demonstrated the highest adjusted percentage of telemedicine use (86.5%, P < .0003), which may reflect this specialty’s heavy reliance on imaging findings and lung function tests, rather than physical exam findings for surgical planning. Overall, these results have suggested that physical examination as a part of the surgical evaluation may be more flexible than previous dogma, and some of the aspects of physical exam can be replicated through a telemedicine interface.
This study had several limitations. Because this is a retrospective, observational study, we could not fully account for selection bias or determine if telemedicine prevented patients from seeking care. National data indicate that 60% to 70% of patients deferred care during the pandemic.
The decrease in surgical referrals is likely related to multiple factors, but avoidance of medical care, cessation of screening procedures (ie, mammograms, Pap smears, and colonoscopies), and fewer nonurgent surgical referrals were probably the largest contributors. We were also not able to directly study the efficiency of telemedicine based on number or type of preoperative encounters. However, in a smaller KPNC study evaluating breast cancer care during COVID-19 restrictions, the number of visits before breast surgery was not significantly impacted.
Another limitation is that patient and provider satisfaction with the process was not measured. Although Barsom et al found that surgeons were highly satisfied with video consultation for postoperative colorectal care,
Kemp et al surveyed a variety of surgical providers at the University of Michigan during COVID-19 and found a lower satisfaction with telemedicine for the ability to perform physical exams and to break bad news.
In KPNC, a similar survey of >200 multidisciplinary cancer care clinicians post-SIP showed that most providers were satisfied with telehealth and wished to maintain or increase future usage; however, almost all clinicians believed that an in-person visit promoted a strong clinician-patient connection.
This finding was mirrored in a study done by Grenda et al examining the transition to telehealth in a thoracic surgery practice during COVID-19, citing building rapport and trust as limitations of virtual visits.
Despite this shortcoming, 65% of KPNC cancer providers still thought that a telemedicine visit was sufficient for discussing a new cancer diagnosis, but providers again cited an inability to perform a physical exam or assess performance status as limitations.
We were also not able to measure outcomes. However, in a randomized trial comparing video based or in-person postoperative visits for laparoscopic appendectomy or laparoscopic cholecystectomy, the primary outcome of a 30-day hospital encounter was noninferior.
In a larger narrative review of the use of telemedicine in surgical clinics, 7 studies showed no statistical difference in complication rates for low-risk surgical patients who had an in-person follow-up compared to a telemedicine visit.
In addition, our organization’s capitated model and EHR-integrated telemedicine practice may not generalize directly to other payment models for surgical practice or standalone telehealth models that are not as well integrated with patients’ ongoing medical history and documentation, and fee-for service reimbursement models may limit the ability of some practices to implement changes based on our findings.
Although we did not find differences in telemedicine use across patient characteristics in our large and diverse population, further research is needed to continue to examine the equitable use of telemedicine.
Finally, although COVID led to significantly greater use of telemedicine, our post-SIP time period was limited, and additional postpandemic studies are needed to evaluate telemedicine as an acceptable and sustainable alternative in our daily surgical practices.
In conclusion, within an integrated health care system with preexisting infrastructure for telemedicine, surgical subspecialties were able to implement telemedicine rapidly and equitably in the care of patients post-SIP compared to pre-SIP. The patients who had a referral placed to surgical specialties post-SIP received timely care, suggesting that telemedicine is a useful tool for ongoing surgical care, even after the COVID-19 pandemic has ended.
This project was supported by the Permanente Medical Group’s Delivery Science and Applied Research program. None of the authors have received any outside funding.
Conflict of interest/Disclosure:
The authors have no conflicts of interests or disclosures to report.
Patient-provider telemedicine integrated with clinical care: patient experiences.