|Year : 2011 | Volume
| Issue : 4 | Page : 451-456
Comparative performance of high-fidelity training models for flexible ureteroscopy: Are all models effective?
Shashikant Mishra, Rajan Sharma, Akhilesh Kumar, Pradeep Ganatra, Ravindra B Sabnis, Mahesh R Desai
Department of Urology, Muljibhai Patel Urological Hospital, Nadaid, Gujarat - 387 001, India
|Date of Web Publication||4-Jan-2012|
Mahesh R Desai
Department of Urology, Muljibhai Patel Urological Hospital, Nadiad, Gujarat - 387 001
Source of Support: None, Conflict of Interest: None
| Abstract|| |
Objective: We performed a comparative study of high-fidelity training models for flexible ureteroscopy (URS). Our objective was to determine whether high-fidelity non-virtual reality (VR) models are as effective as the VR model in teaching flexible URS skills.
Materials and Methods: Twenty-one trained urologists without clinical experience of flexible URS underwent dry lab simulation practice. After a warm-up period of 2 h, tasks were performed on a high-fidelity non-VR (Uro-scopic Trainer TM ; Endo-Urologie-Modell TM ) and a high-fidelity VR model (URO Mentor TM ). The participants were divided equally into three batches with rotation on each of the three stations for 30 min. Performance of the trainees was evaluated by an expert ureteroscopist using pass rating and global rating score (GRS). The participants rated a face validity questionnaire at the end of each session.
Results: The GRS improved statistically at evaluation performed after second rotation (P<0.001 for batches 1, 2 and 3). Pass ratings also improved significantly for all training models when the third and first rotations were compared (P<0.05). The batch that was trained on the VR-based model had more improvement on pass ratings on second rotation but could not achieve statistical significance. Most of the realistic domains were higher for a VR model as compared with the non-VR model, except the realism of the flexible endoscope.
Conclusions: All the models used for training flexible URS were effective in increasing the GRS and pass ratings irrespective of the VR status.
Keywords: Flexible ureteroscopy training, high fidelity, simulators, training
|How to cite this article:|
Mishra S, Sharma R, Kumar A, Ganatra P, Sabnis RB, Desai MR. Comparative performance of high-fidelity training models for flexible ureteroscopy: Are all models effective?. Indian J Urol 2011;27:451-6
|How to cite this URL:|
Mishra S, Sharma R, Kumar A, Ganatra P, Sabnis RB, Desai MR. Comparative performance of high-fidelity training models for flexible ureteroscopy: Are all models effective?. Indian J Urol [serial online] 2011 [cited 2020 May 28];27:451-6. Available from: http://www.indianjurol.com/text.asp?2011/27/4/451/91431
| Introduction|| |
The growth of flexible ureteroscopy (URS) in the Asian subcontinent has not been as fast as its potential appears to be. The factors responsible for this are perhaps the high initial cost and the maintenance cost involved in flexible URS.  It is a well-known fact that the damage to the instrument is more so during the initial learning curve. The high maintenance cost is probably related to the wear and tear of the flexible endoscope. The learning curve reduces with the utilization of a skills laboratory for the human body. The higher the fidelity of a model, i.e. the ability to simulate a more life-like situation, the better it is supposed to be.  High-fidelity simulators are those that are more lifelike, often with the ability to move beyond simple skill or task training and simulate partial or whole operations. Commercially available high-fidelity simulators such as the Uro-scopic Trainer TM ; Limbs and Things and Endo-Urologie-Modell TM ; Karl Storz offer the advantages of reusability, realistic anatomy and the ability to use real surgical instruments such as flexible endoscopes. A relatively new category of simulation, VR, has arisen as a result of significant improvements in computing and graphics capabilities. While expensive and requiring maintenance, VR simulators such as URO mentor TM offer the opportunity to practice basic skills or entire surgical procedures in virtually rendered environments. The cost of a high-fidelity model, especially the VR model, is high and prohibitive.  This enables them to acquire the necessary psychomotor skills and confidence to start the procedure for a real human scenario. Surgical skills laboratory relies on bench models and virtual reality (VR) simulators, which serve as surrogates.  We determined the "closeness to reality" of each model and also whether any had advantages in terms of hastening skill acquisition. For this, we used two high-fidelity non-VR bench models and a VR model.
| Materials and Methods|| |
The inclusion criteria were: A participant with a board-certified urology degree involved in a private practice without association to a teaching institute and willing to learn flexible URS. There were a total of 21 participants interested in the training session. All the 21 subjects received an intensive teaching session involving a total of 3 h regarding the tips and tricks of performing flexible URS before the study day. An initial warm-up period of 2 h was provided to the participants to overcome operative and climatic inhibition. On the next day (study day), all the subjects watched a 20-min power point presentation reviewing the instruments, models and flexible URS video demonstrating the finer tricks involving the psychomotor movements on the two non-VR bench models and a VR model.
High-fidelity non-VR-based bench models [Figure 1] involved for the training included an Uro-scopic trainer TM ; Limbs and Things and Endo-Urologie-Modell TM ; Karl Storz while the VR model was a URO mentor TM (Simbionix, Israel). Both Uro-Scopic Trainer TM and Endo-Urologie-Modell TM consist of a mannequin of the male genitourinary tract through which standard instruments may be passed. There is an obvious advantage as trainees practice with the real-time flexible ureteroscope, which they will be using subsequently in their operating rooms. The trainer allows the user to simulate several endourological techniques, including examination of the urinary tract, stent and guide wire insertion, lithotripsy and stone retrieval. URO mentor TM is a VR-based simulator specifically developed for training in percutaneous renal access and URS. It allows simulation of cytoscopic and ureteroscopic procedures performed using either flexible or semirigid endoscopes. The users interact with the haptic interface device containing flexible endoscopes and ancillary equipments, such as baskets and intracorporeal lithotripsy devices, linked to force feedback mechanisms. Geometric models of urinary tract anatomy and devices used during URS provide tissue-tool interactions. The training consists of choosing different task scenarios of increasing complexity. (Readers can view website for more details; www.simbionix.com/URO_Mentot.html,http://limbsandthings.com/404.shtml)
The study design is as depicted in [Figure 1]. The participants were divided into three batches of seven participants each with rotation on each of the three models. Once the training session of 30 min was over, they proceeded to the next station. In this way, all the participants received a total of 90 min of training. The rotations of each batch were different but the overall time utilized by each batch was 90 min (30 min at each station). The evaluations were carried out at the end of each 30-min session. Each training model was assigned a station. Station A had an Uro-scopic Trainer™, station B an Endo-Urologie-Modell™ and station C a URO Mentor™. A Flex-X™; Karl Storz flexible ureteroscope was used at stations A and B while VR flexible URS (in-built URO Mentor™) was used at station C. Batch 1 rotated from stations A to B to C, Batch 2 rotated from stations B to C to A and Batch 3 from stations C to A to B.
An expert ureteroscopist first performed the simulation exercise at all the stations for a period totaling 3 h. He then devised a specific task for each station of similar complexity assumed to take a similar time. The purpose of doing this was to make the tasks comparable for evaluation. At station A, the task given to the trainee was introducing the flexible ureteroscope into the ureteric orifice without a guide wire across and reaching the mid ureter on both sides sequentially. The task at station B was introducing the flexible ureteroscope into the middle calyx of either kidney sequentially without a guide wire. At station C, a specific task scenario (task no. 9) of stone manipulation in the training module of the URO mentor software was selected. It consisted of passing the simulation-flexible ureteroscope to the middle calyx of the left kidney, retrieving the stone with dormia basket, placing the stone in the upper calyx and fragmenting the stone with holmium laser.
At all the stations, the performance of the trainees was evaluated with a single experienced, non-blinded expert ureteroscopist using a global rating score (GRS) and pass rating. The GRS, adapted from Matsumoto et al.,  was modified as per White et al. to exclude bladder and urethra and standardize the models for flexible URS [Table 1]. The evaluator assigned a value of 1-5 for each of the seven aspects on a GRS. The higher the score, the better was the participant performance. If the participant could perform the desired task, he was given a pass score. Pass rating of the batch at a particular station was defined as percentage of participants passing with respect to total batch strength. Each batch as a whole was evaluated rather than evaluating individuals.
Face validity is defined as the "judgment of novices regarding realism and usefulness of the simulator." The participants rated a standardized face validity questionnaire based on Likert's scale 1-10 at each station to evaluate the realism and usefulness of each training model. Domains of overall realism, flexible endoscope, training model, tissue feel, respiratory movements, urethra and bladder, negotiating ureteric orifice and negotiating endoscope in the ureter and pelvi-calyceal systems were evaluated to assess the realism of the training model. The objectiveness of the face validity results was compared using numerical data on the Likert's scale as provided by the participants.
The results of face validity of the models and GRSs were calculated using Student's t-test. P-value <0.05 was considered statistically significant. The pass rating on each station by the batch was analyzed by the Chi square (χ2 ) significance test for comparing two proportions (with continuity correction). χ2 ≥3.84 was considered significant at the 0.05 confidence level.
| Results|| |
[Table 2] describes the results of the face validity of the three models. Most of the realistic domains were higher for the VR model as compared with the non-VR training models, although these were not statistically significant. The only significant change observed in the VR model was the realism of respiratory movements. One interesting observation made in the face validity results was the higher realism of the flexible endoscope domain in the non-VR-based models.
[Figure 2] shows the task results of Batches 1, 2 and 3, respectively. The GRS improved statistically significantly by the time the second rotation was over. This was observed in all the batches, irrespective of whether it was a VR or non-VR model to start with. There was not much improvement in the GRSs when the second and third rotations were compared; however, it was more marked when the third rotation was compared with the first rotation.
|Figure 2: The global rating scores improved statistically at evaluation performed after the second rotation (<0.001 for batches 1, 2 and 3, respectively). This was observed in all batches, irrespective of the starting training model, whether VR or non-VR model. This was marked when the third was compared with the first rotation (<0.001 for all batches)|
Click here to view
Pass ratings as depicted in [Figure 3] also improved statistically significantly (χ2 > 3.84) for all training models when the third and first rotations were compared. Batch 3, i.e. the batch that was trained initially on the VR-based training model, had more improvement on evaluation performed at the end of the second rotation as compared with the batches that had initial training on non-VR training models, although this did not achieve statistical significance at the 0.05 confidence interval.
Overall, the results show that both the non-VR- and VR-based training models provide almost a comparable level of training skills.
| Discussion|| |
Flexible URS is a relatively new technology, which often requires cognitive specific motor skills by the operator.  These skills have a steep learning curve and are not easily mastered.  It is often difficult for urologists in training and practice to acquire adequate experience because of limited opportunities in the operating room. The simulators offer an opportunity for the trainees to perfect their skills in an inanimate but dynamic model in which anatomical structures are accurately reproduced and the feel of the actual procedure is captured. 
A lot of models on ascending scale of cost are available to offer training modules in a skills lab. VR models have a higher number of individual procedure constructs and offer real life-like situations. They are limited by the exorbitant prices. In developing countries like India, there are only handful skill labs. Even more rare is the availability of VR simulators in these labs. In any training module, if we do away with the high-cost VR model, do we compromise in the training? If not, this may be more cost-effective. To address this issue, we performed the current study.
Before discussing the objectivity of the results, a brief knowledge of the simulators is essential. Simulators can be categorized as low- and high-fidelity simulators depending on the ability to reproduce life-like scenarios. Low-fidelity simulators are those simulators that are not life-like. Advantages of low-fidelity simulators include low cost and portability. The main disadvantages are the lack of realism and reduced number of constructs required for the operation. High-fidelity simulators are those that simulate life-like situations more realistically, often with the capability to move beyond the simple skill or task training and to simulate partial or whole operations. 
We hypothesized that performance of the necessary basic skills in the skills lab improved the cognitive motor skills and boosted the confidence level to start the flexible URS; irrespective of the model status. To study whether it was the impact of training on a specific model or the number of hours of training, the batch as a whole was evaluated. We measured the operative performance by objective structured assessment of technical skills examination, relying on global rating scales for the evaluation of particular tasks and characteristics.  Time comparisons to assess the completeness of the task were checked with pass rating. As expected, the GRS and pass ratings improved with increasing time at each station. The improvement was more marked and statistically significant during the first two bench rotations, irrespective of the bench model. This signifies that tips and tricks are essentially learnt in the early learning phase. The skills acquisition was slow during the rotation from the second to the third model, although the overall improvement was marked when the first and third rotations were compared. This may imply that over the period, the skills acquisition reaches a plateau. As the task is performed repeatedly, the results improve marginally; we may not get statistically significant improvement when comparison of the last few procedures is carried out. Beyond this level, it is only the matter of practice. Batches trained initially on VR model did not show any difference with respect to other models, although the pass rating at rotation 2 was more than that in the other models. This may have been due to a higher number of individual constructs of the real procedure incorporated in the model. The other reason could also be the fact that due to smaller width of the traversing road map on this model, it made subsequent interventions traversing in wider traversing passage in non-VR models easier. The face validity results in our study and the results were similar between all the models. This could have been due to eagerness on the part of novices to learn the procedure, irrespective of its fidelity status. The movement of respiration was rated superior in the VR-based training model. This construct is not incorporated in the non-VR model. The non-VR models have high realism of the endoscope scores as compared with the VR models. One of the possible explanations could be that the novices being impressed by their working with the real endoscope provided high scores.
The limitation of the study includes the short number of participants, an overlap of test-retest reliability in improvement of successive scores, lack of validated instruments for assessing skill acquisition and the unblinded expert reviewer. This could impart few biases in interpreting the results. But, the remarkable differences in the P-value could definitely account for improvement in skills occurring during the practice sessions. The validation instrument, the global rating scale, was altered to account for the lack of the bladder and the urethra. The experienced ureteroscopist was not blinded and, thus, bias could have been introduced. Also, currently, there are no current publications and public acknowledgement on the cost implications of various models imparting training in flexible URS. Even the non-VR models in this manuscript were funded and provided by the sponsors (see acknowledgement). Therefore, the cost of setting up and running each model is largely unstudied. Further studies looking at the cost-effectiveness should be an interesting area of research.
The high-fidelity simulators are significantly more expensive than low-fidelity models with more advantages.  Schout et al. have detailed a comprehensive review of the validation models for ureterorenoscopy.  In general, all the models discriminate between various levels of expertise. Matsumoto et al. found no significant difference between students who trained on the low-fidelity model and those who trained on the high-fidelity model. The low-fidelity model (20$) was produced at less than 1% of the purchase cost of the high-fidelity model (3700$). , Chou et al. examined the ability of two simulators, a high-fidelity bench model (Uro-Scopic Trainer TM ) and a VR simulator (URO Mentor TM ), to teach basic ureteroscopic skills to inexperienced medical students.  The authors found no significant difference between the two groups in their ability to perform the steps of the procedure and concluded that either of these training modalities may improve the initial clinical performance of urological trainees. Matsumoto et al. compared VR simulation with the URO Mentor TM to a high-fidelity bench trainer (Uro-Scopic Trainer TM ) for the assessment of endourological skills.  The authors concluded that performance on the VR simulator was comparable to performance on the high-fidelity bench model and that the URO Mentor TM URS simulator is a useful tool for the assessment of resident performance.
| Conclusion|| |
All the models used for training flexible URS in the current study were effective with respect to increasing the GRSs and pass ratings. It is spending time on the high-fidelity models that matters in acquiring the skills rather than the type of the model. The VR models have the advantage of assimilating a higher number of individual constructs required for the procedure, thereby increasing the confidence level much higher. Also, once the training in the VR model was imparted, subsequent training in the non-VR model increased pass ratings of the batch more than vice versa, but the findings were not statistically significant. The advantage of using non-VR-based models is its higher realism of the real-time flexible endoscope, which makes training on these models attractive to the novices, but the disadvantages are use of fragile flexible endoscopes, maintenance of the endoscope and recurring costs.
| Acknowledgment|| |
The authors would like to thank Karl Storz for lending their armamentarium Endo-Urologie Modell, Uro-scopic trainer and Flex-X scope to be utilized for training without charge.
| References|| |
|1.||Landman J, Lee DI, Lee C, Monga M. Evaluation of overall costs of currently available small flexible ureteroscopes. Urology 2003; 62:218-22. |
|2.||Knoll T, Trojan L, Haecker A, Alken P, Michel MS. Validation of computer-based training in ureterorenoscopy. BJU Int 2005;95:1276-9. |
|3.||Wignall GR, Denstedt JD, Preminger GM, Cadeddu JA, Pearle MS, Sweet RM, et al. Surgical simulation: A urological perspective. J Urol 2008;179:1690-9. |
|4.||Brehmer M, Swartz R. Training on bench models improves dexterity in ureteroscopy. Eur Urol 2005;48:458-63. |
|5.||Matsumoto ED, Hamstra SJ, Radomski SB, Cusimano MD. A novel approach to endourological training: Training at the Surgical Skills Center. J Urol 2001;166:1261-6. |
|6.||White MA, Dehaan AP, Stephens DD, Maes AA, Maatman TJ. Validation of a high fidelity adult ureteroscopy and renoscopy simulator. J Urol 2010;183:673-7. |
|7.||McDougall EM. Editorial Comment. J Urol 2006; 176:2172. |
|8.||Watterson JD, Beiko DT, Kuan JK, Denstedt JD. Randomized prospective blinded study validating acquistion of ureteroscopy skills using computer based virtual reality endourological simulator. J Urol 2002;168:1928-32. |
|9.||Schout BM, Hendrikx AJ, Scherpbier AJ, Bemelmans BL. Update on training models in endourology: A qualitative systematic review of the literature between January 1980 and April 2008. Eur Urol 2008; 54:1247-61. |
|10.||Matsumoto ED, Hamstra SJ, Radomski SB, Cusimano MD. The effect of bench model fidelity on endourological skills: A randomized controlled study. J Urol 2002; 167:1243-7. |
|11.||Chou DS, Abdelshehid C, Clayman RV, McDougall EM. Comparison of results of virtual-reality simulator and training model for basic ureteroscopy training. J Endourol 2006; 20:266-71. |
|12.||Matsumoto ED, Pace KT, D'A Honey RJ. Virtual reality ureteroscopy simulator as a valid tool for assessing endourological skills. Int J Urol 2006;13:896-901. |
[Figure 1], [Figure 2], [Figure 3]
[Table 1], [Table 2]
|This article has been cited by|
||A Novel Cadaveric Simulation Program in Urology
| ||Kamran Ahmed,Abdullatif Aydin,Prokar Dasgupta,Muhammad Shamim Khan,John E. McCabe |
| ||Journal of Surgical Education. 2015; |
|[Pubmed] | [DOI]|
||Flexible ureterorenoscopic management of upper tract pathologies
| ||Athanasios Papatsoris,Kemal Sarica |
| ||Urological Research. 2012; 40(6): 639 |
|[Pubmed] | [DOI]|
||Flexible ureterorenoscopic management of upper tract pathologies
| ||Papatsoris, A. and Sarica, K. |
| ||Urological Research. 2012; 40(6): 639-646 |