PI-QUAL version 2: A Multi-Reader reproducibility study on multiparametric MRI from a tertiary referral center

Purpose: To assess the inter-reader and intra-reader agreement of the Prostate imaging quality version 2 (PI-QUAL v.2) for multiparametric magnetic resonance imaging (mpMRI) among radiologists with varying levels of expertise. Methods: Fifty men underwent 3 T mpMRI scans in a tertiary referral center. Images were anonymized and assessed by six readers of different expertise (2 expert, 2 basic and 2 beginners) in two sessions: first using PI-QUAL v.2, and then using both PI-QUAL v.2 and v.1 after a 2-week interval. PI-QUAL v.2 scores were considered overall and, for comparison with PI-QUAL v.1, dichotomized according to the threshold of acceptable image quality. Gwet AC1 index was used to calculate the inter-reader and intra-reader agreement of the scores. Results: The inter-reader agreement for PI-QUAL v.2 scores was overall moderate (Gwet's AC1 = 0.55), being higher for expert readers compared to the beginner and basic ones (Gwet's AC1 = 0.66 versus 0.45–0-58). Intra-reader agreement varied from moderate to perfect (Gwet's AC1 = 0.43–1.00) and improved with increasing levels of expertise. The ratings were more reproducible for DWI and DCE sequences (Gwet's AC1 = 0.62–1.00) compared to T2w (Gwet's AC1 = 0.24–0.70). The intra-reader agreement between PI-QUAL v.2 and v.1 scores across readings ranged from almost perfect to perfect (Gwet's AC1 = 0.96–1.00). Conclusions: In a tertiary referral center context, PI-QUAL v.2 is a moderately reliable tool for standardizing prostate mpMRI quality evaluations among readers with varying expertise.