GRAPHICAL ABSTRACT

INTRODUCTION

The Liver Imaging Reporting and Data System (LI-RADS), which was last updated in 2018, offers a comprehensive framework for standardizing the terminology, technique, interpretation, reporting, and data collection when performing liver imaging for patients with a high risk for hepatocellular carcinoma (HCC), such as those with cirrhosis or chronic hepatitis B virus infection.¹ LI-RADS has gained recognition as the standard for imaging diagnosis of HCC, and has been increasingly adopted worldwide after recent endorsement by the American Association for the Study of Liver Diseases.¹

In the computed tomography (CT)/magnetic resonance imaging (MRI) LI-RADS v2018, LI-RADS category M (LR-M) is defined as observations that are probably or definitely malignant but not specific for HCC, thereby preserving the specificity of HCC diagnosis without compromising the sensitivity for detection of malignancy.¹ Most LR-M category lesions are non-HCC malignancies such as intrahepatic cholangiocarcinomas (CCAs) and combined hepatocellular-CCAs (cHCC-CCAs), whereas 28-29% of LR-M lesions are HCCs with atypical imaging features.^2,3 LR-M criteria include five targetoid imaging features of rim arterial phase hyperenhancement (APHE), peripheral washout, delayed central enhancement (DCE), targetoid restriction, and targetoid transitional phase (TP)/hepatobiliary phase (HBP) appearance, and the three non-targetoid features of infiltrative appearance, marked diffusion restriction, and necrosis or severe ischemia.¹ LR-M is an exclusion criterion, which means that any hepatic observation showing only one targetoid imaging feature can be categorized into LR-M, and therefore, assessing individual LR-M features and assigning LR-M category accurately and reproducibly is important in the context of HCC diagnosis.

A recent multicenter retrospective study reported that agreement was poor for the diagnostic decision of LR-M compared with not LR-M (intraclass correlation coefficient [ICC], 0.46).⁴ However, this study did not provide the agreement for assessments of individual LR-M features, and was therefore of limited value for exploring the causes of poor agreement in the assignment of LR-M. Additionally, although several previous studies reported inter-reader agreement for the assessment of individual LR-M features,^5-13 they were subject to limitations such as a single-center study design, a small number of study subjects, and variable results among studies.

Therefore, we performed a systematic review and meta-analysis of inter-reader agreement in the assessment of individual LR-M imaging features in the CT/MRI LI-RADS v2018 and explored the causes of poor agreement in LR-M assignment.

METHODS

This study was conducted and reported according to the guidelines of the meta-analysis of observational studies in epidemiology¹⁴ and preferred reporting items for systematic reviews and meta-analyses (Supplementary Table 1),¹⁵ and was prospectively registered in PROSPERO (ID, CRD42024505011). Two abdominal radiologists, each with ≥10 years of experience in liver imaging and ≥5 years of experience in performing systematic reviews with meta-analysis, independently conducted the literature search, study selection, data extraction, and assessment of study quality.

Literature search strategy

Comprehensive searches were conducted across the MEDLINE, EMBASE, and Cochrane databases to identify original research articles that reported inter-reader agreement in the evaluation of LI-RADS v2018 LR-M imaging features on dynamic contrast-enhanced CT/MRI. The search query was designed to ensure a sensitive literature search and to minimize the risk of overlooking relevant articles. Key search terms included HCC, LI-RADS, CT, and MRI. Search terms are defined in detail in Supplementary Table 2. The start date of the search period was set to January 1, 2018, to encompass all original articles utilizing LI-RADS v2018, and updates were performed until December 6, 2023. The search was restricted to studies comprising human subjects and those published in English. To broaden the search further, the bibliographies of the selected articles were scrutinized for potentially relevant studies.

Inclusion and exclusion criteria

After removing duplicates, articles underwent a review for eligibility based on the following criteria: (1) population-patients with HCC risk factors defined by LI-RADS,¹ (2) index test: multiphasic CT or MRI, (3) comparator: no specific requirements, (4) outcomes: inter-reader agreement of LR-M imaging features, including five targetoid features (rim APHE, peripheral washout, DCE, targetoid restriction, and targetoid TP/HBP appearance) and three non-targetoid features (infiltrative appearance, marked diffusion restriction, and necrosis or severe ischemia), and (5) study design: any type of study, including observational studies and clinical trials. The exclusion criteria were as follows: (1) animal studies, case reports, review articles, editorials, abstracts/conference proceedings, meta-analyses, and systematic reviews, (2) studies outside the scope of interest of this investigation, (3) studies lacking sufficient data for extracting inter-reader agreement (e.g., absence of standard variance or 95% confidence intervals [CI]), and (4) studies with overlapping patient cohorts. The initial screening involved titles and abstracts, followed by full-text reviews of the selected potentially eligible abstracts.

Data extraction

Each study’s data were extracted using a standardized form prepared using Microsoft Excel (Microsoft, Redmond, WA, USA). The extracted information included study characteristics such as first author, year of publication, and study design (prospective vs. retrospective), subject characteristics, including sample size, sex, age, underlying liver disease, number of hepatic observations, percentages of HCC and/or other malignancy among observations, and observation size, type of imaging modality (CT vs. MRI), MRI scanner field strength, and contrast agent used, details related to imaging interpretation, such as the number of readers, level of experience, and whether readers were blinded to the reference standard, statistical method for evaluating inter-reader agreement, and study outcomes of interest. Inter-reader agreement was determined using kappa values (κ) with 95% CI. When an article lacked sufficient data, the corresponding authors were contacted via email to request additional information or clarification.

Assessment of study quality

The guidelines for Reporting Reliability and Agreement Studies¹⁶ served as the basis for evaluating the quality of the selected articles. The risk of bias was appraised across seven domains: (1) description of the index test (MRI techniques and imaging sequences), (2) the study subjects (recruitment methods and demographic characteristics), (3) the readers (number and experience level), (4) the reading process (availability of clinical information and independent review), (5) blinding to reference standard, (6) statistical analysis, and (7) the actual numbers of subjects and observations. A high-quality score was assigned to each domain if it was sufficiently detailed with no apparent potential bias.

Statistical analysis

The pooled κ values and their 95% CIs for the assessment of eight LR-M features were calculated using the DerSimonian-Laird random-effects model with Knapp and Hartung adjustment,¹⁷ based on the κ and 95% CI reported in the individual studies. κ values were categorized according to Landis and Koch as follows: poor (<0.20), fair (0.21-0.40), moderate (0.41-0.60), substantial (0.61-0.80), or almost perfect agreement (0.81- 1.00).¹⁸ Heterogeneity was assessed using Cochran’s Q test (P<0.1 indicating high heterogeneity) and I² statistic (I² >50% indicating substantial heterogeneity).¹⁹ Publication bias was evaluated using a funnel plot with a rank test. Subgroup analyses were performed using Cohen’s κ as a statistical method.

Subgroup meta-regression analysis with the following covariates was performed to explore the causes of study heterogeneity: (1) proportion of liver cirrhosis (≥50% vs. <50%), (2) dominant etiology of underlying liver disease (hepatitis B virus vs. others), (3) proportion of other malignancy among hepatic observations (≥10% vs. <10%), (4) mean size of hepatic observations (≥3 cm vs. <3 cm), (5) MRI scanner field strength (3.0-T only vs. 1.5-T or both), (6) MRI contrast agent (hepatobiliary contrast agent [HBA] only vs. extracellular contrast agent or both), and (7) number of readers (≥3 vs. 2). The analyses were performed using R version 4.3.2 (The R Foundation for Statistical Computing, Vienna, Austria), with P<0.05 considered statistically significant.

RESULTS

Literature search

Among the 1,029 articles identified through the search, 627 were screened after removing duplicates. Subsequently, 364 articles were excluded based on their titles and abstracts (Fig. 1). After a thorough full-text review, 239 articles were excluded from analysis. Finally, 24 eligible articles reporting the inter-reader agreement of LR-M imaging features, encompassing a total of 25 datasets,^5-13,20-34 were included. Among them, 22 utilized only MRI. The remaining two articles used either CT or MRI, with one presenting the CT and MRI results separately²² and the other presenting them together.⁶

Study characteristics

The characteristics of the included articles are summarized in Table 1. The 24 eligible studies included 4,789 patients with 5,163 hepatic observations. One study was prospective,²⁰ whereas the other studies were retrospective. Three studies exclusively included patients with cirrhosis.^5,9,32 The dominant etiology of liver disease was hepatitis B virus in all but two studies, of which alcoholic liver disease was the dominant etiology in one⁹ and hepatitis C virus in the other.⁶ Twelve studies reported proportions of other malignancy such as CCA or cHCC-CCA that exceeded 10% of total observations.^{5-9,11,12,20,21,25,30,34} Twelve studies utilized only 3.0-T MRI scanners.^{8,11,12,20,21,23-25,27,31,33,34} Eighteen studies employed only HBA including gadoxetate or gadobenate dimeglumine as the MRI contrast agent.^{5,7,9-13,20,21,23,26-29,31-34} In all studies, CT/MRI interpretation was conducted by multiple image readers, with three or more readers participating in two studies.^11,22

Quality assessment

All the included studies demonstrated a low risk of bias in at least six of the seven domains evaluated (Supplementary Table 3). However, five studies were deemed to have a high risk of bias concerning the study subject domain, primarily due to the absence of demographic characteristics such as patient age or the proportion of individuals with liver cirrhosis.^7,8,10,27,28 One study was identified as having a high risk of bias in the readers’ domain because the imaging analysis was conducted by inexperienced readers, specifically radiology residents.³³ Additionally, one study did not provide information regarding the experience levels of the image readers.²⁵ Furthermore, the availability of clinical information during the review process was unclear in two studies.^12,21

Inter-reader agreement for individual features in the LR-M criteria

All 24 eligible articles reported inter-reader agreement for rim APHE.^5-13,20-34 Of them, 20 studies reported inter-reader agreement for peripheral washout,^{5-8,10-13,20-25,27,30-34} 20 for DCE,^{5-12,21-27,29-32,34} 18 for targetoid restriction,^{5-7,9-12,21-25,27,30-34} 19 for targetoid TP/HBP appearance,^{5-12,21,23-27,30-34} 12 for infiltrative appearance,^{5,6,8,9,12,13,20,22,25-27,29} 12 for marked diffusion restriction,^{5-7,9,10,13,24-27,29,30} and 15 for necrosis or severe ischemia.^{5-7,9,12,13,20,22,24-27,29,30,33}

Table 2 and Fig. 2 summarize the inter-reader agreements for the LR-M features. For rim APHE, 24 studies with 5,163 observations reported κ values ranging from 0.39 to 0.94 (Fig. 2A); with a pooled κ of 0.72 (95% CI, 0.65-0.78), indicating substantial inter-reader agreement. For peripheral washout, 20 studies comprising 4,134 observations showed κ ranging from 0.05 to 0.87 (Fig. 2B); with a pooled κ of 0.52 (95% CI, 0.39-0.65), suggesting moderate inter-reader agreement. For DCE, 20 studies with 3,900 observations reported κ ranging from 0.09 to 0.92 (Fig. 2C); with a pooled κ of 0.60 (95% CI, 0.50-0.70), indicating moderate inter-reader agreement. For targetoid restriction, 18 studies with 3,849 observations presented κ ranging from 0.20 to 0.97 (Fig. 2D); with a pooled κ of 0.68 (95% CI, 0.57-0.78), indicating substantial inter-reader agreement. For targetoid TP/HBP appearance, 19 studies with 3,998 observations reported κ ranging from 0.33 to 0.98 (Fig. 2E); with a pooled κ of 0.74 (95% CI, 0.65-0.83), indicating substantial inter-reader agreement. For infiltrative appearance, 12 studies with 2,192 observations reported κ ranging from 0.24 to 0.92 (Fig. 2F); with a pooled κ of 0.64 (95% CI, 0.49-0.78), indicating substantial inter-reader agreement. For marked diffusion restriction, 12 studies with 2,291 observations reported κ ranging from 0.03 to 0.98 (Fig. 2G); with a pooled κ of 0.49 (95% CI, 0.30-0.68), indicating moderate inter-reader agreement. For necrosis or severe ischemia, 16 studies with 3,118 observations reported κ ranging from 0.07 to 0.91 (Fig. 2H); with a pooled κ of 0.61 (95% CI, 0.48-0.73), indicating substantial inter-reader agreement. Additionally, the proportion of studies showing substantial to almost perfect agreement was the highest for targetoid TP/HBP appearance at 76.5% and lowest for marked diffusion restriction at 45.5% (Fig. 3). The results of the subgroup of 21 studies using Cohen’s κ, which were similar to the values obtained for all included studies, are provided in Supplementary Table 4.

Substantial heterogeneity was observed across all LR-M imaging features (Cochran’s Q test, P<0.01; I² ≥89.2%). The features showed no significant publication bias (P>0.05; Table 2 and Supplementary Fig. 1), except for peripheral washout (P=0.04) and necrosis or severe ischemia (P=0.03; Table 2 and Supplementary Fig. 1).

Of the 24 included studies, 22 reported inter-reader agreement for the assessment of MRI LR-M features. One reported combined CT and MRI results, and the other reported CT and MRI results separately. Therefore, the inter-reader agreement for the assessment of CT LR-M features was available in only one study:²² κ=0.70 (95% CI, 0.50-0.91) for rim APHE, 0.32 (95% CI, -0.10 to 0.81) for peripheral washout, 0.68 (95% CI, 0.52-0.85) for infiltrative appearance, and 0.66 (95% CI, 0.56-0.76) for necrosis or severe ischemia.

Subgroup meta-regression analysis

The results of the subgroup meta-regression analyses of targetoid features are summarized in Table 3. In the subgroup analysis of rim APHE, studies with a small observation size (<3 cm) showed significantly lower inter-reader agreement than those with a large observation size (≥3 cm; κ=0.59 vs. κ=0.78, P<0.01). Additionally, studies that used 1.5-T MRI showed poorer agreement than those that used 3.0-T MRI (κ=0.65 vs. κ=0.79, P=0.04). For peripheral washout, studies with a small observation size (<3 cm) exhibited poorer agreement than those with a large observation size (≥3 cm; κ=0.64 vs. κ=0.35, P<0.01). For the features of DCE and targetoid restriction, studies using 1.5-T MRI demonstrated significantly poorer agreement than those using 3.0-T MRI (DCE, κ=0.55 vs. κ=0.78, P=0.01; targetoid restriction, κ=0.49 vs. κ=0.70, P=0.02). For targetoid TP/HBP appearance, studies using 1.5-T MRI showed poorer agreement than those using 3.0-T MRI (κ=0.64 vs. κ=0.82, P=0.02), and studies with multiple readers (≥3) showed poorer agreement than those with two readers (κ=0.33 vs. κ=0.87, P<0.01).

The results of the subgroup meta-regression analysis for non-targetoid features are summarized in Table 4. For infiltrative appearance, marked diffusion restriction, and necrosis or severe ischemia, only MRI field strength was demonstrated as a contributing factor to significant differences in agreement (infiltrative appearance, 3.0-T, κ=0.83 vs. 1.5-T, κ=0.48, P=0.03; marked diffusion restriction, 3.0-T, κ=0.78 vs. 1.5-T, κ=0.39, P=0.04; necrosis or severe ischemia, 3.0-T, κ=0.77 vs. 1.5-T, κ=0.46, P=0.02).

DISCUSSION

For the 24 published studies with 5,163 hepatic observations, the overall pooled inter-reader agreement for the assessment of LR-M features was moderate to substantial. Most LR-M features showed substantial agreement, whereas peripheral washout (κ=0.52) and marked diffusion restriction (κ=0.49) showed moderate agreement. In addition, studies with small observation sizes, 1.5-T MRI scans, and multiple imaging readers were significantly associated with poor agreement for the assessment of LR-M features.

Our results regarding the pooled inter-reader agreement for the assessment of LR-M features compare favorably with the inter-reader agreement for assessment of the LI-RADS major features of non-rim APHE (κ=0.72), non-peripheral washout (κ=0.69), and enhancing capsule (κ=0.66).³⁵ Although LI-RADS has shown excellent performance when used to stratify the probability of HCC based on the assigned categories,³⁶ concerns about imperfect inter-reader agreement using LI-RADS have been raised.³⁷ In particular, poor agreement for the LR-M criteria was reported in both research and clinical reading (ICC=0.46 for both).⁴ This may be important because LR-M lesions usually result in a biopsy for diagnostic confirmation and cannot be classified non-invasively.⁴ Given these imperfect agreements for the LR-M criteria, our study can be useful for understanding the causes of poor agreement because it provides the pooled inter-reader agreement for individual LR-M features and explores the factors associated with poor agreement in LR-M features. Furthermore, since our study included a large number of hepatic observations and involved a robust analysis, it should be helpful in understanding the current status and limitations of the CT/MRI LI-RADS v2018 LR-M criteria.

Of the LR-M features, poor agreement was found for peripheral washout and marked diffusion restriction (κ=0.52 and 0.49, respectively), with these features showing the lowest pooled κ among targetoid and non-targetoid features. LI-RADS has released a lexicon that provides a standardized vocabulary for liver imaging.^38,39 In this LI-RADS lexicon, peripheral washout and marked diffusion restriction are defined as subtypes of washout that are mainly observed at the periphery and have an intensity higher than that of the liver on diffusion-weighted imaging that is not caused only by T2 shine-through, if marked in degree.³⁸ However, these definitions have limitations because several details remain unclear. For example, it is not clear which area should be considered the “observation periphery,” or how “marked diffusion restriction” can be differentiated from “general restricted diffusion.” Furthermore, given the lower frequency of peripheral washout and marked diffusion restriction compared with other LR-M features (3% vs. 44% for rim APHE or 38% for targetoid TP/HBP appearance),¹⁰ it is possible that these two features are less familiar and experienced by radiologists. As all LR-M features are assessed qualitatively, some variability owing to the inherent subjectivity of imaging feature characterization may be unavoidable. Although the adoption of standardized lexicons has promoted clarity and consistency of communication in clinical practice and scientific literature, the situation is not perfect. Therefore, efforts to overcome inter-reader variability, including clearer definitions for qualitative assessment, illustrations of specific imaging features, and education, should continue.

The subgroup meta-regression analysis identified three significant factors associated with poor agreement in the assessment of LR-M features. Notably, for most LR-M features, 1.5-T MRI showed a poorer agreement than 3.0-T MRI. This result can be explained by the fact that 3.0-T MRI has a higher signal-to-noise ratio and lesion-to-liver contrast than 1.5-T MRI.⁴⁰ Furthermore, meta-regression analysis revealed that a small observation size and large number of image readers were significant factors associated with poor agreement for LR-M features. In particular, for small HCC lesions, the definition of the peripheral portion may differ between readers, and ancillary features such as fat and blood products may prevent readers from detecting rim APHE.¹¹ In addition, the interpretation may differ due to bias between readers, which might result from differences in training, experience, and frame of reference among readers. Therefore, as the number of image readers increases, variability in the reader experience is more likely to occur.

Our study has several limitations. First, substantial study heterogeneity was noted, which precludes the creation of solid meta-analytic summary estimates. To overcome this limitation, the causes of study heterogeneity (i.e., the causes of poor agreement) were robustly analyzed by performing a subgroup meta-regression analysis using various covariates. Second, during the second round of screening for eligibility, we excluded 19 articles because of insufficient data for extracting inter-reader agreement (Supplementary Table 5). As the meta-analysis for inter-reader agreement required variance as well as κ from each original study, the necessary information was sought by emailing the authors of studies that reported only the value of κ without variance, and if the variance could not be obtained, these studies were excluded. Third, most studies included in this meta-analysis evaluated LR-M imaging features using MRI, with only one study presenting results using CT. Consequently, drawing conclusions regarding the differences in inter-reader agreement between CT and MRI in LR-M evaluations based on the results of this study is challenging.

In conclusion, substantial inter-reader agreement was found for most LR-M features; however, the agreement for peripheral washout and marked diffusion restriction was limited. Additionally, a small observation size, the use of 1.5-T MRI, and multiple image readers contributed to the poor agreement for LR-M features. Therefore, the LI-RADS should continue to clarify and refine the definitions of individual LR-M features and provide educational programs, including a comprehensive manual with both schematic figures and clinical examples, to illustrate these features.

Article information

Conflicts of Interest

The authors have no conflicts of interest to disclose.

Ethics Statement

Not applicable.

Funding Statement

This study was supported by Korean Liver Cancer Association Research Award 2023.

Data Availability

The data presented in this study are available upon request from the corresponding author.

Author Contributions

Conceptualization: SHC

Data curation: DHK, SHC

Formal analysis: DHK, SHC

Funding acquisition: SHC

Investigation: DHK, SHC

Methodology: DHK, SHC

Project administration: SHC

Resources: DHK, SHC

Software: DHK, SHC

Supervision: SHC

Validation: DHK, SHC

Visualization: DHK, SHC

Writing - original draft: DHK, SHC

Writing - review and editing: DHK, SHC

Approval of final manuscript: DHK, SHC

Supplementary Material

Figure 1.

PRISMA flow diagram of the article selection process. PRISMA, Preferred Reporting Items for Systematic reviews and Meta- Analyses.

Figure 2.

Forest plot of inter-reader agreement for rim arterial phase hyperenhancement (A), peripheral washout (B), delayed central enhancement (C), targetoid restriction (D), targetoid transitional phase/hepatobiliary phase appearance (E), infiltrative appearance (F), marked diffusion restriction (G), and necrosis or severe ischemia (H).

Figure 3.

Distribution of the number of studies by kappa category for each imaging feature. APHE, arterial phase hyperenhancement; DCE, delayed central enhancement; TP, transitional phase; HBP, hepatobiliary phase.

References

• 1. Singal AG, Llovet JM, Yarchoan M, Mehta N, Heimbach JK, Dawson LA, et al. AASLD practice guidance on prevention, diagnosis, and treatment of hepatocellular carcinoma. Hepatology 2023;78:1922−1965.ArticlePubMed
• 2. Kim DH, Choi SH, Park SH, Kim KW, Byun JH, Kim SY, et al. Liver imaging reporting and data system category M: a systematic review and meta-analysis. Liver Int 2020;40:1477−1487.ArticlePubMedPDF
• 3. Hwang SH, Rhee H. Radiologic features of hepatocellular carcinoma related to prognosis. J Liver Cancer 2023;23:143−156.ArticlePubMedPMCPDF
• 4. Hong CW, Chernyak V, Choi JY, Lee S, Potu C, Delgado T, et al. A multicenter assessment of inter-reader reliability of LI-RADS version 2018 for MRI and CT. Radiology 2023;307:e222855.PubMed
• 5. Kim YY, Kim MJ, Kim EH, Roh YH, An C. Hepatocellular carcinoma versus other hepatic malignancy in cirrhosis: performance of LI-RADS version 2018. Radiology 2019;291:72−80.ArticlePubMed
• 6. Ludwig DR, Fraum TJ, Cannella R, Ballard DH, Tsai R, Naeem M, et al. Hepatocellular carcinoma (HCC) versus non-HCC: accuracy and reliability of Liver Imaging Reporting and Data System v2018. Abdom Radiol (NY) 2019;44:2116−2132.ArticlePubMedPMCPDF
• 7. Kim MY, Joo I, Kang HJ, Bae JS, Jeon SK, Lee JM. LI-RADS M (LR-M) criteria and reporting algorithm of v2018: diagnostic values in the assessment of primary liver cancers on gadoxetic acid-enhanced MRI. Abdom Radiol (NY) 2020;45:2440−2448.ArticlePubMedPDF
• 8. Kim SS, Lee S, Choi JY, Lim JS, Park MS, Kim MJ. Diagnostic performance of the LR-M criteria and spectrum of LI-RADS imaging features among primary hepatic carcinomas. Abdom Radiol (NY) 2020;45:3743−3754.ArticlePubMedPDF
• 9. Lim K, Kwon H, Cho J. Inter-reader agreement and imaging-pathology correlation of the LI-RADS M on gadoxetic acid-enhanced magnetic resonance imaging: efforts to improve diagnostic performance. Abdom Radiol (NY) 2020;45:2430−2439.ArticlePubMedPDF
• 10. Jang JK, Choi SH, Byun JH, Park SY, Lee SJ, Kim SY, et al. New strategy for Liver Imaging Reporting and Data System category M to improve diagnostic performance of MRI for hepatocellular carcinoma≤ 3.0 cm. Abdom Radiol (NY) 2022;47:2289−2298.ArticlePubMedPDF
• 11. Min JH, Lee MW, Park HS, Lee DH, Park HJ, Lee JE, et al. LI-RADS version 2018 targetoid appearances on gadoxetic acid-enhanced MRI: interobserver agreement and diagnostic performance for the differentiation of HCC and non-HCC malignancy. AJR Am J Roentgenol 2022;219:421−432.ArticlePubMed
• 12. Zheng W, Huang H, She D, Xiong M, Chen X, Lin X, et al. Added-value of ancillary imaging features for differentiating hepatocellular carcinoma from intrahepatic mass-forming cholangiocarcinoma on Gd-BOPTA-enhanced MRI in LI-RADS M. Abdom Radiol (NY) 2022;47:957−968.ArticlePubMedPDF
• 13. Yang T, Wei H, Wu Y, Qin Y, Chen J, Jiang H, et al. Predicting histologic differentiation of solitary hepatocellular carcinoma up to 5 cm on gadoxetate disodium-enhanced MRI. Insights Imaging 2023;14:3. ArticlePubMedPMCPDF
• 14. Stroup DF, Berlin JA, Morton SC, Olkin I, Williamson GD, Rennie D, et al. Meta-analysis of observational studies in epidemiology: a proposal for reporting. Meta-analysis of observational studies in epidemiology (MOOSE) group. JAMA 2000;283:2008−2012.ArticlePubMed
• 15. McInnes MDF, Moher D, Thombs BD, McGrath TA, Bossuyt PM; The PRISMA-DTA Group, et al. Preferred reporting items for a systematic review and meta-analysis of diagnostic test accuracy studies: the PRISMA-DTA statement. JAMA 2018;319:388−396.ArticlePubMed
• 16. Kottner J, Audigé L, Brorson S, Donner A, Gajewski BJ, Hróbjartsson A, et al. Guidelines for reporting reliability and agreement studies (GRRAS) were proposed. J Clin Epidemiol 2011;64:96−106.ArticlePubMed
• 17. IntHout J, Ioannidis JP, Borm GF. The Hartung-Knapp-Sidik-Jonkman method for random effects meta-analysis is straightforward and considerably outperforms the standard DerSimonian-Laird method. BMC Med Res Methodol 2014;14:25. ArticlePubMedPMCPDF
• 18. Landis JR, Koch GG. The measurement of observer agreement for categorical data. Biometrics 1977;33:159−174.ArticlePubMed
• 19. Higgins JP, Thompson SG, Deeks JJ, Altman DG. Measuring inconsistency in meta-analyses. BMJ 2003;327:557−560.ArticlePubMedPMC
• 20. Jiang H, Liu X, Chen J, Wei Y, Lee JM, Cao L, et al. Man or machine? Prospective comparison of the version 2018 EASL, LI-RADS criteria and a radiomics model to diagnose hepatocellular carcinoma. Cancer Imaging 2019;19:84. ArticlePubMedPMCPDF
• 21. Min JH, Kim SH, Hwang JA, Hyun SH, Ha SY, Choi SY, et al. Prognostic value of LI-RADS category on gadoxetic acid-enhanced MRI and 18FFDG PET-CT in patients with primary liver carcinomas. Eur Radiol 2021;31:3649−3660.ArticlePubMedPDF
• 22. Cannella R, Burgio MD, Beaufrère A, Trapani L, Paradis V, Hobeika C, et al. Imaging features of histological subtypes of hepatocellular carcinoma: Implication for LI-RADS. JHEP Rep 2021;3:100380. ArticlePubMedPMC
• 23. Moon JY, Min JH, Kim YK, Cha D, Hwang JA, Ko SE, et al. Prognosis after curative resection of single hepatocellular carcinoma with a focus on LI-RADS targetoid appearance on preoperative gadoxetic acid-enhanced MRI. Korean J Radiol 2021;22:1786−1796.ArticlePubMedPMCPDF
• 24. Shin J, Lee S, Kim SS, Chung YE, Choi JY, Park MS, et al. Characteristics and early recurrence of hepatocellular carcinomas categorized as LR-M: comparison with those categorized as LR-4 or 5. J Magn Reson Imaging 2021;54:1446−1454.PubMed
• 25. Yoon J, Hwang JA, Lee S, Lee JE, Ha SY, Park YN. Clinicopathologic and MRI features of combined hepatocellular-cholangiocarcinoma in patients with or without cirrhosis. Liver Int 2021;41:1641−1651.ArticlePubMedPDF
• 26. Chen Y, Qin Y, Wu Y, Wei H, Wei Y, Zhang Z, et al. Preoperative prediction of glypican-3 positive expression in solitary hepatocellular carcinoma on gadoxetate-disodium enhanced magnetic resonance imaging. Front Immunol 2022;13:973153. ArticlePubMedPMC
• 27. Liang Y, Xu F, Wang Z, Tan C, Zhang N, Wei X, et al. A gadoxetic acidenhanced MRI-based multivariable model using LI-RADS v2018 and other imaging features for preoperative prediction of macrotrabecularmassive hepatocellular carcinoma. Eur J Radiol 2022;153:110356. ArticlePubMed
• 28. Yang H, Han P, Huang M, Yue X, Wu L, Li X, et al. The role of gadoxetic acid-enhanced MRI features for predicting microvascular invasion in patients with hepatocellular carcinoma. Abdom Radiol (NY) 2022;47:948−956.ArticlePubMedPDF
• 29. Yang J, Jiang H, Xie K, Bashir MR, Wan H, Huang J, et al. Profiling hepatocellular carcinoma aggressiveness with contrast-enhanced ultrasound and gadoxetate disodium-enhanced MRI: an intra-individual comparative study based on the Liver Imaging Reporting and Data System. Eur J Radiol 2022;154:110397. ArticlePubMed
• 30. Hwang JA, Lee S, Lee JE, Yoon J, Choi SY, Shin J. LI-RADS category on MRI is associated with recurrence of intrahepatic cholangiocarcinoma after surgery: a multicenter study. J Magn Reson Imaging 2023;57:930−938.PubMed
• 31. Min JH, Lee MW, Rhim H, Han S, Song KD, Kang TW, et al. LI-RADS category is associated with treatment outcomes of small single HCC: surgical resection vs. radiofrequency ablation. Eur Radiol 2024;34:525−537.ArticlePubMedPDF
• 32. Oh NE, Choi SH, Kim S, Lee H, Jang HJ, Byun JH, et al. Suboptimal performance of LI-RADS v2018 on gadoxetic acid-enhanced MRI for detecting hepatocellular carcinoma in liver transplant candidates. Eur Radiol 2024;34:465−474.ArticlePubMedPDF
• 33. Park JH, Park YN, Kim MJ, Park MS, Choi JY, Chung YE, et al. Steatotic hepatocellular carcinoma: association of MRI findings to underlying liver disease and clinicopathological characteristics. Liver Int 2023;43:1332−1344.PubMed
• 34. Wu H, Liang Y, Wang Z, Tan C, Yang R, Wei X, et al. Optimizing CT and MRI criteria for differentiating intrahepatic mass-forming cholangiocarcinoma and hepatocellular carcinoma. Acta Radiol 2023;64:926−935.ArticlePubMedPDF
• 35. Kang JH, Choi SH, Lee JS, Park SH, Kim KW, Kim SY, et al. Interreader agreement of Liver Imaging Reporting and Data System on MRI: a systematic review and meta-analysis. J Magn Reson Imaging 2020;52:795−804.ArticlePubMedPDF
• 36. van der Pol CB, Lim CS, Sirlin CB, McGrath TA, Salameh JP, Bashir MR, et al. Accuracy of the Liver Imaging Reporting and Data System in computed tomography and magnetic resonance image analysis of hepatocellular carcinoma or overall malignancy-a systematic review. Gastroenterology 2019;156:976−986.ArticlePubMed
• 37. Chernyak V, Sirlin CB. Editorial for “interreader agreement of Liver Imaging Reporting and Data System on MRI: a systematic review and meta analysis”. J Magn Reson Imaging 2020;52:805−806.ArticlePubMedPDF
• 38. American College of Radiology. LI-RADS lexicon [Internet]. Reston, VA (US): American College of Radiology; [cited 2024 Feb 4]. Available from: https://www.acr.org/-/media/ACR/Files/RADS/LI-RADS/LIRAD-SLexicon-Table.pdf
• 39. Fowler KJ, Bashir MR, Fetzer DT, Kitao A, Lee JM, Jiang H, et al. Universal liver imaging lexicon: imaging Atlas for research and clinical practice. Radiographics 2023;43:e220066.ArticlePubMed
• 40. Boll DT, Merkle EM. Imaging at higher magnetic fields: 3 T versus 1.5 T. Magn Reson Imaging Clin N Am 2010;18:549−564, xi-xii.ArticlePubMed

Figure & Data

References

Citations

Citations to this article as recorded by  

• Inter-reader agreement for LR-M imaging features: a premise for better imaging-based diagnosis in liver imaging
  Jaeseung Shin
  Journal of Liver Cancer.2024; 24(2): 124.     CrossRef