Measuring and mitigating racial disparities in LLMs: Evidence from a mortgage underwriting experiment

Revision requested, Journal of Real Estate Finance and Economics

We evaluate LLM responses to a mortgage underwriting task using real loan application data. Experimentally manipulated race is signaled explicitly or through borrower name/location proxies. Multiple generations of LLMs recommend more denials and higher interest rates for Black applicants than otherwise-identical white applicants, with larger disparities for riskier loans. Simple prompt engineering can cost-effectively mitigate these patterns. Race-blind recommendations correlate strongly with real lender decisions and predict delinquency, but LLMs incorporate racial signals when available despite similar delinquency rates across groups. Our findings show potential costs of adopting this new technology in financial settings and raise important questions for regulators.

• Real Estate Finance Manuscript Prize, American Real Estate Society
• Innovative Thinking “Thinking Out of the Box” Manuscript Prize, American Real Estate Society
• Best Paper, 2024 New Zealand Finance Meeting

Revision requested, Journal of Real Estate Finance and Economics.
Bowen, D.E., Price, S.M., Stein, L.C.D., Yang, K., 2024. Measuring and mitigating racial disparities in LLMs: Evidence from a mortgage underwriting experiment (Working Paper No. 4812158). SSRN.
SSRN | PDF