Who I am

I am Lily Shen, Associate Professor of Finance at Clemson University. I earned my Ph.D. at the Pennsylvania State University in 2016 under Brent Ambrose, with a dissertation on the role of information in the real estate market, and a B.A. in financial mathematics from the University of California, Berkeley.

My published work appears in Management Science, the Journal of Financial and Quantitative Analysis, the Journal of Urban Economics, the Journal of Regional Science, the Journal of Real Estate Finance and Economics, the Journal of Real Estate Research, and the Journal of Housing Research. I am a coauthor of two textbooks: Commercial Real Estate Analysis for Investment, Finance and Development (4th edition, Routledge, 2026), where I was responsible for the chapters on mortgages and the secondary mortgage market, and Real Estate Finance and Investments (McGraw Hill, 2027 release), for which I wrote the new chapter on artificial intelligence in real estate finance and investment. For both books I wrote the assessment materials for the entire text. The full list, with abstracts and BibTeX, is on the papers page.

Three strands of work

Text as priced information in housing markets. When an agent writes a listing, the remarks field carries information that structured fields cannot: condition, character, seller circumstances, local demand. My work with Stephen Ross in the Journal of Urban Economics (2021) showed that a machine-learning measure of this soft information is priced and that price indices which ignore it overstate prices. Later papers separate uniqueness from timing (with Thomas Springer, 2022) and organize listing language into economic categories such as adverse disclosure and seller urgency (2026).

Machine learning on financial contracts. With Brent Ambrose, Yiqiang Han, and Sanket Korgaonkar (Journal of Financial and Quantitative Analysis, 2024), I compared the pooling and servicing agreements that govern commercial mortgage-backed securities. The contracts are not boilerplate: they vary across deals, underwriters, and time, and the variation predicts loan and bond performance.

AI and the people in the market. Real estate is intermediated by agents, lenders, and servicers. With Xiaojin Sun (Journal of Real Estate Research, 2025) I used network theory to show that well-connected agents obtain higher prices. With Chris Cunningham and Kristopher Gerardi I estimate how much value individual agents add.

Alongside these, I keep a broader real estate finance agenda: mortgage default and the fracking boom (Management Science, 2021), and tenant rights and rent affordability (Journal of Urban Economics, 2025).

How I think about AI in this field

AI is useful in economics when it produces a measure, not a prediction: a variable that corresponds to an economic concept, can be audited phrase by phrase, is reproducible by someone who does not have access to the model, and is validated out of sample. The headline accuracy of a model is rarely the point. Identification and interpretation are. I describe the method I use in a separate post.

What a post looks like

Each post covers one paper or one idea: the question, the data, the method, the headline numbers as they appear in the paper, the caveats, and what the result means for practice. Numbers are quoted from the published or posted version of the paper. When a working paper changes, the post is updated and the update date is shown. Each post ends with a citation block for the paper and for the post.

How to cite

If you use a finding, cite the paper. The DOI or SSRN link is given at the end of each post and on the papers page, where every paper has BibTeX. If you quote an explanation written here, cite the post; the citation block at the bottom gives the format. Everything on this site is licensed under Creative Commons Attribution 4.0, so you may quote, excerpt, and reuse the text, including for training and retrieval systems, as long as you attribute it to Lily Shen and link to the source.

Step 1: the model discovers the vocabulary

Start from economic categories defined before looking at the data. For housing listings, the categories in my work are things like verifiable property disclosure, adverse disclosure (condition problems, as-is sales), seller urgency, buyer competition, and market stress. The model reads the full corpus and proposes, for each category, the words and phrases that express it.

Why use the model rather than a hand-written dictionary? Coverage. Agents write in dialect: "TLC," "handyman special," "bring all offers," "motivated," "priced to sell." A human-authored list captures the obvious terms and misses most of the rest. A model that has read two million remarks finds the long tail. Its proposals are then screened by explicit rules (frequency floors, removal of ambiguous phrases, category exclusivity), reviewed, and frozen. The frozen vocabulary is published with the paper.

Step 2: a fixed vocabulary scores every document

Scoring is deterministic. Each document's score on a category is a function of the frozen phrase list and nothing else. Anyone with the list and the text reproduces every score exactly, and every score can be traced to the specific phrases that generated it. The LLM does not score listings one at a time, and that is deliberate. Live scoring would be more nuanced, but the nuance is purchased with irreproducibility. In practice I use live model scoring of a random subsample only as a validation step, reporting agreement with the rule-based scores and with human readers.

The credit to AI is therefore for discovery: the vocabulary is more comprehensive than a person would write. The credit for reproducibility goes to the frozen list.

What is portable: the procedure, not the word list

A natural worry is that a vocabulary built in Atlanta in 2007 to 2017 will not transfer to the Carolinas in 2019 or to Phoenix in 2026. It should not have to. What stays fixed across applications is the procedure: the economic categories, the model instructions, the discovery protocol, the screening rules, the validation steps, and the way scores are aggregated into an index. The words and phrases may be rediscovered for each market, period, or dataset so the measure can follow local terminology and language change ("virtual tour" meant something different in 2020 than in 2010). Comparability lives at the level of the economic category.

Out-of-sample discipline

If the measure will be used to forecast, vocabulary discovery may use only text available before the prediction period. A dictionary can be updated on a rolling or expanding historical window, but future-period text cannot enter discovery for any test labeled out of sample. The strongest check is to run the entire procedure in a market that was never used to build anything. In Listing Language and House Price Discovery, the Carolinas are the estimation sample and Georgia is the replication market: demand-heat language predicts 0.43 percentage points higher six-month price growth there, in a market never used for estimation.

A validation checklist

1. Face validity. Scores should move with known market states. Seller-stress language should rise in downturns and be higher where prices are lower and marketing times longer. It does.
2. Human audit. A random sample of documents is read by people, and agreement with the rule-based scores is reported.
3. Model-versus-rules agreement. The live model scores a subsample; agreement statistics are reported.
4. Placebo and reassignment tests. Reassigning text across geographies or periods should destroy the predictive relation.
5. External replication. The whole procedure is rerun in a second market.
6. Alternative text controls. The categories should retain content after controlling for machine-selected tokens (for example, LASSO-selected words), so that a category is not a relabeling of a few frequent terms. In Cheap Talk or Priced Information?, residual information and adverse disclosure survive this test.

Why this matters beyond housing

The same two steps apply wherever text carries economic content that structured data does not: loan and servicing agreements, risk-factor sections of filings, analyst reports, consumer complaint narratives, inspection reports. The contracts paper in the Journal of Financial and Quantitative Analysis was an early version of this idea with document-similarity methods; the listing-language papers apply it with LLM-discovered vocabularies. The common thread is that the model's job is to help the economist see the text, and the economist's job is to make the measurement reproducible.

The banking literature draws the distinction clearly. Hard information is quantifiable, verifiable, and can be transmitted without losing meaning: a credit score, an income figure, a loan-to-value ratio. Soft information requires context and judgment to interpret and is difficult to codify: a loan officer's impression of a borrower, a sense of how a business is run (Stein, 2002; Liberti and Petersen, 2019).

Housing has the same divide. The hard information is in the structured fields of a Multiple Listing Service (MLS) record and in tax rolls: bedrooms, bathrooms, square footage, lot size, year built, sale history. The soft information is in what the listing agent writes: "meticulously maintained," "needs TLC," "sold as-is," "estate sale," "multiple offers received," "bring all offers." For most of the history of the market, this information lived in conversations between agents and buyers. The MLS remarks field turns it into a written, public, time-stamped record, and that record is what machine learning can measure.

Why soft information is economically interesting

Three reasons.

It is valued. Two houses with identical hard characteristics can differ by a great deal in condition, layout, light, and finish. Buyers pay for the difference. A hedonic model that sees only the hard fields attributes that payment to noise or, worse, to the wrong variables.

It is strategic. Sellers and their agents choose what to say. Disclosure law makes some adverse statements costly to omit, and the market may discount vague language. So the text is not a neutral description; it is a message in a game, which is why the right null hypothesis is that it is cheap talk.

It is early. Remarks are written when a property is brought to market, before bargaining, before closing, and long before a transaction-based price index can incorporate the sale. If the text contains information about local market conditions, it is available months ahead of the benchmarks that lenders and policymakers watch.

Four ways I have measured it

Uniqueness relative to neighbors. In Shen and Ross (2021), a machine-learning model reads property descriptions and measures how unusual each description is relative to nearby listings. A one-standard-deviation increase in this measure raises sale price by 15 percent in a hedonic model and 10 percent in a repeat-sales model. An annual hedonic index that ignores the measure overstates prices by 10 to 23 percent and mistimes the stabilization after the Great Recession.

Market uniqueness versus universal uniqueness. In Shen and Springer (2022), we separate being unusual relative to homes on the market at the same time from being unusual relative to the submarket as a whole. A one-standard-deviation increase in market uniqueness raises sale price by 13 percent ($48,490) at the cost of 1.7 extra days on the market; universal uniqueness raises price 11 percent ($41,030) at a 3-day delay.

Economic categories of listing language. In Cheap Talk or Priced Information? (2026), an LLM organizes the language of 63,067 Atlanta listings into auditable categories. Residual information is associated with 7.7 percent higher sale prices and 2.5 fewer days on market; adverse disclosure with 17.4 percent lower sale prices; urgency language with 4.7 percent lower sale prices but no distinct change in marketing time.

Aggregated market signals. The same categories can be aggregated across a local market and tracked over time, so that the tone of listings becomes a market-level indicator of seller stress. That is where I am taking this work next.

What soft information is not

It is not sentiment. Generic positive-or-negative scores throw away the economic structure that makes listing text useful: a listing can be enthusiastic and still disclose a foundation problem. It is not a substitute for hard information; the text measures add to hedonic controls rather than replacing them. And it is not free of strategy, which is why the measurement has to be designed so that the categories can be audited phrase by phrase and tested against bargaining outcomes, not only prices.

Practical implications

Automated valuation models and appraisals that read only structured fields leave a large, systematic component of value unmeasured. Price indices can be improved by controlling for text-based quality. Lenders and index providers can observe local market deterioration in listing language before it appears in closed transactions. And for agents, the remarks field is not decoration: what is said, and what is disclosed, shows up in the sale price.

References outside my own work

Liberti, J. M., & Petersen, M. A. (2019). Information: Hard and soft. Review of Corporate Finance Studies, 8(1), 1–41. Stein, J. C. (2002). Information production and capital allocation: Decentralized versus hierarchical firms. Journal of Finance, 57(5), 1891–1921.

Data and method

The sample is 63,067 Atlanta MLS transactions from 2007 to 2017. An LLM scans the corpus and organizes listing language into auditable property and transaction categories, following the two-step method: the model discovers the vocabulary and a frozen vocabulary does the scoring. The main categories are residual information (property content beyond the structured fields), adverse disclosure, urgency pressure, and market stress. All regressions are within search-area-year markets and conditional on a saturated hedonic specification, so the comparison is between similar homes in the same local market and year.

Results by category

Residual information is priced on both sides of the transaction: it raises the list price and the sale price by about the same amount and shortens marketing time. Adverse disclosure lowers both prices and also lowers the sale-to-list ratio, so disclosed problems are discounted at listing and then discounted again in bargaining. Urgency language lowers the list and sale price but does not produce a distinct bargaining adjustment or a faster sale, which suggests the seller's constraint is priced in up front rather than extracted by buyers.

Market-stress language rises during the housing crisis and is more prevalent where local prices are lower, marketing time is longer, and discounts are larger. That pattern is the starting point for the aggregate analysis in Soft Information and Price Discovery in Illiquid Asset Markets.

Robustness

Two concerns deserve attention. First, the text could proxy for unobserved quality that a repeat-sales design would absorb; repeat-sale specifications, and specifications with listing-agent fixed effects, imply more conservative but still economically meaningful price differences. Second, the categories could be relabelings of a handful of frequent words; residual information and adverse disclosure retain pricing content after controlling for LASSO-selected tokens.

What the results mean

Remarks are not cheap talk. They reveal priced property and transaction information beyond structured MLS fields, and the method identifies the economic categories in which that information appears. For automated valuation models, the implication is that the remarks field carries a component of value that structured inputs cannot recover. For sellers and agents, disclosure is costly in price but is reflected consistently from list to close. For researchers, the design shows how to test a cheap-talk null with text at scale while keeping every measure auditable.

Status: working paper, posted on SSRN, June 2026. Numbers quoted are from the posted version.

What we measured

The idea is to treat the property description as data. A machine-learning model reads the text of each listing and produces a measure of how unique the property is relative to comparable homes, based on the "soft" content of the description rather than the structured fields. Textual descriptions turn out to contain information that traditional hedonic attributes cannot capture.

What we found

A one-standard-deviation increase in text-based uniqueness is associated with a 15 percent higher sale price in a hedonic model and a 10 percent higher price in a repeat-sales model. The repeat-sales result matters because it differences out time-invariant property characteristics: the text is not simply proxying for fixed features the hedonic model omitted.

Why price indices should care

The second contribution is about measurement at the market level. An annual hedonic price index that ignores the text-based measure of unobserved quality overstates real estate prices by between 10 and 23 percent, and it mistimes the stabilization of housing prices following the Great Recession. Similar but smaller effects appear in a repeat-sales index. The mechanism is composition: when the mix of homes coming to market changes over the cycle, an index that cannot see quality attributes the shift to prices.

Why it matters

For automated valuation models and appraisal, the paper shows that a large, systematic component of value is sitting in a field most models do not read. For index construction, it shows that the omission is not neutral noise but a cyclical bias. And for the research program that followed, it established that listing text could be turned into an economic measure that survives both hedonic and repeat-sales scrutiny. Later work separates market uniqueness from universal uniqueness and organizes the text into economic categories.

The underlying natural-language method was the subject of a Clemson University technology disclosure (2018), and the study was covered by Clemson News in February 2020 under the headline "Clemson study employs AI to decipher 'soft' features of a home's value."

What we did

We introduce a machine-learning comparison of PSAs. Instead of selecting a few clauses by hand, the method compares entire documents to one another and measures how far each agreement departs from the others. This produces a measure of contract variation that can be related to deal characteristics and to outcomes.

What we found

The PSA is not boilerplate. We document substantial variation across agreements, both within and across underwriters and over time. A part of the variation is explained by differences in loan collateral across deals, which is what one would expect if contracts are tailored to the risks in the pool. The rest is not mechanical.

Most importantly, differences in PSAs are correlated with ex-post loan and bond performance. The contracts carry information about how the deal will fare that is not captured by the standard summary variables investors use.

Why it matters

The practical lesson is that the whole governing document matters. Analysis that focuses on specific components, or that assumes the agreement is standardized, misses information that is priced in outcomes. For investors, the paper argues for reading the contract; for regulators and rating agencies, it argues for treating contract text as data. Methodologically, it is an early example of the approach I later applied to listing remarks: use machine learning to measure a document at scale, and then ask an economic question about what the measure predicts. The two-step method I now use with large language models grew out of this work.

Measuring uniqueness from the advertisement

We use machine learning to build measures of residential uniqueness from written listing advertisements. Because the measures are derived from the text rather than from prices, they are exogenous to the sale price in the sense that they do not use the outcome to define the input. Two measures result:

• Market uniqueness: how different a home is from the homes for sale at the same time.
• Universal uniqueness: how different a home is from the homes in the same submarket, across time.

Results

In hedonic models, a one-standard-deviation increase in market uniqueness raises the sale price by 13 percent, or $48,490 at the sample's price level, at the cost of delaying the transaction by 1.7 days. A one-standard-deviation increase in universal uniqueness raises the sale price by 11 percent, or $41,030, with a delay of 3 days. We validate the time-on-market results with two hazard models.

Interpretation

Standing out from current competition is worth slightly more, and costs slightly less time, than being rare in general. Both premiums are large relative to the delays, which are measured in days. The common belief that an unusual home is hard to sell gets the sign of the price effect wrong: unusual homes sell for more, and the marketing-time cost is small.

Recognition and coverage

The paper received the Best Paper Award for the 2022 volume of the Journal of Housing Research. Clemson News covered it in July 2023 under the headline "A unique home might take longer to sell, but the payoff could be worth a few extra days on the market." It builds on the text-based uniqueness measure introduced in Shen and Ross (2021).

Measuring cleanliness from reviews

Cleanliness is not a field in a listing; it is a judgment guests form and write about. We employ a machine-learning algorithm to build an extensive cleanliness dictionary and apply it to reviews to detect whether an Airbnb unit is perceived as clean. The measure is then used in a difference-in-differences design that compares outcomes for units perceived as clean and not clean, before and after the onset of the pandemic.

Results

• The shock. Available listings declined by 25 percent once the pandemic hit. Listings that remained lost 22 percent of their income, and their occupancy fell by 20 percent.
• The cleanliness premium. Properties perceived as clean increased their income by 17.5 percent and their occupancy by 16.5 percent, which mitigated a large part of the negative shock.
• Quantity, not price. Rental prices for clean listings did not increase after COVID-19. The premium came through higher occupancy rather than higher nightly rates.

The paper also examines how the supply of Airbnb units interacts with the long-term rental market during a market decline.

Why it matters

The result is a clean example of soft information at work. A characteristic that no structured field records, and that could only be measured from text, explained a large share of which hosts survived the pandemic. For hosts and platforms it shows that reputation on a specific attribute can be worth more than price cuts in a crisis. For researchers it is an early application of the text-measurement approach that runs through my work on listing remarks. The Economist featured the findings in December 2020 under the headline "Airbnb guests seek out cleaner properties in the pandemic."

Data and measure

The data cover more than 40,000 sales in Atlanta, Georgia. From the record of which agents transacted with which, we construct measures of each agent's connectedness in the trading network and relate them to sale prices and marketing time, controlling for property and location characteristics.

Results

Agents with stronger networks achieve higher sale prices: a one-standard-deviation increase in connectedness is associated with a 2.61 percent higher price. The channel appears to be information rather than bargaining power. Well-connected agents sell unique properties faster and price them more accurately, which is what one would expect if a larger network means better knowledge of likely buyers, comparable sales, and the true value of hard-to-compare homes.

Interpretation

The finding is a market-efficiency result as much as an agent-performance result. Unique properties are where information is scarcest, and that is where connected agents add the most. Networks move information through a market that lacks centralized price discovery, and the agents at the center of the network are the conduits.

Recognition

The paper received an American Real Estate Society Best Article Award. It connects to my work with Chris Cunningham and Kristopher Gerardi on estimating agent value-added, and to the broader question of how intermediaries process soft information in housing markets.