We Rebuilt Our Own Firm on AI: What Broke and What Worked
Earlier this year we published a guide to AI tools for real estate developers — what works, what doesn't, and how to implement it. The fair question to ask anyone who writes that guide is whether they actually use the tools themselves, or whether they are describing a capability they read about.
So this is the audit of ourselves. What we automated, what it cost us when it went wrong, and what we still will not hand to a machine.
We are writing it because the failures are more instructive than the wins, and almost nobody publishes them.
What We Actually Automated
Four things, in rough order of how much time they gave back.
Research and market data. Our capital markets and development research pages update themselves weekly. Rate benchmarks pull from public Federal Reserve data, employment and permitting figures refresh from primary sources, and an AI pass drafts the commentary around the new numbers. What used to be a recurring half-day of assembling a market snapshot is now a review task.
Publishing. The article you are reading sits in a pipeline that drafts, fact-checks, and publishes on a fixed cadence without anyone opening a CMS. There is a human review gate before anything goes live and a kill switch that stops the whole thing. Both matter — more on that below.
Deal intake and screening. Lead lists get imported, deduplicated, enriched with contact and company data, and scored against our own criteria before a person looks at them. The scoring does not decide anything. It orders the queue, so the first hour of the week is spent on the twenty prospects most worth calling rather than on sorting a spreadsheet.
Document work. Offering memoranda, loan documents, leases, title reports. First-pass extraction and summarization run through AI; the review is ours. This is the least glamorous item on the list and the one with the clearest return.
Notice what all four have in common: they compress retrieval and drafting. None of them make a decision.
What Broke
Here is the part that usually gets left out of the case study.
A schema default quietly deleted our data. We built version history for the auto-updating research pages so we could roll back a bad generation. The table was wired with a cascading delete — the default choice, and the wrong one. Every time the pipeline replaced a record, the entire version history for that page went with it. The feature had been silently doing nothing for weeks. We only found out when we went to use it.
The lesson is not about databases. It is that automation hides its own failures. A person doing this by hand would have noticed the missing history immediately. A pipeline just keeps running.
A bad deploy took the site down for 35 minutes. A build optimization that looked correct in every local test produced a module loading cycle that crashed the application before it could render. It passed review. It passed a local build. It failed for every real visitor.
What made it survivable was not clever debugging — it was that we could identify and revert the exact change in minutes. What made it happen at all was testing in a warm browser profile, where a cached service worker served the previous working version and masked the breakage completely. We had, in effect, been testing yesterday's site.
An API changed its casing convention and took out 47 data sources. A vendor moved from snake_case to camelCase parameters between versions. Our integration kept sending the old format. The API accepted the requests, returned success, and silently ignored the parameters — so the scraper ran on schedule, reported no errors, and collected nothing.
We shipped a page that downloaded its own stylesheet twice. An optimization inlined our CSS into the page for faster first paint, which meant deleting the stylesheet link. A framework mechanism deduplicated against that link — so with it gone, every visitor silently re-downloaded 223 KB of CSS that was already on the page, and any hiccup on that redundant request threw a visible error. It ran that way for weeks before anyone caught it.
The pattern across all four is the same, and it is worth stating plainly: AI writes code quickly, and the failures do not show up in the code it writes. They show up in the seams — between a schema and its default, a build and a browser, an integration and a vendor's release notes, an optimization and the assumption it broke somewhere else.
If you take one operational lesson from this piece: when you automate a process, you must also automate the check that tells you the process is still working. We did not do that early enough, and every failure above cost us more than the monitoring would have.
What We Refuse to Automate
We are as firm about this as we are about the wins, and it maps closely to the limits we described in the original tools guide.
Entitlement judgment. No model reliably predicts how a council votes, how a neighborhood organizes, or how a particular planner reads a condition. Those outcomes turn on relationships and context that exist in nobody's training data.
Lender and investor relationships. AI can draft a loan package. It cannot carry the credibility of a track record into a credit committee, and it cannot repair a relationship after a deal breaks.
The number that goes in the model. AI is genuinely useful for assembling the inputs to an underwriting — comps, expenses, rate benchmarks. The judgment about which comps are actually comparable, and what happens if the exit assumption is wrong, is the job. That is not a workflow to be compressed. That is the work.
Anything a client receives without a human reading it first. Every automated pipeline we run has a review gate. Not because the output is usually bad — it usually is not — but because "usually" is not a standard you can send to a lender.
What We Would Tell You To Do Differently
If you are a development or investment shop considering this, three things we got wrong that you can skip:
- Map the workflow before you buy the tool. Most firms do it backwards, buy something impressive, and wonder why nothing changed. The audit is the work; the software is the easy part.
- Instrument everything you automate. If a process runs unattended, something has to tell you when it stops producing correct output. Silence is not success.
- Start where the output is reviewable. Document summarization and research drafting are ideal first workflows because a human sees every result and errors surface immediately. Do not begin with something that writes to a database unwatched.
We are roughly two years into this. The honest scorecard: meaningful time returned on document work, research, and deal screening; several self-inflicted outages along the way; and no reduction in the judgment required to do the actual job. That last part is not a disappointment. It is the point.
---
If you want this done to your own firm — the workflow map, the tool decisions, the implementation, and the training — that is exactly what our AI advisory practice does. We will also tell you plainly which of your processes to leave alone.