Good question. Thiel is very wise to say we overvalorize failure and you don't learn much from it because it's so overdetermined. I think I learned more from the parts of the product/business that worked even if overall it was a failure. But some things: 🧵
It's worth saying WHY we didn't market more given it's such a classic failure mode. Couldn't we have avoided it?
Part of it is we gave away $30 free credits and didn't have any fraud prevention. I figured at this scale losing ~$3k to fraud was worth not losing any real customers or spending dev time.
But actually one user did sign up under nearly 100 names to use these free credits! The loss wasn't huge but it did make our retention look very bad. So we figured we still needed to invest in the product.
But actually our retention was fine e.g. 10% of the users who signed up in January 2025 were still using the product and had cumulatively spend over $400 dollars each. So we probably could have spent $50 on every acquired customer.
Something else I maybe would have changed is launch later. We announced #1 on swe bench lite in July 2024 and also launched our initial product.
But objectively the product was extremely shitty then. It crashed frequently. The models weren't up to the task. Our website was confusing.
It wasn't until January 2025 when we switched to an agent mode with every github number getting it's own continuous agent that I think our product became good.
But most of the ~1000 people who tried our first version never came back
I think those are the only two big "bad decisions" that are clear to me. Possibly I'm blind to other big mistakes we made too. Though obviously we could have executed better and moved faster in every little thing
Notably I think cursor had a better strategy than us shipping a useful product in 2023. We sort of had a useful product then (original mentat cli which was very similar to Aider). But we abandoned it and went straight for async agents in March 2024
But I guess Cognition Labs is succeeding with that strategy so who knows
Maybe our biggest mistake is thinking we could compete in this obviously incredibly valuable market as just 4 people.