Early Stage Startups with Office Presence Grow Faster?
3.5 times faster. That’s a bold claim. It’s also a causal claim — implication is that if company, who chose fully remote setup, would introduce at least some office presence, could speed up its growth significantly.
The estimate is taken from the Work Configuration & Culture Survey
by Reach Capital & NSVF Seed Portfolio, and to be precise, the claim is for pre-seed and seed stage startups. Lets dig a little bit deeper and see how valid is this claim. The question I want to answer is — given the data and estimation method, can we trust this estimate to be causal?
Data
Let’s start with the dataset that this estimate is based on. The analysis is based on a small sample of 37 startups in Pre-Seed/Seed, Series A and Series B+ stages. Of these, 15 are in the Pre-Seed/Seed stage, 8 are attributed to the “no office”, while 7 are in the “some office” bucket.
So straight away we see that the sample is very small. What is more, in the public data presentation there is no information on the distribution for of growth rates. Which is most likely quite wide. Already here it would be interesting to see if these numbers are statistically significant, not even considering validity of causality. We do not have these numbers, so let us look into causal implications.
Causal effect of remote work?
Given that the data is observational (not an experiment) we need to be careful about what claims we can make because of different biases that can dominate our estimates.
Recently I have attended a causal inference course — Mixtape Sessio: Causal Inference I by Prof. Scott Cunningham. Probably the most useful concept that I’ve learned was a causal graphs. A DAG (directed acyclic graph) that encodes data generating process. These graphs are great at helping to identify strategy for average causal effects estimation. They also help to have a discussion about the selected model.
So what model is implied by the simple claim of some-office early stage startups grow 3.5 times faster than no-office early stage startups?
Computing a simple average for two groups implies model where treatment (D)— having some required office time — directly affects revenue growth rate (Y) AND it is a single causal link. If this is a true model, we can use simple averages calculated for two groups as average treatment effects. However, this is clearly too simplistic model. There are a lot more variables at play. Let us add some complexity.
We have added two effects in the above graph — talent availability (T) and attitude towards remote work by the founding team (R). This is a bit more complicated model, but it does not imply that the provided estimate of different revenue growth rates is biased. However, how comfortable we are with the notion that only office presence influences revenue growth? Sounds quite implausible for me. So let us introduce another complication.
This last graph introduces confounding effects — effects that impact office presence and revenue growth. This is a lot more believable DAG. `U` can be anything — from market vertical, company stage (pre-seed and seed are quite different stages in their own right), got-to-market strategy, goals of the startup (especially at this early stage, revenue might not be the most important goals of the company). What is more important, given that `U` is a confounder we have to control for it in order to extract causal effects. If we fail to do that, the estimated effects are biased and it’s impossible to tell the size of it. It might be the case that bias is so large, that it would flip the effect size.
Does office setup directly impact revenue growth?
Another question to ponder is about the direct link between office setup and revenue growth? Is this link direct? Or is it mediated by some other variables? I am leaning towards the latter and there are some data points in the presentation on this. First, let us see the causal graph with mediators
This graph incorporates mediators, through which the office setup can influence revenue growth. We can ignore them if what we are interested is the office setup effect, but non the less it is interesting to see can we find some path that would let us have a bit more confidence in our initial estimate? The survey report include three such variables — average self reported team culture health, average eNPS, and average 2022 regrettable attrition rate. I will include the slide from the report made public below
by Reach Capital & NSVF Seed Portfolio
As you can see, there is little no none difference between two types of office arrangements. The remote arrangement even had lower regrettable attrition rate. So it seems that at least these paths had no influence on the revenue growth. It is not difficult to envision other paths, but we do not have data for those.
Summary
So what have we learned? First, that early stage startups that have some mandated office presence grow their revenue 3.5 times faster compared to their remote counterparts. However, this estimate is based on very small sample (8 remote startups and 7 startups with some office presence). It is also tempered with bias as there are a lot of other factors that influence both office setup and revenue growth, and therefore need to be controlled for.
What I like about this study is that tries to estimate effects and I hope that it can lead to more data gathering and research in this area. I do agree that it is easy to believe that early stage startup would benefit greatly from all the interactions that can and do happen when you are together with your co-workers. However, I think that company structure, communication patterns and attitudes towards remote work impact outcome greatly as well.
