The major difference is they don't have Manchester just over the hill, sucking in the majority of their passengers. The only 'nearby' airport to BRS is CWL which at present is hardly a threat to their growth.
I think the MAN factor is definitely a major factor. However there is far too much leakage over the Pennines. You often have to speculate to accumulate. No venture no gain.
BRS does have other competitors apart from CWL.
BHX takes a decent share of passenger traffic, especially from those living to the north of Bristol (495,000 in CAA 2017 survey) and EXT, admittedly to a lot smaller degree, from those living further south. SOU is also handly for those living on the south-east edge of the BRS primary catchment.
By far the biggest 'predator' though is LHR, although with that airport it's mainly but certainly not exclusively long haul passengers. Over six million passengers used the London airports, mainly LHR, from/to the South West in the CAA 2017 survey. I shall be using LHR soon to go 'Down Under' again. It's really not that inconvenient to reach from the Bristol area when you are flying a long distance so BRS is likely to continue making its money mainly from short haul.
Having said this I can see from the same 2017 CAA survey that 4.839 million passengers used MAN from/to Yorkshire and the Humber (the figures are not broken down any more than that) and perhaps, unlike BRS where the LHR cascading is probably mainly on long haul routes that BRS could not replicate, many of the Yorkshire/Humber passengers using MAN are almost certainly on short haul that LBA could reasonably be expected to accommodate in many instances.
BRS has been fortunate that since privatisiation two decades ago its owners, and ownership has changed more than once in that time, have not stinted on investment with well over £300 million pumped into infrastructure. There is a lot more currently in progress with plans to invest even more in the next few years. The management is looking at 12 mppa by 2025 and 20 mppa by the 2040s. The projection in the 2006 master plan of approaching 10 mppa by 2020 is well on track.
So speculation and accumulation can be good mates in the right airport environment.