How can I avoid the survivor affect when doing backtesting with stocks?

Question

I am trying to write a program to backtest different strategies against historical data, but the problem I am running into is factoring in the affect of turnover, in particular the effect of delisting. For example, nasdaq composite appears to have a delisting rate, with over 400 delistings for at least one year (1999). I am using data imported from yahoo finance but the lists provided there do not include delisted stocks. Does anyone know where I can get similar historical price data and stock lists for delisted stocks? Or alternately, what is the best index with the lowest turnover?

Answer 1

If you use a larger cap index, you will reduce the number (but not eliminate) of delisting.
Fewer mid and large caps get delisted than small caps.

You could go S&P 500 and that would give you smaller sample size than NASDAQ, but a lower amount of delistings.

Answer 2

If you are looking for professional-quality data, you have to be prepared to pay professional-level price. Typically, researchers use either CRSP (Center for Research of Security Prices) or Compustat data.

As to indexes with low turnover, check out S&P 500 and Russell 1000.

Answer 3

you can get this information from the CRSP database. Most academic studies are done using these data.

WRDS (Wharton Research Data Services) at U of PA has these data. It costs money to get it -- but if you are at a university -- it is likely that your finance department subscribes. If you are out od school, but have an MBA, then you may be able to access it through the school's library.