Factor Testing: IC, Decile Backtesting, and Turnover

Layer 1: The Intuition

1.1 The Weather Forecaster's Report Card

To evaluate a weather forecaster, you don't just look at whether they were right on one specific day, but rather:

• Average Accuracy: How correlated the forecasts are with actual weather over the long term;
• Stability: Whether they are consistently accurate, or occasionally spot-on but often wildly wrong;
• Reasonableness of Categorization: Whether days they predict as having a "70% chance of rain" actually have more rain than days they predict as "30%".

Factor testing does the exact same thing: IC measures "average accuracy," ICIR measures "stability," and decile backtesting checks "whether the categorization is reasonable."

1.2 Why a Correlation of 0.05 is Considered Good

In daily life, a correlation coefficient of 0.05 is almost equivalent to no relationship at all. However, in the stock market:

• There are hundreds or thousands of stocks available each period to bet on simultaneously;
• There are dozens to hundreds of periods per year to repeat the bets;
• As long as this slight edge exists stably, multiplying it by breadth and frequency can accumulate significant returns.

This is the intuition behind the "Fundamental Law of Active Management" proposed by Grinold:

$$\text{IR} \approx \text{IC} \times \sqrt{\text{Breadth}},$$

The Information Ratio is approximately equal to the "accuracy of each prediction" multiplied by the "square root of the number of independent bets."

Layer 2: Three-Piece Set Explained

2.1 IC

For each period $t$:

$$\text{IC}_t = \text{Corr}_{\text{rank}}\big(f_{i,t},\ r_{i,t+1}\big),$$

Using Spearman rank correlation (instead of Pearson) is more robust to outliers.

2.2 Decile Backtesting

Divide stocks into 5 groups based on factor values each period, hold them with equal weights until the next period, and record the returns for each group:

• A good factor: Returns from Group 1 to Group 5 are monotonically increasing (or decreasing);
• If only the extremes are effective and the middle is disordered, it suggests the factor may only contain information in the extreme values;
• The returns and Sharpe ratio of the long-short portfolio (Group 5 − Group 1) provide the most intuitive "factor return."

2.3 Turnover and Decay

Layer 3: Code Implementation

3.1 Constructing Simulated Panel Data and Calculating IC

Simulate 120 months, with 300 stocks per month, where a true correlation of approximately 0.04 is hidden between the factor and the next period's return:

import numpy as np, pandas as pd
rng = np.random.default_rng(1)
T, N = 120, 300
dates = pd.period_range("2015-01", periods=T, freq="M")
rows = []
for t in dates:
    f = rng.normal(size=N)
    r = 0.04 * f + rng.normal(size=N)        # True IC is approx 0.04
    rows.append(pd.DataFrame({"date": t, "factor": f, "fwd_ret": r * 0.08}))
panel = pd.concat(rows, ignore_index=True)

ic = panel.groupby("date").apply(lambda g: g["factor"].corr(g["fwd_ret"], method="spearman"))
print("IC Mean", round(ic.mean(), 4), "IC Std", round(ic.std(), 4),
      "ICIR (Annualized)", round(ic.mean() / ic.std() * np.sqrt(12), 2), "IC > 0 Ratio", round((ic > 0).mean(), 2))

Output:

IC Mean 0.0465 IC Std Dev 0.06 ICIR (Annualized) 2.69 IC > 0 Ratio 0.77

3.2 Five-Group Backtesting

panel["q"] = panel.groupby("date")["factor"].transform(lambda x: pd.qcut(x, 5, labels=False) + 1)
qret = panel.groupby(["date", "q"])["fwd_ret"].mean().unstack()
print("Monthly average return per group (%)", (qret.mean() * 100).round(3).to_dict())
ls = qret[5] - qret[1]
print("Long-short annualized return (%)", round(ls.mean() * 12 * 100, 2), "Long-short Sharpe", round(ls.mean() / ls.std() * np.sqrt(12), 2))

Output:

Average monthly return per group (%) {1: -0.573, 2: -0.373, 3: -0.063, 4: -0.026, 5: 0.53}
Long-short annualized return (%) 13.24 Long-short Sharpe 2.74

3.3 Interpretation and Real-World Calibration

• The five-group returns are strictly monotonic, and the long-short Sharpe ratio is 2.7. This is a "textbook" good factor;
• However, this is simulated data: the factor has an independent predictive power of 0.04 every period, with no transaction costs, no style rotation, and no factor crowding;
• In the real world, a factor with an IC around 0.03 often has a long-short Sharpe ratio between 0.5 and 1 after deducting costs, and it will experience long drawdown periods;
• Also, be wary: How many factors did you test before finding this one? (see "Multiple Testing and the Factor Zoo")

3.4 Calculating Turnover

def top_quintile_turnover(panel):
    tops = panel[panel["q"] == 5].groupby("date")["stock_id"].apply(set)
    prev, out = None, []
    for s in tops:
        if prev is not None:
            out.append(1 - len(s & prev) / len(s))   # proportion of new entrants this period
        prev = s
    return np.mean(out)

(This requires a stock_id column in the panel; the simulated data above generates stocks independently each period with no identity continuity, so the turnover will be close to 80%, which is the case when "the factor has no autocorrelation.")

3.5 What a Testing Report Should Include

Summary