[ad_1]
Getting began
On this article, we will probably be making a easy skeleton of a customized Python backtester that ought to have the next options
- Modularity – we wish the backtester to be modular in order that elements may be simply reorganized, swapped, or constructed upon.
- Extendability – the code needs to be simply extendable.
- Assist single and multi-asset methods
- Have entry to historic fairness knowledge and a number of knowledge suppliers
- Incorporate buying and selling price fee
- Have efficiency metrics
To make this potential, we might want to have a number of key parts that are the next:
- Knowledge Administration: Handles the ingestion, storage, and retrieval of OHLCV knowledge, in addition to any various knowledge sources for producing alerts.
- Sign Technology: Accommodates the logic for analyzing knowledge and producing purchase/promote alerts primarily based on predefined methods or indicators.
- Execution Engine: Simulates the execution of trades primarily based on alerts, contemplating commissions, slippage, and optionally, bid-ask spreads.
- Efficiency Analysis: Calculates key efficiency metrics reminiscent of return, volatility, Sharpe ratio, drawdowns, and so on., to guage the technique’s effectiveness.
- Utilities: Contains logging, configuration administration, and another supportive performance.
For this, I’ll be leveraging the ability of a number of Python libraries:
- Poetry – this can permit us to regulate all of the dependencies (
pip set up poetry
) - OpenBB Platform – This can present us with seamless entry to market knowledge throughout a number of suppliers. You’ll be able to learn extra about it here.
- Pandas – A staple when dealing with knowledge.
- Numpy – The elemental package deal for scientific computing.
- Matplotlib – The same old plotting library. May be changed or eliminated relying in your wants.
- Ruff, Black, and MyPy – Linters that I want (optionally available). You’ll be able to learn extra about that here.
You could find the code in our GitHub repository. Onde cloned, you may set up every part by operating the next inside your recent new surroundings:
Enable me to share a bit about my considering sample when approaching this.
My overarching design intention is to have a set of modules that govern the outlined key parts. In different phrases, I wish to have a module that makes a speciality of knowledge administration, a module for commerce execution, and so forth.
This permits for ease of extendability, it helps to decouple the code, it makes it cleaner, and extra. The principle ache level I needed to deal with right here is how arduous it’s to simply lengthen and customise present backtesters on the market.
What I disliked about fairly a number of backtesters is how arduous it’s to design and run multi-asset methods, or the truth that they gate-keep the info, that they solely permit buying and selling of a selected asset class, and extra. All of this stuff needs to be mitigated.
The principle sort of design I used to be going for was Object Oriented Programming (OOP) the place courses are used and it permits us to take care of the state of the backtesting course of.
Notice: All methods proven are very fundamental and for demo and studying functions solely. Please don’t attempt to use them in an actual market setting.
Creating a knowledge handler with the OpenBB Platform
Creating a knowledge handler with the OpenBB Platform is a quite simple expertise. All complications primarily based on completely different API conventions, completely different suppliers, messy outputs, knowledge validation, and the like are being dealt with for us.
It additionally mitigates the necessity to create customized courses for knowledge validation and processing. It means that you can seamlessly have entry to many knowledge suppliers, over a whole bunch of knowledge factors, completely different asset courses, and extra. It additionally ensures what’s returned primarily based on the usual it implements.
Saying that, I’ll stick to simply the fairness belongings and constrain it to every day candles. You’ll be able to simply develop this and alter it to your liking. I’ll permit the person to vary the supplier, image, begin and finish dates.
What I like in regards to the OpenBB Platform is that it has endpoints that help you go a number of tickers and that is certainly one of them. Which means we’re already on observe of supporting a number of asset buying and selling by passing a comma-separated record of symbols.
To arrange the OpenBB Platform, I counsel following this information here.
Right here is the DataHandler
code:
"""Knowledge handler module for loading and processing knowledge."""
from typing import Elective
import pandas as pd
from openbb import obb
class DataHandler:
"""Knowledge handler class for loading and processing knowledge."""
def __init__(
self,
image: str,
start_date: Elective[str] = None,
end_date: Elective[str] = None,
supplier: str = "fmp",
):
"""Initialize the info handler."""
self.image = image.higher()
self.start_date = start_date
self.end_date = end_date
self.supplier = supplier
def load_data(self) -> pd.DataFrame | dict[str, pd.DataFrame]:
"""Load fairness knowledge."""
knowledge = obb.fairness.worth.historic(
image=self.image,
start_date=self.start_date,
end_date=self.end_date,
supplier=self.supplier,
).to_df()
if "," in self.image:
knowledge = knowledge.reset_index().set_index("image")
return {image: knowledge.loc[symbol] for image in self.image.cut up(",")}
return knowledge
def load_data_from_csv(self, file_path) -> pd.DataFrame:
"""Load knowledge from CSV file."""
return pd.read_csv(file_path, index_col="date", parse_dates=True)
Discover the way it returns a dictionary of Pandas dataframes when a number of symbols are being handed. I’ve additionally added a operate that may load knowledge from a customized CSV file and use the date
column as its index. Be happy to develop and alter this to your liking and wishes.
To get some knowledge, all we have to do is to initialize the category and name the load_data
methodology like this:
knowledge = DataHandler("AAPL").load_data()
knowledge.head()
Creating a method processor
The following step is to have a module that can course of our methods. By this, I imply to say one thing that may have the ability to generate alerts primarily based on the technique necessities and append them to the info in order that they can be utilized by the executor for backtesting.
What I’m going for right here is one thing like a base class for Methods that builders can inherit from, change, or construct their very own customized ones. I additionally need it to work seamlessly when a number of belongings so it applies the identical sign logic over a number of belongings.
Here’s what the code for it seems like:
class Technique:
"""Base class for buying and selling methods."""
def __init__(self, indicators: dict, signal_logic: Any):
"""Initialize the technique with indicators and sign logic."""
self.indicators = indicators
self.signal_logic = signal_logic
def generate_signals(
self, knowledge: pd.DataFrame | dict[str, pd.DataFrame]
) -> pd.DataFrame | dict[str, pd.DataFrame]:
"""Generate buying and selling alerts primarily based on the technique's indicators and sign logic."""
if isinstance(knowledge, dict):
for _, asset_data in knowledge.gadgets():
self._apply_strategy(asset_data)
else:
self._apply_strategy(knowledge)
return knowledge
def _apply_strategy(self, df: pd.DataFrame) -> None:
"""Apply the technique to a single dataframe."""
for identify, indicator in self.indicators.gadgets():
df[name] = indicator(df)
df["signal"] = df.apply(lambda row: self.signal_logic(row), axis=1)
df["positions"] = df["signal"].diff().fillna(0)
It really works by taking a dictionary of indicators that have to be computed and in addition the logic to make use of for producing the alerts that may be -1 for promoting and +1 for getting. It additionally retains observe of the positions we’re in.
The way in which it’s coded proper now’s that we go it lambda capabilities which it applies to the dataframe.
Right here’s an instance of how we are able to apply it to the info we retrieved within the earlier step:
technique = Technique(
indicators={
"sma_20": lambda row: row["close"].rolling(window=20).imply(),
"sma_60": lambda row: row["close"].rolling(window=60).imply(),
},
signal_logic=lambda row: 1 if row["sma_20"] > row["sma_60"] else -1,
)
knowledge = technique.generate_signals(knowledge)
knowledge.tail()
Within the above instance, I created a gradual and fast-moving common on the closing costs after which outlined my buying and selling logic the place I wish to lengthy when the fast-moving common crosses over the slow-moving common and vice-versa.
Now that we have now a technique to get knowledge and generate buying and selling alerts, all we’re lacking is a technique to really run the backtest. That is probably the most complicated half.
Creating the principle backtester logic
The principle backtester logic will probably be comprised of a number of elements. The principle elements that we have to have are these:
- Commerce executor
- Fee calculator
- Efficiency metric calculator
- Portfolio handler
- The glue between all of them
Allow us to start by defining the category and setting some fundamental variables we wish it to deal with:
class Backtester:
"""Backtester class for backtesting buying and selling methods."""
def __init__(
self,
initial_capital: float = 10000.0,
commission_pct: float = 0.001,
commission_fixed: float = 1.0,
):
"""Initialize the backtester with preliminary capital and fee charges."""
self.initial_capital: float = initial_capital
self.commission_pct: float = commission_pct
self.commission_fixed: float = commission_fixed
self.assets_data: Dict = {}
self.portfolio_history: Dict = {}
self.daily_portfolio_values: Checklist[float] = []
Now, we’ll outline the commerce executor:
def execute_trade(self, asset: str, sign: int, worth: float) -> None:
"""Execute a commerce primarily based on the sign and worth."""
if sign > 0 and self.assets_data[asset]["cash"] > 0: # Purchase
trade_value = self.assets_data[asset]["cash"]
fee = self.calculate_commission(trade_value)
shares_to_buy = (trade_value - fee) / worth
self.assets_data[asset]["positions"] += shares_to_buy
self.assets_data[asset]["cash"] -= trade_value
elif sign < 0 and self.assets_data[asset]["positions"] > 0: # Promote
trade_value = self.assets_data[asset]["positions"] * worth
fee = self.calculate_commission(trade_value)
self.assets_data[asset]["cash"] += trade_value - fee
self.assets_data[asset]["positions"] = 0
The commerce executor will purchase the asset if the sign is bigger than 0 and promote the asset whether it is lower than 0. It should additionally be sure that we have now money so as to purchase and that we’re ready to have the ability to promote. It should additionally calculate what number of shares we are able to purchase and account for the trade fee.
To calculate the fee, we do the next:
def calculate_commission(self, trade_value: float) -> float:
"""Calculate the fee price for a commerce."""
return max(trade_value * self.commission_pct, self.commission_fixed)
Now, we have to observe our positions for the belongings we’re buying and selling and their values and historical past:
def update_portfolio(self, asset: str, worth: float) -> None:
"""Replace the portfolio with the newest worth."""
self.assets_data[asset]["position_value"] = (
self.assets_data[asset]["positions"] * worth
)
self.assets_data[asset]["total_value"] = (
self.assets_data[asset]["cash"] + self.assets_data[asset]["position_value"]
)
self.portfolio_history[asset].append(self.assets_data[asset]["total_value"])
Lastly, the backtester can now be run through the use of these strategies like this:
def backtest(self, knowledge: pd.DataFrame | dict[str, pd.DataFrame]):
"""Backtest the buying and selling technique utilizing the supplied knowledge."""
if isinstance(knowledge, pd.DataFrame): # Single asset
knowledge = {
"SINGLE_ASSET": knowledge
} # Convert to dict format for unified processing
for asset in knowledge:
self.assets_data[asset] = {
"money": self.initial_capital / len(knowledge),
"positions": 0,
"position_value": 0,
"total_value": 0,
}
self.portfolio_history[asset] = []
for date, row in knowledge[asset].iterrows():
self.execute_trade(asset, row["signal"], row["close"])
self.update_portfolio(asset, row["close"])
if len(self.daily_portfolio_values) < len(knowledge[asset]):
self.daily_portfolio_values.append(
self.assets_data[asset]["total_value"]
)
else:
self.daily_portfolio_values[
len(self.portfolio_history[asset]) - 1
] += self.assets_data[asset]["total_value"]
Now, I’ll add a way to calculate some metrics and this may be expanded through the use of third-party libraries or the like. I’ll additionally do the identical for the plotting options. The precise code may be seen within the repo.
def calculate_performance(self, plot: bool = True) -> None:
"""Calculate the efficiency of the buying and selling technique."""
if not self.daily_portfolio_values:
print("No portfolio historical past to calculate efficiency.")
return
portfolio_values = pd.Sequence(self.daily_portfolio_values)
daily_returns = portfolio_values.pct_change().dropna()
total_return = calculate_total_return(
portfolio_values.iloc[-1], self.initial_capital
)
annualized_return = calculate_annualized_return(
total_return, len(portfolio_values)
)
annualized_volatility = calculate_annualized_volatility(daily_returns)
sharpe_ratio = calculate_sharpe_ratio(annualized_return, annualized_volatility)
sortino_ratio = calculate_sortino_ratio(daily_returns, annualized_return)
max_drawdown = calculate_maximum_drawdown(portfolio_values)
print(f"Closing Portfolio Worth: {portfolio_values.iloc[-1]:.2f}")
print(f"Complete Return: {total_return * 100:.2f}%")
print(f"Annualized Return: {annualized_return * 100:.2f}%")
print(f"Annualized Volatility: {annualized_volatility * 100:.2f}%")
print(f"Sharpe Ratio: {sharpe_ratio:.2f}")
print(f"Sortino Ratio: {sortino_ratio:.2f}")
print(f"Most Drawdown: {max_drawdown * 100:.2f}%")
if plot:
self.plot_performance(portfolio_values, daily_returns)
def plot_performance(self, portfolio_values: Dict, daily_returns: pd.DataFrame):
"""Plot the efficiency of the buying and selling technique."""
plt.determine(figsize=(10, 6))
plt.subplot(2, 1, 1)
plt.plot(portfolio_values, label="Portfolio Worth")
plt.title("Portfolio Worth Over Time")
plt.legend()
plt.subplot(2, 1, 2)
plt.plot(daily_returns, label="Each day Returns", colour="orange")
plt.title("Each day Returns Over Time")
plt.legend()
plt.tight_layout()
plt.present()
Now that the backtester is able to go, allow us to attempt it out with a few completely different methods.
backtest a crossover technique with Python?
The objective for this technique will probably be to create a really fundamental crossover technique the place we use a fast-moving easy transferring common (SMA) and a slow-moving one. When the quick crosses above the gradual we purchase and vice-versa.
We will probably be buying and selling the AAPL inventory for this. Right here is how we are able to do it:
from backtester.data_handler import DataHandler
from backtester.backtester import Backtester
from backtester.methods import Technique
image = "AAPL,MSFT"
start_date = "2023-01-01"
end_date = "2023-12-31"
knowledge = DataHandler(
image=image, start_date=start_date, end_date=end_date
).load_data()
# Outline your technique, indicators, and sign logic right here
technique = Technique(
indicators={
"sma_20": lambda row: row["close"].rolling(window=20).imply(),
"sma_60": lambda row: row["close"].rolling(window=60).imply(),
},
signal_logic=lambda row: 1 if row["sma_20"] > row["sma_60"] else -1,
)
knowledge = technique.generate_signals(knowledge)
backtester = Backtester()
backtester.backtest(knowledge)
backtester.calculate_performance()
Closing Portfolio Worth: 11804.58
Complete Return: 18.05%
Annualized Return: 18.20%
Annualized Volatility: 13.06%
Sharpe Ratio: 1.39
Sortino Ratio: 2.06
Most Drawdown: -12.07%
Nice! The Backtester is working because it ought to. To check a single asset, we simply want to vary the image to a single one. For instance, to backtest NFLX with the identical technique all I want to vary is that, and listed below are the outcomes:
Closing Portfolio Worth: 13999.01
Complete Return: 39.99%
Annualized Return: 40.37%
Annualized Volatility: 22.55%
Sharpe Ratio: 1.79
Sortino Ratio: 1.94
Most Drawdown: -15.62%
backtest a mean-reversion technique with Python?
To backtest a mean-reversion technique with Python, we’ll use our customized backtester and leverage its modularity and ease of chaining operations. First, allow us to lay out the technique logic:
The technique has a objective to promote the asset whether it is buying and selling greater than 3 customary deviations above the rolling imply and to purchase the asset whether it is buying and selling greater than 3 customary deviations under the rolling imply.
This has a few implications for it to work correctly:
- We have to have a rolling imply
- We have to calculate the STD from the rolling imply
- We have to calculate the higher and decrease bounds
As a result of our Technique
class applies calculations within the given order, we are able to simply chain these calculations collectively by following their logical order and creating alerts primarily based on them.
Allow us to begin by defining the bottom backtesting parameters:
image = "HE"
start_date = "2022-01-01"
end_date = "2022-12-31"
Now, all we have to do is to get the info, chain the operations collectively, and see what the outcomes are:
knowledge = DataHandler(image=image, start_date=start_date, end_date=end_date).load_data()
# Outline your technique, indicators, and sign logic right here
technique = Technique(
indicators={
"sma_50": lambda row: row["close"].rolling(window=50).imply(),
"std_3": lambda row: row["close"].rolling(window=50).std() * 3,
"std_3_upper": lambda row: row["sma_50"] + row["std_3"],
"std_3_lower": lambda row: row["sma_50"] - row["std_3"],
},
signal_logic=lambda row: (
1
if row["close"] < row["std_3_lower"]
else -1 if row["close"] > row["std_3_upper"] else 0
),
)
knowledge = technique.generate_signals(knowledge)
backtester = Backtester()
backtester.backtest(knowledge)
backtester.calculate_performance()
Closing Portfolio Worth: 10725.54
Complete Return: 7.26%
Annualized Return: 7.29%
Annualized Volatility: 18.32%
Sharpe Ratio: 0.40
Sortino Ratio: 0.53
Most Drawdown: -23.37%
We are able to now simply additionally run experiments by chaining extra operations or altering them. For instance, what occurs if we base the STD off the rolling imply as a substitute?
"std_3": lambda row: row["sma_50"].std() * 3,
Closing Portfolio Worth: 12062.36
Complete Return: 20.62%
Annualized Return: 20.71%
Annualized Volatility: 13.12%
Sharpe Ratio: 1.58
Sortino Ratio: 1.71
Most Drawdown: -7.19%
backtest a pairs buying and selling technique with Python?
Backtesting a pairs buying and selling technique with Python is an much more complicated instance. However, our backtester shouldn’t have points executing it. The principle factor that makes it extra complicated right here is that we are going to wish to have knowledge for each belongings in a single dataframe. Allow us to outline the technique first.
The belongings that we are going to commerce are Roku (ROKU) and Netflix (NFLX) as we have already got a way of their cointegrated nature primarily based on our earlier articles and analyses.
We are going to enter a place (purchase) if one inventory has moved 5% or greater than the opposite one over the course of the final 5 days. We are going to promote the highest one and purchase the underside one till it reverses. Allow us to arrange every part and do some fast knowledge wrangling:
import pandas as pd
image = "NFLX,ROKU"
start_date = "2023-01-01"
knowledge = DataHandler(
image=image,
start_date=start_date,
).load_data()
knowledge = pd.merge(
knowledge["NFLX"].reset_index(),
knowledge["ROKU"].reset_index(),
left_index=True,
right_index=True,
suffixes=("_NFLX", "_ROKU"),
)
# We wish to commerce the ROKU inventory so we rename the close_ROKU column to shut
knowledge = knowledge.rename(columns={"close_ROKU": "shut"})
knowledge.head()
Now, all we require is the buying and selling logic and we are able to run the backtester:
technique = Technique(
indicators={
"day_5_lookback_NFLX": lambda row: row["close_NFLX"].shift(5),
"day_5_lookback_ROKU": lambda row: row["close"].shift(5),
},
signal_logic=lambda row: (
1
if row["close_NFLX"] > row["day_5_lookback_NFLX"] * 1.05
else -1 if row["close_NFLX"] < row["day_5_lookback_NFLX"] * 0.95 else 0
),
)
knowledge = technique.generate_signals(knowledge)
backtester = Backtester()
backtester.backtest(knowledge)
backtester.calculate_performance()
Closing Portfolio Worth: 14387.50
Complete Return: 43.88%
Annualized Return: 34.80%
Annualized Volatility: 55.77%
Sharpe Ratio: 0.62
Sortino Ratio: 0.74
Most Drawdown: -39.86%
backtest a method with various knowledge with Python?
To backtest a method with various knowledge with Python, all we have to do is to make use of the customized backtester to load our customized dataset that will probably be used for buying and selling. Alternatively, we are able to additionally mix various knowledge with the info fetched from the DataHandler
.
For this use case, I’ll be loading a customized CSV file that has a sign column that’s calculated primarily based on the sentiment rating that was extracted. If the sentiment is constructive we’ll purchase and vice-versa.
We load the info and run the backtester:
knowledge = DataHandler(image="HE").load_data_from_csv("example_data.csv")
technique = Technique(
indicators={},
signal_logic=lambda row: (
1
if row["trade_signal_sentiment"] > 0
else -1
),
)
knowledge = technique.generate_signals(knowledge)
backtester = Backtester()
backtester.backtest(knowledge)
backtester.calculate_performance()
Closing Portfolio Worth: 9128.27
Complete Return: -8.72%
Annualized Return: -8.75%
Annualized Volatility: 17.93%
Sharpe Ratio: -0.49
Sortino Ratio: -0.58
Most Drawdown: -19.98%
Discover how we didn’t want any indicators and simply handed an empty dictionary. Working with this skeleton is sort of versatile.
Closing ideas
Generally, the options which are on the market aren’t fairly the appropriate match to your wants and they aren’t straightforward to adapt, lengthen, modify, or work with. A few of them are fairly good however not maintained and have so many dependencies that they turn into unstable.
There are occasions when it is sensible to spend a while making a customized software that can make it easier to along with your day-to-day duties. I’ve proven you a easy backtester skeleton that may simply be tailored, modified, modified, and labored with.
It may be additional polished, have extensions, extra metrics, extra charts, or something that you simply would possibly want. The code is open-sourced and you may play with it, create PRs, and extra.
The identical philosophy may be utilized to different instruments that you simply is perhaps curious about, not solely backtesting.
[ad_2]
Source link