This report explains how oil and interest rate curve data is produced. The methodology proceeds in the following steps:
Fetch Treasury yield curve. The Python script
(curves.py) queries the U.S. Treasury XML feed for daily
par yield curve rates (constant-maturity). The feed provides 13 tenors
from 1 Month to 30 Years.
Fetch overnight policy rates. The effective Federal Funds Rate (EFFR) and Secured Overnight Financing Rate (SOFR) are retrieved from FRED (Federal Reserve Economic Data) via the public CSV endpoint.
Fetch oil spot price. The latest Brent crude settlement price is fetched from Yahoo Finance (front-month contract BZ=F).
Build oil forward curve. A strip of Brent crude futures contracts is fetched from Yahoo Finance. Standard tenors (1m, 3m, 6m, 12m) are then computed via linear interpolation between the two bracketing contracts.
Build interest rate forward curve. Fed Funds (ZQ) and SOFR (SR1, SR3) futures are fetched from Yahoo Finance. Implied rates are computed as \(100 - \text{settlement price}\).
Store incrementally. Treasury daily yields are saved to Parquet. On each run, only dates not already present are appended.
python curves.py # Fetch and display
python curves.py --save # Also save to parquet (incremental)| Source | Data provided | URL | Lag |
|---|---|---|---|
| U.S. Treasury | Daily par yield curve (13 tenors, 1 Mo – 30 Yr) | home.treasury.gov XML feed | ~1 business day |
| FRED | Effective Federal Funds Rate (DFF), SOFR | fred.stlouisfed.org/graph/fredgraph.csv | ~1 business day |
| Yahoo Finance | Brent crude spot (BZ=F) | finance.yahoo.com | Real-time / EOD |
| Yahoo Finance | Brent crude futures strip (BZK, BZN, BZX, BZF, BZH) | finance.yahoo.com | Real-time / EOD |
| Yahoo Finance | Fed Funds futures (ZQ), SOFR futures (SR1, SR3) | finance.yahoo.com | Real-time / EOD |
All values in this section are market prices — directly observed, not computed or interpolated.
| Instrument | Price | As of | Source | Classification |
|---|---|---|---|---|
| Brent Crude (BZ=F) | $98.91/bbl | 2026-03-13 | Yahoo Finance | MARKET PRICE |
This is the most recent settlement (close) price for the front-month Brent crude futures contract traded on NYMEX.
| Rate | Value | As of | Source | Classification |
|---|---|---|---|---|
| Federal Funds (EFFR) | 3.64% | 2026-03-12 | FRED series DFF | MARKET PRICE |
| SOFR | 3.65% | 2026-03-12 | FRED series SOFR | MARKET PRICE |
Both are actual transacted rates, not targets — the current FOMC target range is 3.50%–3.75%.
| Tenor | Yield | Classification |
|---|---|---|
| 1 Mo | 3.75% | MARKET PRICE |
| 2 Mo | 3.71% | MARKET PRICE |
| 3 Mo | 3.72% | MARKET PRICE |
| 4 Mo | 3.69% | MARKET PRICE |
| 6 Mo | 3.70% | MARKET PRICE |
| 1 Yr | 3.66% | MARKET PRICE |
| 2 Yr | 3.73% | MARKET PRICE |
| 3 Yr | 3.74% | MARKET PRICE |
| 5 Yr | 3.87% | MARKET PRICE |
| 7 Yr | 4.07% | MARKET PRICE |
| 10 Yr | 4.28% | MARKET PRICE |
| 20 Yr | 4.89% | MARKET PRICE |
| 30 Yr | 4.90% | MARKET PRICE |
These are constant-maturity Treasury (CMT) rates published daily by the U.S. Department of the Treasury. The Treasury derives them from the daily yield curve for non-inflation-indexed Treasury securities, using closing market bid yields on actively traded securities in the over-the-counter market.
The data is fetched from the Treasury’s XML feed at:
https://home.treasury.gov/resource-center/data-chart-center/interest-rates/pages/xml
?data=daily_treasury_yield_curve
&field_tdr_date_value_month=202603These are not FRED values. While FRED also publishes these as DGS series (DGS1MO, DGS3MO, etc.), the Treasury XML feed is the primary source and is typically available earlier.
Forward curves reflect market expectations of future prices. They are derived from futures contracts.
These are actual settlement prices of Brent crude futures contracts at various expiry dates. Each is a market price — the price at which the contract last traded or settled.
| Ticker | Contract Expiry | Days Forward | Price | Classification |
|---|---|---|---|---|
| BZ=F | 2026-04-13 | 30 | $98.91/bbl | MARKET PRICE |
| BZK26.NYM | 2026-05-20 | 67 | $103.14/bbl | MARKET PRICE |
| BZN26.NYM | 2026-07-20 | 128 | $94.67/bbl | MARKET PRICE |
| BZX26.NYM | 2026-11-20 | 251 | $83.64/bbl | MARKET PRICE |
| BZF27.NYM | 2027-01-20 | 312 | $80.48/bbl | MARKET PRICE |
| BZH27.NYM | 2027-03-20 | 371 | $78.15/bbl | MARKET PRICE |
The raw futures contracts do not fall on neat 1-month, 3-month, 6-month, and 12-month boundaries. To produce standard tenors, we use linear interpolation between the two bracketing contracts.
| Tenor | Price | Left bracket | Right bracket | Weight (w) | Classification |
|---|---|---|---|---|---|
| 1m (30d) | $98.91/bbl | 30d @ $98.91 | (exact match) | — | COMPUTED |
| 3m (91d) | $99.81/bbl | 67d @ $103.14 | 128d @ $94.67 | 0.393 | COMPUTED |
| 6m (182d) | $89.83/bbl | 128d @ $94.67 | 251d @ $83.64 | 0.439 | COMPUTED |
| 12m (365d) | $78.39/bbl | 312d @ $80.48 | 371d @ $78.15 | 0.898 | COMPUTED |
Given two known futures prices at days \(d_0\) and \(d_1\) with prices \(P_0\) and \(P_1\), the price at target day \(d\) is:
\[w = \frac{d - d_0}{d_1 - d_0}\]
\[P(d) = P_0 + w \cdot (P_1 - P_0)\]
This is sometimes called “lerp” (linear interpolation), the most common and transparent interpolation method in quantitative finance. It draws a straight line between two known points and reads off the value at the desired position.
3-month tenor (target: 91 days)
The two bracketing contracts are:
\[w = \frac{91 - 67}{128 - 67} = \frac{24}{61} = 0.393\]
\[P(91) = \$103.14 + 0.393 \times (\$94.67 - \$103.14) = \$103.14 + 0.393 \times (-\$8.47) = \$103.14 - \$3.33 = \$99.81\]
6-month tenor (target: 182 days)
\[w = \frac{182 - 128}{251 - 128} = \frac{54}{123} = 0.439\]
\[P(182) = \$94.67 + 0.439 \times (\$83.64 - \$94.67) = \$94.67 + 0.439 \times (-\$11.03) = \$94.67 - \$4.84 = \$89.83\]
12-month tenor (target: 365 days)
\[w = \frac{365 - 312}{371 - 312} = \frac{53}{59} = 0.898\]
\[P(365) = \$80.48 + 0.898 \times (\$78.15 - \$80.48) = \$80.48 + 0.898 \times (-\$2.33) = \$80.48 - \$2.09 = \$78.39\]
1-month tenor (target: 30 days)
The nearest contract is exactly at 30 days ($98.91/bbl), so no interpolation is needed.
Linear interpolation assumes the curve between two points is a straight line. It may understate curvature, particularly:
| Method | Advantage | Common usage |
|---|---|---|
| Linear (lerp) | Simple, transparent, no overshoot | Everywhere — default |
| Cubic spline | Smooth curve, captures curvature | Yield curve construction |
| Nelson-Siegel | Parsimonious yield curve model (3 params) | Central banks, fixed income |
| Log-linear | Better for discount factors | Swap curve bootstrapping |
| Contract | Implied Rate | Computation | Classification |
|---|---|---|---|
| Fed Funds front | 3.62% | 100 − futures price | MARKET PRICE (derived) |
| Fed Funds Apr-2026 | 3.64% | 100 − futures price | MARKET PRICE (derived) |
| Fed Funds Jul-2026 | 3.57% | 100 − futures price | MARKET PRICE (derived) |
| Fed Funds Oct-2026 | 3.50% | 100 − futures price | MARKET PRICE (derived) |
| Fed Funds Jan-2027 | 3.43% | 100 − futures price | MARKET PRICE (derived) |
| SOFR 1m front | 3.67% | 100 − futures price | MARKET PRICE (derived) |
| SOFR 3m front | 3.69% | 100 − futures price | MARKET PRICE (derived) |
| SOFR 3m Mar-2026 | 3.69% | 100 − futures price | MARKET PRICE (derived) |
| SOFR 3m Jun-2026 | 3.63% | 100 − futures price | MARKET PRICE (derived) |
| SOFR 3m Sep-2026 | 3.56% | 100 − futures price | MARKET PRICE (derived) |
| SOFR 3m Dec-2026 | 3.50% | 100 − futures price | MARKET PRICE (derived) |
These implied rates are classified as market-price-derived rather than “interpolated,” because each rate corresponds to a specific traded futures contract. The computation is deterministic:
\[\text{implied rate} = 100 - \text{futures settlement price}\]
For example, if the front-month Fed Funds futures (ZQ=F) settles at 96.38, the implied rate is \(100 - 96.38 = 3.62\%\).
This is the standard convention for CME interest rate futures (Fed Funds, SOFR). No interpolation is involved — each contract directly encodes the market’s expectation of the average rate over that contract’s reference period.
| Prefix | Exchange | Underlying |
|---|---|---|
| ZQ | CBOT | 30-Day Federal Funds Rate |
| SR1 | CME | 1-Month SOFR |
| SR3 | CME | 3-Month SOFR |
| Data point | Classification | Source | Method |
|---|---|---|---|
| Brent crude spot | MARKET PRICE | Yahoo Finance | Last close |
| Fed Funds (EFFR) | MARKET PRICE | FRED (DFF) | Last published value |
| SOFR | MARKET PRICE | FRED (SOFR) | Last published value |
| Treasury yield curve (13 tenors) | MARKET PRICE | U.S. Treasury XML feed | Last published daily curve |
| Oil futures strip (6 contracts) | MARKET PRICE | Yahoo Finance | Last settlement price |
| Oil forward 1m, 3m, 6m, 12m | COMPUTED (interpolated) | Linear interpolation of futures strip | Linear interpolation between brackets |
| Rate futures implied rates (11 contracts) | MARKET PRICE (derived) | Yahoo Finance (100 − price) | Arithmetic: 100 − settlement price |
Key distinction: Only the oil forward curve standard tenors (1m, 3m, 6m, 12m) are interpolated. Everything else is either a direct market observation or a deterministic arithmetic transformation of one.
Results are stored in Parquet format under
curves_data/:
| File | Contents | Update method |
|---|---|---|
| treasury_yields.parquet | Daily Treasury yield curve (one row per business day, 13 tenor columns) | Incremental — only new dates are appended |
| curves/ (partitioned) | Daily snapshot of all spot + forward data (one row per fetch date) | Overwrite-or-ignore per year/month partition |
## **Treasury yields stored:** 10 dates, from 2026-03-02 to 2026-03-13| date | 1MONTH | 2MONTH | 3MONTH | 4MONTH | 6MONTH | 1YEAR | 2YEAR | 3YEAR | 5YEAR | 7YEAR | 10YEAR | 20YEAR | 30YEAR |
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 2026-03-13 | 3.75 | 3.71 | 3.72 | 3.69 | 3.70 | 3.66 | 3.73 | 3.74 | 3.87 | 4.07 | 4.28 | 4.89 | 4.90 |
| 2026-03-12 | 3.76 | 3.72 | 3.72 | 3.69 | 3.70 | 3.66 | 3.76 | 3.75 | 3.88 | 4.06 | 4.27 | 4.86 | 4.88 |
| 2026-03-11 | 3.75 | 3.70 | 3.71 | 3.69 | 3.68 | 3.60 | 3.64 | 3.64 | 3.79 | 3.98 | 4.21 | 4.82 | 4.86 |
| 2026-03-10 | 3.75 | 3.71 | 3.71 | 3.69 | 3.68 | 3.56 | 3.57 | 3.58 | 3.73 | 3.93 | 4.15 | 4.74 | 4.78 |
| 2026-03-09 | 3.75 | 3.72 | 3.71 | 3.68 | 3.68 | 3.56 | 3.56 | 3.58 | 3.71 | 3.90 | 4.12 | 4.70 | 4.72 |
To inspect the stored data:
library(arrow)
read_parquet("curves_data/treasury_yields.parquet")python -c "import pandas as pd; print(pd.read_parquet('curves_data/treasury_yields.parquet'))"The full source code for curves.py is provided below.
Copy and paste to run independently.
Requirements:
pip install yfinance pandas pyarrow requests```python
"""
curves.py
=========
Fetches and displays oil and interest rate data in a clear structure:
1. SPOT (latest observed)
- Latest oil price (Brent front)
- Latest yield curve (Treasury XML feed — real data, not FRED)
2. FORWARD CURVES (market expectations)
- Oil forward curve (1m, 3m, 6m, 12m) — interpolated from futures strip
- Interest rate forward curve (Fed Funds, SOFR implied from futures)
3. PARQUET STORAGE
- Treasury daily yields stored incrementally (only new dates added)
Data sources:
- Treasury yield curve : home.treasury.gov XML feed (daily, ~1 day lag)
- Oil futures : Yahoo Finance (BZ=F strip)
- Rate futures : Yahoo Finance (ZQ, SR1, SR3)
Usage:
pip install yfinance pandas pyarrow requests
python curves.py # Fetch and display
python curves.py --save # Also save to parquet (incremental)
"""
import argparse
import os
import xml.etree.ElementTree as ET
from datetime import date, timedelta
from pathlib import Path
from typing import Dict, List, Tuple
import pandas as pd
import requests
import yfinance as yf
# Load .env if present
_env_file = Path(__file__).resolve().parent / ".env"
if _env_file.exists():
for line in _env_file.read_text().splitlines():
line = line.strip()
if line and not line.startswith("#") and "=" in line:
k, _, v = line.partition("=")
k, v = k.strip(), v.strip().strip('"').strip("'")
if k and k not in os.environ:
os.environ[k] = v
# ---------------------------------------------------------------------------
# Config
# ---------------------------------------------------------------------------
PARQUET_DIR = Path("./curves_data")
FRED_CSV = "https://fred.stlouisfed.org/graph/fredgraph.csv"
# Treasury daily par yield curve XML feed
# https://home.treasury.gov/treasury-daily-interest-rate-xml-feed
TREASURY_XML = "https://home.treasury.gov/resource-center/data-chart-center/interest-rates/pages/xml"
# XML element names -> display labels
TREASURY_TENORS = [
("BC_1MONTH", "1 Mo"),
("BC_2MONTH", "2 Mo"),
("BC_3MONTH", "3 Mo"),
("BC_4MONTH", "4 Mo"),
("BC_6MONTH", "6 Mo"),
("BC_1YEAR", "1 Yr"),
("BC_2YEAR", "2 Yr"),
("BC_3YEAR", "3 Yr"),
("BC_5YEAR", "5 Yr"),
("BC_7YEAR", "7 Yr"),
("BC_10YEAR", "10 Yr"),
("BC_20YEAR", "20 Yr"),
("BC_30YEAR", "30 Yr"),
]
# CME month codes: F=Jan, G=Feb, H=Mar, J=Apr, K=May, M=Jun, N=Jul, Q=Aug, U=Sep, V=Oct, X=Nov, Z=Dec
OIL_MONTH = {"F": 1, "G": 2, "H": 3, "J": 4, "K": 5, "M": 6, "N": 7, "Q": 8, "U": 9, "V": 10, "X": 11, "Z": 12}
# Oil strip (Brent)
def _oil_strip(year: int):
y2, y2n = str(year)[-2:], str(year + 1)[-2:]
return {
"BZ=F": "Front",
f"BZK{y2}.NYM": "May",
f"BZN{y2}.NYM": "Jul",
f"BZX{y2}.NYM": "Nov",
f"BZF{y2n}.NYM": "Jan",
f"BZH{y2n}.NYM": "Mar",
}
# Rate futures: Fed Funds (ZQ) + SOFR (SR1, SR3). Price = 100 - implied_rate.
def _rate_futures_strip(year: int):
y2, y2n = str(year)[-2:], str(year + 1)[-2:]
return {
"ZQ=F": "Fed Funds front",
f"ZQJ{y2}.CBT": f"Fed Funds Apr-{year}",
f"ZQN{y2}.CBT": f"Fed Funds Jul-{year}",
f"ZQV{y2}.CBT": f"Fed Funds Oct-{year}",
f"ZQF{y2n}.CBT": f"Fed Funds Jan-{year+1}",
"SR1=F": "SOFR 1m front",
"SR3=F": "SOFR 3m front",
f"SR3H{y2}.CME": f"SOFR 3m Mar-{year}",
f"SR3M{y2}.CME": f"SOFR 3m Jun-{year}",
f"SR3U{y2}.CME": f"SOFR 3m Sep-{year}",
f"SR3Z{y2}.CME": f"SOFR 3m Dec-{year}",
}
# ---------------------------------------------------------------------------
# Fetch: Treasury yield curve (XML feed — real data from Treasury)
# ---------------------------------------------------------------------------
NS = {"d": "http://schemas.microsoft.com/ado/2007/08/dataservices",
"m": "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata",
"a": "http://www.w3.org/2005/Atom"}
def fetch_treasury_xml(year_month: str = None) -> pd.DataFrame:
"""
Fetch daily Treasury par yield curve from XML feed.
year_month: "202603" for a specific month, or None for current year.
Returns DataFrame with columns: date, BC_1MONTH, BC_2MONTH, … BC_30YEAR.
All values are REAL data published by Treasury (not interpolated by us).
"""
if year_month:
params = {"data": "daily_treasury_yield_curve", "field_tdr_date_value_month": year_month}
else:
params = {"data": "daily_treasury_yield_curve", "field_tdr_date_value": str(date.today().year)}
r = requests.get(TREASURY_XML, params=params, timeout=30)
r.raise_for_status()
root = ET.fromstring(r.content)
rows = []
for entry in root.findall("a:entry", NS):
props = entry.find(".//m:properties", NS)
if props is None:
continue
dt_el = props.find("d:NEW_DATE", NS)
if dt_el is None or dt_el.text is None:
continue
row = {"date": dt_el.text[:10]}
for xml_key, _ in TREASURY_TENORS:
el = props.find(f"d:{xml_key}", NS)
if el is not None and el.text:
try:
row[xml_key] = float(el.text)
except ValueError:
pass
rows.append(row)
df = pd.DataFrame(rows)
if not df.empty:
df["date"] = pd.to_datetime(df["date"])
df = df.sort_values("date")
return df
def fetch_yield_curve() -> Tuple[dict, str]:
"""
Latest Treasury yield curve from XML feed.
Returns (dict of label->value, date_str). All values are real Treasury data.
"""
today = date.today()
ym = today.strftime("%Y%m")
df = fetch_treasury_xml(ym)
if df.empty:
prev = (today.replace(day=1) - timedelta(days=1))
df = fetch_treasury_xml(prev.strftime("%Y%m"))
if df.empty:
return {}, "N/A"
row = df.iloc[-1]
dt_str = row["date"].strftime("%Y-%m-%d")
result = {}
for xml_key, label in TREASURY_TENORS:
if xml_key in row and pd.notna(row[xml_key]):
result[label] = row[xml_key]
return result, dt_str
def fetch_latest_oil() -> Tuple[float, str]:
"""Latest Brent crude price ($/bbl). REAL market close price."""
t = yf.Ticker("BZ=F")
hist = t.history(period="5d")
if hist.empty:
return 0.0, "N/A"
price = float(hist["Close"].iloc[-1])
dt = hist.index[-1].strftime("%Y-%m-%d")
return price, dt
def _fred_series(series_id: str) -> Tuple[float, str]:
"""Fetch latest value for a single FRED series via public CSV endpoint."""
from io import StringIO
try:
r = requests.get(FRED_CSV, params={"id": series_id}, timeout=20)
r.raise_for_status()
df = pd.read_csv(StringIO(r.text))
date_col = next((c for c in ["DATE", "Date", "date"] if c in df.columns), df.columns[0])
if series_id not in df.columns:
return float("nan"), "N/A"
df = df[df[series_id].astype(str) != "."].dropna(subset=[series_id])
if df.empty:
return float("nan"), "N/A"
df[date_col] = pd.to_datetime(df[date_col])
row = df.iloc[-1]
return float(row[series_id]), row[date_col].strftime("%Y-%m-%d")
except Exception:
return float("nan"), "N/A"
def fetch_policy_rates() -> dict:
"""
Fetch actual policy rates from FRED:
- DFF : Effective Federal Funds Rate (daily, ~1 day lag)
- SOFR : Secured Overnight Financing Rate (daily, ~1 day lag)
These are REAL observed rates, not futures-implied.
"""
result = {}
for series_id, label in [("DFF", "Fed Funds (EFFR)"), ("SOFR", "SOFR")]:
val, dt = _fred_series(series_id)
result[label] = {"value": val, "as_of": dt}
return result
# ---------------------------------------------------------------------------
# Fetch: Forward curves
# ---------------------------------------------------------------------------
def _contract_date(ticker: str, fetch_date: date):
"""Approx expiry from ticker. BZK26.NYM, ZQJ26.CBT, SR1J26.CME."""
if "=F" in ticker:
return fetch_date + timedelta(days=30)
try:
if ticker.startswith("BZ") and len(ticker) >= 5:
m = OIL_MONTH.get(ticker[2])
y = 2000 + int(ticker[3:5])
elif ticker.startswith("ZQ") and len(ticker) >= 5:
m = OIL_MONTH.get(ticker[2])
y = 2000 + int(ticker[3:5])
elif ticker.startswith("SR") and len(ticker) >= 6:
m = OIL_MONTH.get(ticker[3])
y = 2000 + int(ticker[4:6])
else:
return None
return date(y, m, 20) if m else None
except (IndexError, ValueError):
return None
def fetch_oil_forward(fetch_date: date) -> Tuple[List[Tuple[date, float]], list]:
"""
Oil forward curve.
Returns:
raw_points : [(expiry_date, price), ...] -- REAL settlement prices
interpolated: [{"tenor", "price", "method"}, ...] -- standard tenors with formulas
"""
strip = _oil_strip(fetch_date.year)
points = []
for ticker, _ in strip.items():
try:
hist = yf.Ticker(ticker).history(period="5d")
if hist.empty:
continue
price = float(hist["Close"].iloc[-1])
exp = _contract_date(ticker, fetch_date)
if exp and (exp - fetch_date).days > 0:
points.append((exp, price))
except Exception:
continue
if len(points) < 2:
return [], []
points.sort()
days_arr = [(p[0] - fetch_date).days for p in points]
price_arr = [p[1] for p in points]
tenors_days = {"1m": 30, "3m": 91, "6m": 182, "12m": 365}
result = []
for label, days in tenors_days.items():
if days <= days_arr[0]:
result.append({"tenor": label, "price": price_arr[0],
"method": f"(nearest: {days_arr[0]}d = ${price_arr[0]:.2f})"})
elif days >= days_arr[-1]:
result.append({"tenor": label, "price": price_arr[-1],
"method": f"(nearest: {days_arr[-1]}d = ${price_arr[-1]:.2f})"})
else:
for i in range(len(days_arr) - 1):
if days_arr[i] <= days <= days_arr[i + 1]:
w = (days - days_arr[i]) / (days_arr[i + 1] - days_arr[i])
p = price_arr[i] + w * (price_arr[i + 1] - price_arr[i])
result.append({
"tenor": label, "price": p,
"method": (f"(lerp: w=({days}-{days_arr[i]})/({days_arr[i+1]}-{days_arr[i]})="
f"{w:.3f}, ${price_arr[i]:.2f}+{w:.3f}*"
f"(${price_arr[i+1]:.2f}-${price_arr[i]:.2f})=${p:.2f})")
})
break
return points, result
def fetch_rate_forward(fetch_date: date) -> List[Tuple[str, float]]:
"""Interest rate forward curve: [(contract, implied %), ...]. Implied = 100 - price."""
strip = _rate_futures_strip(fetch_date.year)
result = []
for ticker, label in strip.items():
try:
hist = yf.Ticker(ticker).history(period="5d")
if hist.empty:
continue
price = float(hist["Close"].iloc[-1])
implied = 100 - price
result.append((label, implied))
except Exception:
continue
return result
# ---------------------------------------------------------------------------
# Display
# ---------------------------------------------------------------------------
def display(data: dict):
"""Print all data with clear annotations on what is REAL vs INTERPOLATED."""
fd = data["fetch_date"]
print()
print("=" * 70)
print(f" OIL & INTEREST RATE CURVES -- {fd}")
print("=" * 70)
# ── SPOT: Oil ──
oil_price, oil_dt = data["oil_spot"]
print()
print("+-- SPOT (Latest observed) -------")
print("|")
print(f"| Brent crude (BZ=F) ${oil_price:.2f}/bbl as of {oil_dt}")
print("| [REAL: market close price from Yahoo Finance]")
print("|")
# ── SPOT: Policy rates ──
pr = data.get("policy_rates", {})
print("| Overnight policy rates")
print("| [REAL: actual observed rates from FRED (DFF, SOFR)]")
print("|")
for lbl in ["Fed Funds (EFFR)", "SOFR"]:
info = pr.get(lbl, {})
val = info.get("value", float("nan"))
dt = info.get("as_of", "N/A")
if pd.notna(val):
print(f"| {lbl:<20} {val:.2f}% as of {dt}")
else:
print(f"| {lbl:<20} N/A")
print("|")
# ── SPOT: Treasury yield curve ──
yc = data["yield_curve"]
yc_dt = data.get("yield_curve_date", "N/A")
print(f"| Treasury par yield curve (as of {yc_dt})")
print("| [REAL: published daily by U.S. Treasury, constant-maturity rates]")
print("|")
for label in ["1 Mo", "2 Mo", "3 Mo", "4 Mo", "6 Mo", "1 Yr", "2 Yr",
"3 Yr", "5 Yr", "7 Yr", "10 Yr", "20 Yr", "30 Yr"]:
if label in yc:
print(f"| {label:<6} {yc[label]:.2f}%")
print("+--")
print()
# ── FORWARD: Oil ──
oil_fwd = data["oil_forward"]
oil_raw = data.get("oil_forward_raw", [])
print("+-- OIL FORWARD CURVE (Market expectations) -------")
print("|")
if oil_raw:
print("| Raw futures (REAL: settlement prices from Yahoo Finance):")
for exp, price in oil_raw:
days = (exp - date.fromisoformat(fd)).days
print(f"| {exp} ({days:>3}d) ${price:.2f}/bbl")
if oil_fwd:
print("|")
print("| Standard tenors (INTERPOLATED via linear interpolation):")
for rec in oil_fwd:
print(f"| {rec['tenor']:>4} ${rec['price']:.2f}/bbl {rec['method']}")
else:
print("| (no data)")
print("+--")
print()
# ── FORWARD: Rates ──
rate_fwd = data["rate_forward"]
print("+-- INTEREST RATE FORWARD CURVE -------")
print("| [REAL: implied rate = 100 - futures settlement price]")
print("|")
if rate_fwd:
for label, rate in rate_fwd:
print(f"| {label:<25} {rate:.2f}%")
else:
print("| (no data)")
print("+--")
print()
# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------
def _log(msg):
import sys
sys.stderr.write(msg + "\n")
sys.stderr.flush()
def _existing_treasury_dates(path: Path) -> set:
"""Read existing treasury yield parquet and return set of dates already stored."""
pf = path / "treasury_yields.parquet"
if not pf.exists():
return set()
try:
df = pd.read_parquet(pf)
return set(pd.to_datetime(df["date"]).dt.strftime("%Y-%m-%d"))
except Exception:
return set()
def _save_treasury_incremental(df: pd.DataFrame, path: Path):
"""Append only new dates to the treasury yields parquet file."""
pf = path / "treasury_yields.parquet"
if pf.exists():
try:
existing = pd.read_parquet(pf)
existing["date"] = pd.to_datetime(existing["date"])
df["date"] = pd.to_datetime(df["date"])
existing_dates = set(existing["date"].dt.strftime("%Y-%m-%d"))
new_rows = df[~df["date"].dt.strftime("%Y-%m-%d").isin(existing_dates)]
if new_rows.empty:
_log(f" Treasury yields: 0 new dates (all {len(existing_dates)} already stored)")
return 0
combined = pd.concat([existing, new_rows], ignore_index=True)
combined = combined.sort_values("date")
combined.to_parquet(pf, index=False)
_log(f" Treasury yields: +{len(new_rows)} new dates (total: {len(combined)})")
return len(new_rows)
except Exception as e:
_log(f" Warning re-reading parquet: {e}, overwriting")
df.to_parquet(pf, index=False)
_log(f" Treasury yields: saved {len(df)} dates (new file)")
return len(df)
def _save_curves_snapshot(data: dict, fetch_date: date, path: Path):
"""Save daily snapshot of oil/rate curves to parquet (one row per fetch_date)."""
import pyarrow as pa
import pyarrow.parquet as pq
record = {"fetch_date": fetch_date.isoformat(), "oil_spot": data["oil_spot"][0]}
for label, val in data["yield_curve"].items():
record[f"yc_{label.replace(' ', '_')}"] = val
for label, info in data.get("policy_rates", {}).items():
key = label.replace(" ", "_").replace("(", "").replace(")", "")
record[f"spot_{key}"] = info.get("value")
for rec in data["oil_forward"]:
record[f"oil_fwd_{rec['tenor']}"] = rec["price"]
for label, rate in data["rate_forward"]:
key = label.replace(" ", "_").replace("-", "_")[:30]
record[f"rate_fwd_{key}"] = rate
df = pd.DataFrame([record])
df["fetch_date"] = pd.to_datetime(df["fetch_date"])
df["fetch_year"] = df["fetch_date"].dt.year
df["fetch_month"] = df["fetch_date"].dt.month
curves_path = path / "curves"
pq.write_to_dataset(
pa.Table.from_pandas(df),
root_path=str(curves_path),
partition_cols=["fetch_year", "fetch_month"],
existing_data_behavior="overwrite_or_ignore",
)
_log(f" Curves snapshot saved to {curves_path}/")
def run(save: bool = False):
fetch_date = date.today()
_log("[1/6] Fetching Treasury yield curve (XML feed) ...")
yield_curve, yc_date = fetch_yield_curve()
_log(f" {len(yield_curve)} tenors as of {yc_date}")
_log("[2/6] Fetching policy rates (FRED: EFFR + SOFR) ...")
policy_rates = fetch_policy_rates()
for lbl, info in policy_rates.items():
v = f"{info['value']:.2f}%" if pd.notna(info["value"]) else "N/A"
_log(f" {lbl}: {v} (as of {info['as_of']})")
_log("[3/6] Fetching latest oil price ...")
oil_price, oil_dt = fetch_latest_oil()
_log(f" Brent ${oil_price:.2f} (as of {oil_dt})")
_log("[4/6] Fetching oil forward curve ...")
oil_raw, oil_forward = fetch_oil_forward(fetch_date)
_log(f" {len(oil_raw)} contracts, {len(oil_forward)} interpolated tenors")
_log("[5/6] Fetching rate futures (Fed Funds, SOFR) ...")
rate_forward = fetch_rate_forward(fetch_date)
_log(f" {len(rate_forward)} contracts")
data = {
"fetch_date": fetch_date.isoformat(),
"oil_spot": (oil_price, oil_dt),
"yield_curve": yield_curve,
"yield_curve_date": yc_date,
"policy_rates": policy_rates,
"oil_forward_raw": oil_raw,
"oil_forward": oil_forward,
"rate_forward": rate_forward,
}
display(data)
if save:
_log("[6/6] Saving to parquet (incremental) ...")
PARQUET_DIR.mkdir(parents=True, exist_ok=True)
existing_dates = _existing_treasury_dates(PARQUET_DIR)
_log(f" Existing Treasury dates in parquet: {len(existing_dates)}")
today_ym = fetch_date.strftime("%Y%m")
treas_df = fetch_treasury_xml(today_ym)
if treas_df.empty:
prev = (fetch_date.replace(day=1) - timedelta(days=1))
treas_df = fetch_treasury_xml(prev.strftime("%Y%m"))
if not treas_df.empty:
_save_treasury_incremental(treas_df, PARQUET_DIR)
_save_curves_snapshot(data, fetch_date, PARQUET_DIR)
_log("")
print(f" Data saved to {PARQUET_DIR}/\n")
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Oil & interest rate curves (spot + forward)")
parser.add_argument("--save", action="store_true", help="Save to parquet")
args = parser.parse_args()
run(save=args.save)
```