0. Process

This report explains how oil and interest rate curve data is produced.

Data flow (v2): Run python curves.py --save daily. It fetches live data from external APIs (Treasury, FRED, Yahoo Finance, Hyperliquid) and writes a snapshot to curves_data/curves_py/ (separate from fetch_curves to preserve full schema). When this Rmd is knitted, it reads the latest row from that parquet and uses it for all tables — no hardcoded numbers. For an up-to-date report, run curves.py --save then knit this Rmd.

python curves.py --save   # Fetch and save (run daily)
# Then knit this Rmd

The methodology proceeds in the following steps:

Fetch Treasury yield curve. The Python script (curves.py) queries the U.S. Treasury XML feed for daily par yield curve rates (constant-maturity). The feed provides 13 tenors from 1 Month to 30 Years.
Fetch overnight policy rates. The effective Federal Funds Rate (EFFR) and Secured Overnight Financing Rate (SOFR) are retrieved from FRED (Federal Reserve Economic Data) via the public CSV endpoint.
Fetch oil spot price. The latest Brent crude settlement price is fetched from Yahoo Finance (front-month contract BZ=F).
Fetch WTI perpetual price. The WTI Crude Oil perpetual contract mark price (flx:OIL) is fetched from Hyperliquid’s Felix Exchange via the info API (metaAndAssetCtxs with dex: "flx"). This provides a 24/7, on-chain reference price for oil that can be compared to the traditional Brent spot.
Fetch US Bond perpetual price. The US Bond perpetual (km:USBOND) mark price is fetched from Hyperliquid’s Markets by Kinetiq (km DEX) via the info API (metaAndAssetCtxs with dex: "km"). This provides a 24/7, on-chain reference price for U.S. Treasury bond exposure, with oracle pricing by Kaiko.
Build oil forward curve. A strip of Brent crude futures contracts is fetched from Yahoo Finance. Standard tenors (1m, 3m, 6m, 12m) are then computed via linear interpolation between the two bracketing contracts.
Build interest rate forward curve. Fed Funds (ZQ) and SOFR (SR1, SR3) futures are fetched from Yahoo Finance. Implied rates are computed as $100 - \text{settlement price}$.
Store incrementally. Treasury daily yields and km:USBOND daily candles (OHLC from Hyperliquid candleSnapshot API) are saved to Parquet. On each run, only dates not already present are appended.

1. Data Sources

Source	Data provided	URL	Lag
U.S. Treasury	Daily par yield curve (13 tenors, 1 Mo – 30 Yr)	home.treasury.gov XML feed	~1 business day
FRED	Effective Federal Funds Rate (DFF), SOFR	fred.stlouisfed.org/graph/fredgraph.csv	~1 business day
Yahoo Finance	Brent crude spot (BZ=F)	finance.yahoo.com	Real-time / EOD
Yahoo Finance	Brent crude futures strip (BZK, BZN, BZX, BZF, BZH)	finance.yahoo.com	Real-time / EOD
Yahoo Finance	Fed Funds futures (ZQ), SOFR futures (SR1, SR3)	finance.yahoo.com	Real-time / EOD
Hyperliquid DEX (Felix)	WTI Perpetual mark price (flx:OIL)	api.hyperliquid.xyz (dex: flx)	Real-time (24/7)
Hyperliquid DEX (Kinetiq)	US Bond Perpetual mark price (km:USBOND)	api.hyperliquid.xyz (dex: km)	Real-time (24/7)

2. Spot Data (Market Prices)

All values in this section are market prices — directly observed, not computed or interpolated. All values read from latest parquet snapshot.

2.1 Oil Spot

Instrument	Price	As of	Source	Classification
Brent Crude (BZ=F)	$98.91/bbl	2026-03-13	Yahoo Finance	MARKET PRICE

This is the most recent settlement (close) price for the front-month Brent crude futures contract traded on NYMEX.

2.2 WTI Perpetual (Hyperliquid DEX)

Instrument	Price	As of	Source	Classification
WTI Perpetual (flx:OIL)	$99.48/bbl	2026-03-15 08:00:00	Hyperliquid Felix Exchange	MARKET PRICE

This is the mark price for the flx:OIL (WTI Crude Oil) perpetual contract on Hyperliquid’s Felix Exchange, a perp DEX. The perpetual tracks WTI oil. Unlike traditional futures with fixed expiry dates, perpetuals use a funding rate mechanism to keep the price anchored to the underlying spot.

Live prices (24/7, including weekends): Trade flx:OIL on Hyperliquid

2.3 US Bond Perpetual (Hyperliquid Kinetiq)

Metric	Value	As of
km:USBOND (current)	86.58 pts	2026-03-15 08:00:00
Treasury 10Y @ last close	4.28%	2026-03-13
km:USBOND @ last close	86.71 pts	2026-03-13
Implied 10Y (Friday ratio)	4.29%	2026-03-15 08:00:00
Implied 10Y (10-day avg)	4.27%	2026-03-15 08:00:00

km:USBOND is a U.S. Treasury bond perpetual on Hyperliquid (Markets by Kinetiq). It trades 24/7; the U.S. 10Y cash market does not. We derive implied yields from the observed relationship between km:USBOND and Treasury 10Y:

Implied 10Y (Friday ratio): Regress Treasury 10Y on km:USBOND close over the past 10 days to get slope $b$ ($\Delta y / \Delta P$). Anchor at last close: implied = $Y_{\text{ref}} + b \times (P_{\text{now}} - P_{\text{ref}})$.
Implied 10Y (10-day avg): Same regression $Y = a + b \times P$. Implied = $a + b \times P_{\text{now}}$.

Both use the empirical slope from data; nothing is hardcoded.

Live prices (24/7): Hyperliquid | Markets by Kinetiq

2.4 Overnight Policy Rates

Rate	Value	As of	Source	Classification
Federal Funds (EFFR)	3.64%	2026-03-12	FRED series DFF	MARKET PRICE
SOFR	3.65%	2026-03-12	FRED series SOFR	MARKET PRICE

EFFR (Effective Federal Funds Rate): The volume-weighted median rate at which depository institutions lend reserve balances to each other overnight. Published daily by the Federal Reserve Bank of New York.
SOFR (Secured Overnight Financing Rate): The broad measure of the cost of borrowing cash overnight collateralised by U.S. Treasury securities. Published daily by the Federal Reserve Bank of New York.

Both are actual transacted rates, not targets — the current FOMC target range is 3.50%–3.75%.

2.5 Treasury Par Yield Curve

As of 2026-03-13 — Source: U.S. Treasury XML feed
Tenor	Yield	Classification
1 Mo	3.75%	MARKET PRICE
2 Mo	3.71%	MARKET PRICE
3 Mo	3.72%	MARKET PRICE
4 Mo	3.69%	MARKET PRICE
6 Mo	3.70%	MARKET PRICE
1 Yr	3.66%	MARKET PRICE
2 Yr	3.73%	MARKET PRICE
3 Yr	3.74%	MARKET PRICE
5 Yr	3.87%	MARKET PRICE
7 Yr	4.07%	MARKET PRICE
10 Yr	4.28%	MARKET PRICE
20 Yr	4.89%	MARKET PRICE
30 Yr	4.90%	MARKET PRICE

These are constant-maturity Treasury (CMT) rates published daily by the U.S. Department of the Treasury.

3. Forward Curves

Forward curves reflect market expectations of future prices. They are derived from futures contracts. All values read from latest parquet snapshot.

3.1 Oil Forward Curve

3.1.1 Raw Futures (Market Prices)

These are actual settlement prices of Brent crude futures contracts at various expiry dates. Each is a market price — the price at which the contract last traded or settled. Days Forward is measured from the curve reference date (fetch date).

Source: Yahoo Finance (BZ=F futures strip). Prices as of market close 2026-03-13. Curve reference date 2026-03-15.
Ticker	Contract Expiry	Days Forward	Price	Classification
BZ=F	2026-04-14	30	$98.91/bbl	MARKET PRICE
BZK26.NYM	2026-05-20	66	$103.14/bbl	MARKET PRICE
BZN26.NYM	2026-07-20	127	$94.67/bbl	MARKET PRICE
BZX26.NYM	2026-11-20	250	$83.64/bbl	MARKET PRICE
BZF27.NYM	2027-01-20	311	$80.48/bbl	MARKET PRICE
BZH27.NYM	2027-03-20	370	$78.15/bbl	MARKET PRICE

3.1.2 Interpolated Standard Tenors (Computed Prices)

The raw futures contracts do not fall on neat 1-month, 3-month, 6-month, and 12-month boundaries. To produce standard tenors, we use linear interpolation between the two bracketing contracts.

Interpolated via linear interpolation from raw futures
Tenor	Price	Classification
1m (30d)	$98.91/bbl	COMPUTED
3m (91d)	$99.67/bbl	COMPUTED
6m (182d)	$89.74/bbl	COMPUTED
12m (365d)	$78.35/bbl	COMPUTED

3.1.3 Interpolation Method: Linear Interpolation

Given two known futures prices at days $d_0$ and $d_1$ with prices $P_0$ and $P_1$, the price at target day $d$ is:

\[w = \frac{d - d_0}{d_1 - d_0}\]

\[P(d) = P_0 + w \cdot (P_1 - P_0)\]

This is sometimes called “lerp” (linear interpolation), the most common and transparent interpolation method in quantitative finance.

3.1.4 Limitations

Linear interpolation assumes the curve between two points is a straight line. It may understate curvature, particularly:

Seasonal patterns in oil (e.g. summer driving demand)
Contango/backwardation transitions

Alternative interpolation methods
Method	Advantage	Common usage
Linear (lerp)	Simple, transparent, no overshoot	Everywhere — default
Cubic spline	Smooth curve, captures curvature	Yield curve construction
Nelson-Siegel	Parsimonious yield curve model (3 params)	Central banks, fixed income
Log-linear	Better for discount factors	Swap curve bootstrapping

3.2 Interest Rate Forward Curve

Source: Yahoo Finance (ZQ, SR1, SR3 futures)
Contract	Implied Rate	Computation	Classification
Fed Funds front	3.62%	100 − futures price	MARKET PRICE (derived)
Fed Funds Apr 2026	3.64%	100 − futures price	MARKET PRICE (derived)
Fed Funds Jul 2026	3.57%	100 − futures price	MARKET PRICE (derived)
Fed Funds Oct 2026	3.50%	100 − futures price	MARKET PRICE (derived)
Fed Funds Jan 2027	3.43%	100 − futures price	MARKET PRICE (derived)
SOFR 1m front	3.67%	100 − futures price	MARKET PRICE (derived)
SOFR 3m front	3.63%	100 − futures price	MARKET PRICE (derived)
SOFR 3m Mar 2026	3.69%	100 − futures price	MARKET PRICE (derived)
SOFR 3m Jun 2026	3.63%	100 − futures price	MARKET PRICE (derived)
SOFR 3m Sep 2026	3.56%	100 − futures price	MARKET PRICE (derived)
SOFR 3m Dec 2026	3.50%	100 − futures price	MARKET PRICE (derived)

These implied rates are classified as market-price-derived rather than “interpolated,” because each rate corresponds to a specific traded futures contract. The computation is deterministic:

\[\text{implied rate} = 100 - \text{futures settlement price}\]

This is the standard convention for CME interest rate futures (Fed Funds, SOFR).

Prefix	Exchange	Underlying
ZQ	CBOT	30-Day Federal Funds Rate
SR1	CME	1-Month SOFR
SR3	CME	3-Month SOFR

4. Data Classification Summary

Data point	Classification	Source	Method
Brent crude spot	MARKET PRICE	Yahoo Finance	Last close
WTI Perpetual (flx:OIL)	MARKET PRICE	Hyperliquid Felix Exchange	Perpetual mark price (24/7)
US Bond Perpetual (km:USBOND)	MARKET PRICE	Hyperliquid Kinetiq Markets	Perpetual mark price (24/7)
Fed Funds (EFFR)	MARKET PRICE	FRED (DFF)	Last published value
SOFR	MARKET PRICE	FRED (SOFR)	Last published value
Treasury yield curve (13 tenors)	MARKET PRICE	U.S. Treasury XML feed	Last published daily curve
Oil futures strip (6 contracts)	MARKET PRICE	Yahoo Finance	Last settlement price
Oil forward 1m, 3m, 6m, 12m	COMPUTED (interpolated)	Linear interpolation of futures strip	Linear interpolation between brackets
Rate futures implied rates (11 contracts)	MARKET PRICE (derived)	Yahoo Finance (100 − price)	Arithmetic: 100 − settlement price

Key distinction: Only the oil forward curve standard tenors (1m, 3m, 6m, 12m) are interpolated. Everything else is either a direct market observation or a deterministic arithmetic transformation of one.

5. Data Storage

Results are stored in Parquet format under curves_data/:

File	Contents	Update method
treasury_yields.parquet	Daily Treasury yield curve (one row per business day, 13 tenor columns)	Incremental — only new dates are appended
hl_usbond_daily.parquet	km:USBOND daily OHLC (open, high, low, close, volume, n_trades) from Hyperliquid candleSnapshot API	Incremental — only new dates are appended
curves_py/ (partitioned)	Daily snapshot of all spot + forward data (one row per fetch date)	Overwrite-or-ignore per year/month partition

## **Treasury yields stored:** 10 dates, from 2026-03-02 to 2026-03-13

Most recent 5 trading days stored in parquet
date	1MONTH	2MONTH	3MONTH	4MONTH	6MONTH	1YEAR	2YEAR	3YEAR	5YEAR	7YEAR	10YEAR	20YEAR	30YEAR
2026-03-13	3.75	3.71	3.72	3.69	3.70	3.66	3.73	3.74	3.87	4.07	4.28	4.89	4.90
2026-03-12	3.76	3.72	3.72	3.69	3.70	3.66	3.76	3.75	3.88	4.06	4.27	4.86	4.88
2026-03-11	3.75	3.70	3.71	3.69	3.68	3.60	3.64	3.64	3.79	3.98	4.21	4.82	4.86
2026-03-10	3.75	3.71	3.71	3.69	3.68	3.56	3.57	3.58	3.73	3.93	4.15	4.74	4.78
2026-03-09	3.75	3.72	3.71	3.68	3.68	3.56	3.56	3.58	3.71	3.90	4.12	4.70	4.72

Appendix: Source Code

The full source code for curves.py is provided below.

Requirements:

pip install yfinance pandas pyarrow requests

```python
"""
curves.py
=========
Fetches and displays oil and interest rate data in a clear structure:

  1. SPOT (latest observed)
     - Latest oil price (Brent front)
     - Latest yield curve (Treasury XML feed — real data, not FRED)

  2. FORWARD CURVES (market expectations)
     - Oil forward curve (1m, 3m, 6m, 12m) — interpolated from futures strip
     - Interest rate forward curve (Fed Funds, SOFR implied from futures)

  3. PARQUET STORAGE
     - Treasury daily yields stored incrementally (only new dates added)

Data sources:
  - Treasury yield curve : home.treasury.gov XML feed (daily, ~1 day lag)
  - Oil futures          : Yahoo Finance (BZ=F strip)
  - Rate futures         : Yahoo Finance (ZQ, SR1, SR3)

Usage:
    pip install yfinance pandas pyarrow requests

    python curves.py          # Fetch and display
    python curves.py --save   # Also save to parquet (incremental)
"""

import argparse
import os
import xml.etree.ElementTree as ET
from datetime import date, datetime as dt, timedelta
from pathlib import Path
from typing import Dict, List, Tuple

import pandas as pd
import requests
import yfinance as yf

# Load .env if present
_env_file = Path(__file__).resolve().parent / ".env"
if _env_file.exists():
    for line in _env_file.read_text().splitlines():
        line = line.strip()
        if line and not line.startswith("#") and "=" in line:
            k, _, v = line.partition("=")
            k, v = k.strip(), v.strip().strip('"').strip("'")
            if k and k not in os.environ:
                os.environ[k] = v

# ---------------------------------------------------------------------------
# Config
# ---------------------------------------------------------------------------

PARQUET_DIR = Path("./curves_data")

FRED_CSV = "https://fred.stlouisfed.org/graph/fredgraph.csv"

# Treasury daily par yield curve XML feed
# https://home.treasury.gov/treasury-daily-interest-rate-xml-feed
TREASURY_XML = "https://home.treasury.gov/resource-center/data-chart-center/interest-rates/pages/xml"
# XML element names -> display labels
TREASURY_TENORS = [
    ("BC_1MONTH",  "1 Mo"),
    ("BC_2MONTH",  "2 Mo"),
    ("BC_3MONTH",  "3 Mo"),
    ("BC_4MONTH",  "4 Mo"),
    ("BC_6MONTH",  "6 Mo"),
    ("BC_1YEAR",   "1 Yr"),
    ("BC_2YEAR",   "2 Yr"),
    ("BC_3YEAR",   "3 Yr"),
    ("BC_5YEAR",   "5 Yr"),
    ("BC_7YEAR",   "7 Yr"),
    ("BC_10YEAR",  "10 Yr"),
    ("BC_20YEAR",  "20 Yr"),
    ("BC_30YEAR",  "30 Yr"),
]

# CME month codes: F=Jan, G=Feb, H=Mar, J=Apr, K=May, M=Jun, N=Jul, Q=Aug, U=Sep, V=Oct, X=Nov, Z=Dec
OIL_MONTH = {"F": 1, "G": 2, "H": 3, "J": 4, "K": 5, "M": 6, "N": 7, "Q": 8, "U": 9, "V": 10, "X": 11, "Z": 12}

# Oil strip (Brent)
def _oil_strip(year: int):
    y2, y2n = str(year)[-2:], str(year + 1)[-2:]
    return {
        "BZ=F": "Front",
        f"BZK{y2}.NYM": "May",
        f"BZN{y2}.NYM": "Jul",
        f"BZX{y2}.NYM": "Nov",
        f"BZF{y2n}.NYM": "Jan",
        f"BZH{y2n}.NYM": "Mar",
    }

# Rate futures: Fed Funds (ZQ) + SOFR (SR1, SR3). Price = 100 - implied_rate.
def _rate_futures_strip(year: int):
    y2, y2n = str(year)[-2:], str(year + 1)[-2:]
    return {
        "ZQ=F": "Fed Funds front",
        f"ZQJ{y2}.CBT": f"Fed Funds Apr-{year}",
        f"ZQN{y2}.CBT": f"Fed Funds Jul-{year}",
        f"ZQV{y2}.CBT": f"Fed Funds Oct-{year}",
        f"ZQF{y2n}.CBT": f"Fed Funds Jan-{year+1}",
        "SR1=F": "SOFR 1m front",
        "SR3=F": "SOFR 3m front",
        f"SR3H{y2}.CME": f"SOFR 3m Mar-{year}",
        f"SR3M{y2}.CME": f"SOFR 3m Jun-{year}",
        f"SR3U{y2}.CME": f"SOFR 3m Sep-{year}",
        f"SR3Z{y2}.CME": f"SOFR 3m Dec-{year}",
    }


# ---------------------------------------------------------------------------
# Fetch: Treasury yield curve (XML feed — real data from Treasury)
# ---------------------------------------------------------------------------

NS = {"d": "http://schemas.microsoft.com/ado/2007/08/dataservices",
      "m": "http://schemas.microsoft.com/ado/2007/08/dataservices/metadata",
      "a": "http://www.w3.org/2005/Atom"}


def fetch_treasury_xml(year_month: str = None) -> pd.DataFrame:
    """
    Fetch daily Treasury par yield curve from XML feed.
    year_month: "202603" for a specific month, or None for current year.
    Returns DataFrame with columns: date, BC_1MONTH, BC_2MONTH, … BC_30YEAR.
    All values are REAL data published by Treasury (not interpolated by us).
    """
    if year_month:
        params = {"data": "daily_treasury_yield_curve", "field_tdr_date_value_month": year_month}
    else:
        params = {"data": "daily_treasury_yield_curve", "field_tdr_date_value": str(date.today().year)}
    r = requests.get(TREASURY_XML, params=params, timeout=30)
    r.raise_for_status()
    root = ET.fromstring(r.content)
    rows = []
    for entry in root.findall("a:entry", NS):
        props = entry.find(".//m:properties", NS)
        if props is None:
            continue
        dt_el = props.find("d:NEW_DATE", NS)
        if dt_el is None or dt_el.text is None:
            continue
        row = {"date": dt_el.text[:10]}
        for xml_key, _ in TREASURY_TENORS:
            el = props.find(f"d:{xml_key}", NS)
            if el is not None and el.text:
                try:
                    row[xml_key] = float(el.text)
                except ValueError:
                    pass
        rows.append(row)
    df = pd.DataFrame(rows)
    if not df.empty:
        df["date"] = pd.to_datetime(df["date"])
        df = df.sort_values("date")
    return df


def fetch_yield_curve() -> Tuple[dict, str]:
    """
    Latest Treasury yield curve from XML feed.
    Returns (dict of label->value, date_str). All values are real Treasury data.
    """
    today = date.today()
    ym = today.strftime("%Y%m")
    df = fetch_treasury_xml(ym)
    if df.empty:
        prev = (today.replace(day=1) - timedelta(days=1))
        df = fetch_treasury_xml(prev.strftime("%Y%m"))
    if df.empty:
        return {}, "N/A"
    row = df.iloc[-1]
    dt_str = row["date"].strftime("%Y-%m-%d")
    result = {}
    for xml_key, label in TREASURY_TENORS:
        if xml_key in row and pd.notna(row[xml_key]):
            result[label] = row[xml_key]
    return result, dt_str


def fetch_latest_oil() -> Tuple[float, str]:
    """Latest Brent crude price ($/bbl). REAL market close price."""
    t = yf.Ticker("BZ=F")
    hist = t.history(period="5d")
    if hist.empty:
        return 0.0, "N/A"
    price = float(hist["Close"].iloc[-1])
    dt = hist.index[-1].strftime("%Y-%m-%d")
    return price, dt


def _fred_series(series_id: str) -> Tuple[float, str]:
    """Fetch latest value for a single FRED series via public CSV endpoint."""
    from io import StringIO
    try:
        r = requests.get(FRED_CSV, params={"id": series_id}, timeout=20)
        r.raise_for_status()
        df = pd.read_csv(StringIO(r.text))
        date_col = next((c for c in ["DATE", "Date", "date"] if c in df.columns), df.columns[0])
        if series_id not in df.columns:
            return float("nan"), "N/A"
        df = df[df[series_id].astype(str) != "."].dropna(subset=[series_id])
        if df.empty:
            return float("nan"), "N/A"
        df[date_col] = pd.to_datetime(df[date_col])
        row = df.iloc[-1]
        return float(row[series_id]), row[date_col].strftime("%Y-%m-%d")
    except Exception:
        return float("nan"), "N/A"


def fetch_policy_rates() -> dict:
    """
    Fetch actual policy rates from FRED:
      - DFF  : Effective Federal Funds Rate (daily, ~1 day lag)
      - SOFR : Secured Overnight Financing Rate (daily, ~1 day lag)
    These are REAL observed rates, not futures-implied.
    """
    result = {}
    for series_id, label in [("DFF", "Fed Funds (EFFR)"), ("SOFR", "SOFR")]:
        val, dt = _fred_series(series_id)
        result[label] = {"value": val, "as_of": dt}
    return result


def fetch_hyperliquid_oil() -> Tuple[float, str]:
    """
    Fetch WTI Oil perpetual (flx:OIL) mark price from Hyperliquid.
    flx:OIL is on Felix Exchange, a perp DEX on Hyperliquid.
    Returns (price, source_label) or (0.0, "N/A") on failure.
    """
    try:
        # Felix Exchange (flx) has flx:OIL; main meta has no oil
        r = requests.post(
            "https://api.hyperliquid.xyz/info",
            json={"type": "metaAndAssetCtxs", "dex": "flx"},
            timeout=15,
        )
        r.raise_for_status()
        data = r.json()
        meta, asset_ctxs = data[0], data[1]
        oil_idx = next((i for i, a in enumerate(meta["universe"]) if a["name"] == "flx:OIL"), None)
        if oil_idx is not None and oil_idx < len(asset_ctxs):
            ctx = asset_ctxs[oil_idx]
            mark_px = ctx.get("markPx")
            if mark_px is not None:
                return float(mark_px), "Hyperliquid (Felix)"
    except Exception:
        pass
    return 0.0, "N/A"


def fetch_hyperliquid_usbond() -> Tuple[float, str]:
    """
    Fetch US Bond perpetual (km:USBOND) mark price from Hyperliquid.
    km:USBOND is on Markets by Kinetiq (km DEX), a perp DEX on Hyperliquid.
    Tracks U.S. Treasury bond exposure; oracle by Kaiko. Trades 24/7.
    Returns (price, source_label) or (0.0, "N/A") on failure.
    """
    try:
        r = requests.post(
            "https://api.hyperliquid.xyz/info",
            json={"type": "metaAndAssetCtxs", "dex": "km"},
            timeout=15,
        )
        r.raise_for_status()
        data = r.json()
        meta, asset_ctxs = data[0], data[1]
        idx = next((i for i, a in enumerate(meta["universe"]) if a["name"] == "km:USBOND"), None)
        if idx is not None and idx < len(asset_ctxs):
            ctx = asset_ctxs[idx]
            mark_px = ctx.get("markPx")
            if mark_px is not None:
                return float(mark_px), "Hyperliquid (Kinetiq)"
    except Exception:
        pass
    return 0.0, "N/A"


def fetch_hyperliquid_usbond_candles(days: int = 14) -> pd.DataFrame:
    """
    Fetch km:USBOND daily candles from Hyperliquid candleSnapshot API.
    Returns DataFrame with columns: date, open, high, low, close, volume, n_trades.
    """
    try:
        end = dt.utcnow()
        start = end - timedelta(days=days)
        start_ms = int(start.timestamp() * 1000)
        end_ms = int(end.timestamp() * 1000)
        r = requests.post(
            "https://api.hyperliquid.xyz/info",
            json={
                "type": "candleSnapshot",
                "req": {
                    "coin": "km:USBOND",
                    "interval": "1d",
                    "startTime": start_ms,
                    "endTime": end_ms,
                },
            },
            timeout=15,
        )
        r.raise_for_status()
        candles = r.json()
        if not candles:
            return pd.DataFrame()
        rows = []
        for c in candles:
            ts_ms = c["t"]
            dt_val = dt.utcfromtimestamp(ts_ms / 1000).date()
            rows.append({
                "date": dt_val.isoformat(),
                "open": float(c["o"]),
                "high": float(c["h"]),
                "low": float(c["l"]),
                "close": float(c["c"]),
                "volume": float(c["v"]),
                "n_trades": int(c["n"]),
            })
        return pd.DataFrame(rows)
    except Exception:
        return pd.DataFrame()


# ---------------------------------------------------------------------------
# Fetch: Forward curves
# ---------------------------------------------------------------------------

def _contract_date(ticker: str, fetch_date: date):
    """Approx expiry from ticker. BZK26.NYM, ZQJ26.CBT, SR1J26.CME."""
    if "=F" in ticker:
        return fetch_date + timedelta(days=30)
    try:
        if ticker.startswith("BZ") and len(ticker) >= 5:
            m = OIL_MONTH.get(ticker[2])
            y = 2000 + int(ticker[3:5])
        elif ticker.startswith("ZQ") and len(ticker) >= 5:
            m = OIL_MONTH.get(ticker[2])
            y = 2000 + int(ticker[3:5])
        elif ticker.startswith("SR") and len(ticker) >= 6:
            m = OIL_MONTH.get(ticker[3])
            y = 2000 + int(ticker[4:6])
        else:
            return None
        return date(y, m, 20) if m else None
    except (IndexError, ValueError):
        return None


def fetch_oil_forward(fetch_date: date) -> Tuple[List[Tuple[str, date, float]], list, str]:
    """
    Oil forward curve.
    Returns:
        raw_points : [(ticker, expiry_date, price), ...] -- REAL settlement prices
        interpolated: [{"tenor", "price", "method"}, ...]  -- standard tenors with formulas
        price_date : date of last market close (YYYY-MM-DD)
    """
    strip = _oil_strip(fetch_date.year)
    points = []
    price_date_str = "N/A"
    for ticker, _ in strip.items():
        try:
            hist = yf.Ticker(ticker).history(period="5d")
            if hist.empty:
                continue
            price = float(hist["Close"].iloc[-1])
            if not points:  # capture date from first successful fetch
                price_date_str = hist.index[-1].strftime("%Y-%m-%d")
            exp = _contract_date(ticker, fetch_date)
            if exp and (exp - fetch_date).days > 0:
                points.append((ticker, exp, price))
        except Exception:
            continue
    if len(points) < 2:
        return [], [], price_date_str
    points.sort(key=lambda p: p[1])  # sort by expiry date
    days_arr = [(p[1] - fetch_date).days for p in points]
    price_arr = [p[2] for p in points]
    tenors_days = {"1m": 30, "3m": 91, "6m": 182, "12m": 365}
    result = []
    for label, days in tenors_days.items():
        if days <= days_arr[0]:
            result.append({"tenor": label, "price": price_arr[0],
                           "method": f"(nearest: {days_arr[0]}d = ${price_arr[0]:.2f})"})
        elif days >= days_arr[-1]:
            result.append({"tenor": label, "price": price_arr[-1],
                           "method": f"(nearest: {days_arr[-1]}d = ${price_arr[-1]:.2f})"})
        else:
            for i in range(len(days_arr) - 1):
                if days_arr[i] <= days <= days_arr[i + 1]:
                    w = (days - days_arr[i]) / (days_arr[i + 1] - days_arr[i])
                    p = price_arr[i] + w * (price_arr[i + 1] - price_arr[i])
                    result.append({
                        "tenor": label, "price": p,
                        "method": (f"(lerp: w=({days}-{days_arr[i]})/({days_arr[i+1]}-{days_arr[i]})="
                                   f"{w:.3f}, ${price_arr[i]:.2f}+{w:.3f}*"
                                   f"(${price_arr[i+1]:.2f}-${price_arr[i]:.2f})=${p:.2f})")
                    })
                    break
    return points, result, price_date_str


def fetch_rate_forward(fetch_date: date) -> List[Tuple[str, float]]:
    """Interest rate forward curve: [(contract, implied %), ...]. Implied = 100 - price."""
    strip = _rate_futures_strip(fetch_date.year)
    result = []
    for ticker, label in strip.items():
        try:
            hist = yf.Ticker(ticker).history(period="5d")
            if hist.empty:
                continue
            price = float(hist["Close"].iloc[-1])
            implied = 100 - price
            result.append((label, implied))
        except Exception:
            continue
    return result


# ---------------------------------------------------------------------------
# Display
# ---------------------------------------------------------------------------

def display(data: dict):
    """Print all data with clear annotations on what is REAL vs INTERPOLATED."""
    fd = data["fetch_date"]
    print()
    print("=" * 70)
    print(f"  OIL & INTEREST RATE CURVES -- {fd}")
    print("=" * 70)

    # ── SPOT: Oil ──
    oil_price, oil_dt = data["oil_spot"]
    print()
    print("+-- SPOT (Latest observed) -------")
    print("|")
    print(f"|  Brent crude (BZ=F)     ${oil_price:.2f}/bbl     as of {oil_dt}")
    print("|  [REAL: market close price from Yahoo Finance]")

    hl_price, hl_src = data.get("hl_oil", (0.0, "N/A"))
    if hl_price:
        print(f"|  WTI Perp (flx:OIL)    ${hl_price:.2f}/bbl     via {hl_src}")
        print("|  [REAL: mark price from Hyperliquid Felix Exchange perpetual]")
    usbond_price, usbond_src = data.get("hl_usbond", (0.0, "N/A"))
    if usbond_price:
        print(f"|  US Bond Perp (km:USBOND)  {usbond_price:.2f} pts   via {usbond_src}")
        print("|  [REAL: mark price from Hyperliquid Kinetiq Markets perpetual]")
    print("|")

    # ── SPOT: Policy rates ──
    pr = data.get("policy_rates", {})
    print("|  Overnight policy rates")
    print("|  [REAL: actual observed rates from FRED (DFF, SOFR)]")
    print("|")
    for lbl in ["Fed Funds (EFFR)", "SOFR"]:
        info = pr.get(lbl, {})
        val = info.get("value", float("nan"))
        dt = info.get("as_of", "N/A")
        if pd.notna(val):
            print(f"|     {lbl:<20} {val:.2f}%     as of {dt}")
        else:
            print(f"|     {lbl:<20} N/A")
    print("|")

    # ── SPOT: Treasury yield curve ──
    yc = data["yield_curve"]
    yc_dt = data.get("yield_curve_date", "N/A")
    print(f"|  Treasury par yield curve   (as of {yc_dt})")
    print("|  [REAL: published daily by U.S. Treasury, constant-maturity rates]")
    print("|")
    for label in ["1 Mo", "2 Mo", "3 Mo", "4 Mo", "6 Mo", "1 Yr", "2 Yr",
                   "3 Yr", "5 Yr", "7 Yr", "10 Yr", "20 Yr", "30 Yr"]:
        if label in yc:
            print(f"|     {label:<6}  {yc[label]:.2f}%")
    print("+--")
    print()

    # ── FORWARD: Oil ──
    oil_fwd = data["oil_forward"]
    oil_raw = data.get("oil_forward_raw", [])
    print("+-- OIL FORWARD CURVE (Market expectations) -------")
    print("|")
    if oil_raw:
        print("|  Raw futures (REAL: settlement prices from Yahoo Finance):")
        for ticker, exp, price in oil_raw:
            days = (exp - date.fromisoformat(fd)).days
            print(f"|     {exp} ({days:>3}d)   ${price:.2f}/bbl")
    if oil_fwd:
        print("|")
        print("|  Standard tenors (INTERPOLATED via linear interpolation):")
        for rec in oil_fwd:
            print(f"|     {rec['tenor']:>4}   ${rec['price']:.2f}/bbl   {rec['method']}")
    else:
        print("|     (no data)")
    print("+--")
    print()

    # ── FORWARD: Rates ──
    rate_fwd = data["rate_forward"]
    print("+-- INTEREST RATE FORWARD CURVE -------")
    print("|  [REAL: implied rate = 100 - futures settlement price]")
    print("|")
    if rate_fwd:
        for label, rate in rate_fwd:
            print(f"|     {label:<25}  {rate:.2f}%")
    else:
        print("|     (no data)")
    print("+--")
    print()


# ---------------------------------------------------------------------------
# Main
# ---------------------------------------------------------------------------

def _log(msg):
    import sys
    sys.stderr.write(msg + "\n")
    sys.stderr.flush()


def _existing_treasury_dates(path: Path) -> set:
    """Read existing treasury yield parquet and return set of dates already stored."""
    pf = path / "treasury_yields.parquet"
    if not pf.exists():
        return set()
    try:
        df = pd.read_parquet(pf)
        return set(pd.to_datetime(df["date"]).dt.strftime("%Y-%m-%d"))
    except Exception:
        return set()


def _save_treasury_incremental(df: pd.DataFrame, path: Path):
    """Append only new dates to the treasury yields parquet file."""
    pf = path / "treasury_yields.parquet"
    if pf.exists():
        try:
            existing = pd.read_parquet(pf)
            existing["date"] = pd.to_datetime(existing["date"])
            df["date"] = pd.to_datetime(df["date"])
            existing_dates = set(existing["date"].dt.strftime("%Y-%m-%d"))
            new_rows = df[~df["date"].dt.strftime("%Y-%m-%d").isin(existing_dates)]
            if new_rows.empty:
                _log(f"       Treasury yields: 0 new dates (all {len(existing_dates)} already stored)")
                return 0
            combined = pd.concat([existing, new_rows], ignore_index=True)
            combined = combined.sort_values("date")
            combined.to_parquet(pf, index=False)
            _log(f"       Treasury yields: +{len(new_rows)} new dates (total: {len(combined)})")
            return len(new_rows)
        except Exception as e:
            _log(f"       Warning re-reading parquet: {e}, overwriting")
    df.to_parquet(pf, index=False)
    _log(f"       Treasury yields: saved {len(df)} dates (new file)")
    return len(df)


def _existing_hl_usbond_dates(path: Path) -> set:
    """Read existing hl_usbond_daily parquet and return set of dates already stored."""
    pf = path / "hl_usbond_daily.parquet"
    if not pf.exists():
        return set()
    try:
        df = pd.read_parquet(pf)
        return set(pd.to_datetime(df["date"]).dt.strftime("%Y-%m-%d"))
    except Exception:
        return set()


def _save_hl_usbond_incremental(df: pd.DataFrame, path: Path):
    """Append only new dates to hl_usbond_daily.parquet."""
    pf = path / "hl_usbond_daily.parquet"
    if pf.exists():
        try:
            existing = pd.read_parquet(pf)
            existing["date"] = pd.to_datetime(existing["date"])
            df["date"] = pd.to_datetime(df["date"])
            existing_dates = set(existing["date"].dt.strftime("%Y-%m-%d"))
            new_rows = df[~df["date"].dt.strftime("%Y-%m-%d").isin(existing_dates)]
            if new_rows.empty:
                _log(f"       km:USBOND daily: 0 new dates (all {len(existing_dates)} already stored)")
                return 0
            combined = pd.concat([existing, new_rows], ignore_index=True)
            combined = combined.sort_values("date")
            combined.to_parquet(pf, index=False)
            _log(f"       km:USBOND daily: +{len(new_rows)} new dates (total: {len(combined)})")
            return len(new_rows)
        except Exception as e:
            _log(f"       Warning re-reading hl_usbond parquet: {e}, overwriting")
    df.to_parquet(pf, index=False)
    _log(f"       km:USBOND daily: saved {len(df)} dates (new file)")
    return len(df)


def _save_curves_snapshot(data: dict, fetch_date: date, path: Path):
    """Save daily snapshot of oil/rate curves to parquet (one row per fetch_date)."""
    import pyarrow as pa
    import pyarrow.parquet as pq

    record = {"fetch_date": fetch_date.isoformat(), "oil_spot": data["oil_spot"][0]}
    record["oil_spot_date"] = data["oil_spot"][1]
    record["yc_date"] = data.get("yield_curve_date", "N/A")

    hl_px, _ = data.get("hl_oil", (0.0, "N/A"))
    if hl_px:
        record["hl_wti_perp"] = hl_px
    usbond_px, _ = data.get("hl_usbond", (0.0, "N/A"))
    record["hl_usbond_perp"] = usbond_px if usbond_px else None
    for label, val in data["yield_curve"].items():
        record[f"yc_{label.replace(' ', '_')}"] = val
    for label, info in data.get("policy_rates", {}).items():
        key = label.replace(" ", "_").replace("(", "").replace(")", "")
        record[f"spot_{key}"] = info.get("value")
        record[f"spot_{key}_as_of"] = info.get("as_of", "N/A")
    for rec in data["oil_forward"]:
        record[f"oil_fwd_{rec['tenor']}"] = rec["price"]

    # Oil raw futures: (ticker, expiry, price)
    oil_raw = data.get("oil_forward_raw", [])
    if oil_raw:
        record["oil_raw_tickers"] = [p[0] for p in oil_raw]
        record["oil_raw_expiries"] = [p[1].isoformat() for p in oil_raw]
        record["oil_raw_days"] = [(p[1] - fetch_date).days for p in oil_raw]
        record["oil_raw_prices"] = [p[2] for p in oil_raw]
        record["oil_raw_price_date"] = data.get("oil_raw_price_date", "N/A")

    for label, rate in data["rate_forward"]:
        key = label.replace(" ", "_").replace("-", "_")[:30]
        record[f"rate_fwd_{key}"] = rate

    df = pd.DataFrame([record])
    df["fetch_date"] = pd.to_datetime(df["fetch_date"])
    df["fetch_year"] = df["fetch_date"].dt.year
    df["fetch_month"] = df["fetch_date"].dt.month
    curves_path = path / "curves_py"  # Separate from fetch_curves to preserve oil_raw_* schema
    pq.write_to_dataset(
        pa.Table.from_pandas(df),
        root_path=str(curves_path),
        partition_cols=["fetch_year", "fetch_month"],
        existing_data_behavior="overwrite_or_ignore",
    )
    _log(f"       Curves snapshot saved to {curves_path}/")


def run(save: bool = False):
    fetch_date = date.today()

    _log("[1/8] Fetching Treasury yield curve (XML feed) ...")
    yield_curve, yc_date = fetch_yield_curve()
    _log(f"       {len(yield_curve)} tenors as of {yc_date}")

    _log("[2/8] Fetching policy rates (FRED: EFFR + SOFR) ...")
    policy_rates = fetch_policy_rates()
    for lbl, info in policy_rates.items():
        v = f"{info['value']:.2f}%" if pd.notna(info["value"]) else "N/A"
        _log(f"       {lbl}: {v} (as of {info['as_of']})")

    _log("[3/8] Fetching latest oil price ...")
    oil_price, oil_dt = fetch_latest_oil()
    _log(f"       Brent ${oil_price:.2f} (as of {oil_dt})")

    _log("[4/8] Fetching Hyperliquid WTI Perpetual (flx:OIL) ...")
    hl_price, hl_src = fetch_hyperliquid_oil()
    if hl_price:
        _log(f"       WTI Perp ${hl_price:.2f} (via {hl_src})")
    else:
        _log(f"       WTI Perp unavailable ({hl_src})")

    _log("[5/8] Fetching Hyperliquid US Bond Perpetual (km:USBOND) ...")
    usbond_price, usbond_src = fetch_hyperliquid_usbond()
    if usbond_price:
        _log(f"       US Bond Perp {usbond_price:.2f} (via {usbond_src})")
    else:
        _log(f"       US Bond Perp unavailable ({usbond_src})")

    _log("[6/8] Fetching oil forward curve ...")
    oil_raw, oil_forward, oil_raw_price_date = fetch_oil_forward(fetch_date)
    _log(f"       {len(oil_raw)} contracts, {len(oil_forward)} interpolated tenors (prices as of {oil_raw_price_date})")

    _log("[7/8] Fetching rate futures (Fed Funds, SOFR) ...")
    rate_forward = fetch_rate_forward(fetch_date)
    _log(f"       {len(rate_forward)} contracts")

    data = {
        "fetch_date": fetch_date.isoformat(),
        "oil_spot": (oil_price, oil_dt),
        "hl_oil": (hl_price, hl_src),
        "hl_usbond": (usbond_price, usbond_src),
        "yield_curve": yield_curve,
        "yield_curve_date": yc_date,
        "policy_rates": policy_rates,
        "oil_forward_raw": oil_raw,
        "oil_forward": oil_forward,
        "oil_raw_price_date": oil_raw_price_date,
        "rate_forward": rate_forward,
    }

    display(data)

    if save:
        _log("[8/8] Saving to parquet (incremental) ...")
        PARQUET_DIR.mkdir(parents=True, exist_ok=True)

        existing_dates = _existing_treasury_dates(PARQUET_DIR)
        _log(f"       Existing Treasury dates in parquet: {len(existing_dates)}")

        today_ym = fetch_date.strftime("%Y%m")
        treas_df = fetch_treasury_xml(today_ym)
        if treas_df.empty:
            prev = (fetch_date.replace(day=1) - timedelta(days=1))
            treas_df = fetch_treasury_xml(prev.strftime("%Y%m"))
        if not treas_df.empty:
            _save_treasury_incremental(treas_df, PARQUET_DIR)

        _log("       Fetching km:USBOND daily candles (Hyperliquid API) ...")
        usbond_candles = fetch_hyperliquid_usbond_candles(days=14)
        if not usbond_candles.empty:
            _save_hl_usbond_incremental(usbond_candles, PARQUET_DIR)
        else:
            _log("       km:USBOND candles: no data")

        _save_curves_snapshot(data, fetch_date, PARQUET_DIR)
        _log("")
        print(f"  Data saved to {PARQUET_DIR}/\n")


if __name__ == "__main__":
    import sys
    import os
    parser = argparse.ArgumentParser(description="Oil & interest rate curves (spot + forward)")
    parser.add_argument("--save", action="store_true", help="Save to parquet")
    args = parser.parse_args()
    run(save=args.save)
    sys.stdout.flush()
    sys.stderr.flush()
    # Force exit: yfinance/requests can leave background threads that block normal exit
    os._exit(0)

```

Oil & Interest Rate Curves Report (v2 — Live Data)

ML, Cursor AI

2026-03-15