MarineTraffic Optimization

1. Current Workflow Audit

What's Working

Where the Friction Lives

The bottleneck is past track and historical data. Daily positions work great. But when Will asks "Where has OCTOPUS been for the last 3 months?" or "Show me HAMPSHIRE II's seasonal pattern," the system has to:

1. Look up the vessel's SHIP_ID (requires navigating to the vessel page on MT)
2. Manually construct the Dense Track URL with date ranges
3. Execute via Chrome MCP JS injection
4. Parse the raw JSON response
5. Manually interpret lat/lon clusters into port calls and passages
6. Have no way to compare against prior seasons

2. Past Track Extraction Strategy

2.1 The Dense Track Endpoint (Already Documented)

https://www.marinetraffic.com/map/gettrackjson/shipid:{SHIP_ID}/stdate:{YYYY-MM-DD}/endate:{YYYY-MM-DD}/trackorigin:livetrack

This is an internal browser endpoint — not the paid API. It works as long as there's an active MarineTraffic session cookie in Chrome. The Essential plan ($1,000/year, currently active) gives 7 days of past track.

2.2 SHIP_ID Discovery — The Missing Piece

The Dense Track endpoint requires MarineTraffic's internal SHIP_ID, not MMSI or IMO. Currently we only have M&EM's SHIP_ID (6996019) documented. We need all 54.

How to extract SHIP_IDs in bulk (one-time operation):

// Run this in Chrome MCP on the EXPLORE DATA page after loading a fleet
const gridEl = document.querySelector('[role="grid"]');
const fiberKey = Object.keys(gridEl).find(k => k.startsWith('__reactFiber'));
let fiber = gridEl[fiberKey];
let rows = null;
let attempts = 0;
while (fiber && attempts < 50) {
  if (fiber.memoizedProps?.rows?.length > 5) {
    rows = fiber.memoizedProps.rows;
    break;
  }
  fiber = fiber.return;
  attempts++;
}
// Extract SHIP_ID mapping
rows.map(r => [r.SHIPNAME, r.SHIP_ID, r.MMSI, r.IMO].join(',')).join('\n')

2.3 Optimized Past Track Pull

For a single vessel:

1. Ensure Chrome has an active MarineTraffic session
2. Execute Dense Track fetch via javascript_tool in Chrome MCP
3. Parse into structured position records
4. Store in vessel_positions table

Practical constraint: Chrome MCP JS execution has output length limits. For 17 TIER ONE vessels x 7 days x ~77 positions/day = ~9,200 positions. Solution: batch in groups of 3-5 vessels per call.

3. Itinerary Reverse-Engineering Engine

3.1 The Algorithm

INPUT:  [(lat, lon, speed, course, timestamp), ...]  sorted by timestamp
OUTPUT: [
  { type: 'port_call', port: 'Nassau', arrival: T1, departure: T2, duration: '3d 4h' },
  { type: 'passage', from: 'Nassau', to: 'Palm Beach', distance: 185, avg_speed: 11.2 },
  { type: 'anchorage', location: 'Exuma Cays', lat: 24.2, lon: -76.4, duration: '1d 8h' },
]

3.2 Position Clustering (Stop Detection)

Group consecutive positions where speed < 0.5kn into stops. Merge nearby stops (vessel swinging on anchor) within 0.5nm.

3.3 Seasonal Pattern Detection

With sufficient historical data (collected daily over weeks/months), detect primary cruising grounds per month, typical ports, average days at sea vs in port, and repositioning months.

3.4 Speed Profile Analysis

4. Mainframe Schema for Vessel Intelligence

Six tables: vessels, vessel_positions (time-series AIS data), port_calls (derived from position clustering), voyages (reconstructed legs between stops), anchorages (known anchorage locations), and daily_snapshots (one row per vessel per day).

5. Automation Plan

5.1 Daily Position Pull

After each pull, automatically write positions to the mainframe vessel_positions and daily_snapshots tables. This builds the historical record passively.

5.2 Weekly Past Track Harvest

Run once per week for each TIER ONE vessel (17 vessels). 17 x 539 positions = ~9,163 positions per week. ~2 minutes total execution.

5.3 Monthly Full Fleet Harvest

Same as weekly, but for all 51 MT-tracked vessels. ~27,500 positions per month. Still tiny for SQLite.

5.4 SHIP_ID Bootstrap (One-Time)

Run the bulk extraction script once, store in vessels.ship_id. Takes ~30 seconds.

5.5 Error Handling

6. Quick Wins — Implement Now

1. Extract All SHIP_IDs Today — Single highest-value action. Unlocks past track for all 51 vessels.
2. Store Every Daily Pull in the Mainframe — After 30 days: a month of time-series data. After 90 days: seasonal patterns.
3. Build the Known Ports / Anchorages Reference Table — Seed from superyacht hubs already documented.
4. Compact Output Format for Dense Track — Reduces output size by ~70%.
5. Pre-Built Dense Track URLs — Generate ready-to-use URLs for all TIER ONE vessels.

7. Architecture Summary

                Chrome MCP (Browser)
                       |
          +------------+-------------+
          |                          |
EXPLORE DATA Grid            Dense Track Endpoint
(React Fiber extract)        (gettrackjson fetch)
~51 vessels, live positions  ~77 pos/day per vessel
          |                          |
          v                          v
     Parse + Normalize          Parse + Normalize
          |                          |
          +------------+-------------+
                       |
                mainframe.py
                (SQLite DB)
                       |
          +------+-----+------+------+
          |      |           |       |
      vessels  vessel_    port_   voyages
               positions  calls
               daily_     anchorages
               snapshots
                       |
                Intelligence Layer
                       |
          +------+-----+------+
          |      |            |
      Daily    Weekly      On-Demand
      Brief    Report      Queries

The Flywheel Effect

Each daily pull feeds the database. The database enables pattern detection. Patterns enable better briefs. Better briefs justify the pull. The system gets smarter every day with zero additional effort from Will.

After 90 days: ~90 daily snapshots per vessel, ~8,100 Dense Track positions per TIER ONE vessel, enough data to reconstruct full seasonal itineraries.

After 12 months: full annual patterns — the ability to predict where any vessel is likely to be in any given month.

Appendix A: Essential Plan Limitation

The current Essential plan limits past track to 7 days. To build longer histories, we must harvest weekly and stitch together. This is why the weekly harvest cadence is critical — miss a week and we lose that week's Dense Track resolution forever.

Appendix B: Haversine Distance Function

import math

def haversine(lat1, lon1, lat2, lon2):
    """Return distance in nautical miles between two coordinates."""
    R = 3440.065  # Earth radius in nautical miles
    lat1, lon1, lat2, lon2 = map(math.radians, [lat1, lon1, lat2, lon2])
    dlat = lat2 - lat1
    dlon = lon2 - lon1
    a = math.sin(dlat/2)**2 + math.cos(lat1) * math.cos(lat2) * math.sin(dlon/2)**2
    return 2 * R * math.asin(math.sqrt(a))

This document is the blueprint. The mainframe schema is implemented and seeded. The fleet is loaded. The system is ready to start accumulating intelligence.

-- Neo