logan-markewich/jeff

A self-hosted implementation of TypeSafe's jev System One API, powered by GLiFormer (400M parameters). Use the official typesafe-sdk by pointing TYPESAFEBASEURL at jeff. Supports choice (pick an option), score (rate on ordered levels), and noul (probability of yes). Cheaper to self-host, but less accurate than jev on reasoning-heavy tasks. See benchmarks. Quickstart · Deploy · Configuration · API · Development Requires uv and Python 3.12. Run from the repository root: Serves at http://localhost:8000. Device selection: CUDA → MPS → CPU. The sync above also installs typesafe-sdk. Save this as example.py: In another terminal: For an existing SDK app, set TYPESAFEAPIKEY=devkey and TYPESAFEBASEURL=http://localhost:8000 instead of passing client arguments. curl example Use L4 for the HTTP API. These commands download weights once and deploy with one warm container: For an ephemeral URL with no warm container: Deploy settings: JEFFGPU=L4, JEFFMINCONTAINERS=1, JEFFMAXCONTAINERS=8, JEFFMAXINPUTS=64, JEFFTARGETINPUTS=16. Server JEFF variables are forwarded. The GPU image defaults…