Adam M. Smith rndmcnlly

The Case for Scoped JWTs in LLM Inference APIs

Anyone operating an LLM inference API with API key authentication -- whether a direct provider or an aggregator -- should consider supporting scoped JWT tokens. DeepInfra already does this well. The pattern is general and solves real problems that the rest of the industry is working around with proxies and key management sprawl.

The problem

Organizations that distribute LLM API access to their users (universities, SaaS platforms, dev teams) currently have two options:

Give each user a real API key via a management API. This works, but the organization loses control the moment the key leaves their hands. Keys can be shared, leaked, or used in ways the organization didn't intend. Revoking a key often destroys its analytics history. And provisioning keys is a heavyweight operation -- there's no cheap way to issue thousands of ephemeral credentials.

USER

Let’s test this new fs tool

ASSISTANT

Sure! Let me start by asking you to share a local folder so I can browse it.

Tool Executed

A folder has been shared! Let me take a look at what's inside:

Framework for Student Project Teams Developing GenAI Policies

The Decision Context

Your team must write an AGENTS.md file that will govern AI tool behavior for your project this quarter. This isn't a personal moral stance—it's a collective operational policy that must be:

Specific enough for tools to parse and conform to
Justified enough that teammates with different underlying concerns can commit to it
Revisable as circumstances or understanding change

	<!-- Vibe-coded with Gambit v1.4 (https://bayleaf.chat/?model=gambit) -->
	<!-- PITCH TIMER: Because every second of their attention is precious -->
	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="UTF-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<title>⏱️ PITCH TIMER</title>
	<style>
	/* === TWEAK THESE === */

	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="UTF-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no">
	<title>Ambient Synth Rack</title>
	<style>
	* { margin: 0; padding: 0; box-sizing: border-box; -webkit-user-select: none; user-select: none; }

	body {


	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="UTF-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<style>
	body {
	background-color: white; /* Ensure the iframe has a white background */
	}

	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="UTF-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0">
	<title>MERIDIAN</title>
	<style>
	* {
	margin: 0;
	padding: 0;

	<!DOCTYPE html>
	<html lang="en">
	<head>
	<meta charset="UTF-8">
	<meta name="viewport" content="width=device-width, initial-scale=1.0, user-scalable=no">
	<title>THE PLANT</title>
	<script src="https://cdnjs.cloudflare.com/ajax/libs/tone/14.8.49/Tone.js"></script>
	<style>
	* { margin: 0; padding: 0; box-sizing: border-box; }
	@font-face { font-family: 'Brutalist'; src: local('Courier New'); }

	<!-- Vibe-coded with Gambit v1.3 (https://bayleaf.chat/?model=gambit) -->
	<!-- DESIGN QUESTION: Does a bounded park with optional destinations feel like -->
	<!-- peaceful contentment or subtle confinement? Watch how you move. Do you -->
	<!-- visit things purposefully, or drift? Do you test the fences? -->
	<!DOCTYPE html>
	<html>
	<head>
	<title>Park Walk</title>
	<style>
	* { margin: 0; box-sizing: border-box; }