Case study · Poridhi.io · 2025–26
GPU workspaces, from a machine per demand to a shared GPU server.
AI Studio gives each user a GPU workspace in the browser, with VS Code, Jupyter or marimo. I built its Next.js frontend, maintained the first version on AWS, and partnered with the team to move the platform onto the company’s own GPU server.
User 3's job takes the whole card. Users 1 and 2 run out of memory.
Up to 3 users, no isolation. Each could allocate the whole 16 GB, so one heavy job could starve the others.
User 3's request past 8 GB is refused. Everyone else keeps running.
6 users, each capped at 8 GB. HAMi intercepts every GPU memory allocation inside a container and refuses anything past its slice, so the cap is enforced on the GPU, not left to good behaviour.
Both bars use the same scale. Before: up to three users on one 16 GB card, nothing stopping one of them taking all of it. After: six users on one 48 GB card, each held to 8 GB.
The first version
The first version ran on AWS. When no machine had room for a new session, a workflow booted a new EC2 GPU instance. The instance installed its tools and joined a Kubernetes (K3s) cluster before an editor could start. Up to three users then shared its T4 GPU, and nothing stopped one of them using all 16 GB.
1 A whole GPU machine boots for a new demand, installs its tools and joins the cluster: 5–10 minutes before the editor can even start.
2 Up to 3 users share one T4, and nothing stops one of them using all 16 GB.
- AI Studio web appYou choose an editor and press launch. Status streams back live.
- API server (Go)Checks who you are and starts a launch workflow.
- TemporalRuns the launch and the cleanup. Machine state is kept in PostgreSQL; Redis queues tasks and credits.
- EC2 GPU instance, on AWSA spot g4dn (1× T4, 16 GB) boots and joins the K3s cluster. 5–10 minutes. Up to 3 users share it, with no isolation.
- IDE container + storage sidecarDeployed onto the instance. The sidecar mounts your files from S3 through JuiceFS.
- Ingress and secure tunnelReturn the workspace URL to your browser.
Why move
- A new machine took 5–10 minutes to be ready, before the editor even started.
- No memory isolation on the GPU: one heavy job could starve everyone else on the card.
- Spot instances could be interrupted in the middle of a session.
- Every instance cost money by the hour, even when it sat half used.
- One instance type, one region, one GPU model.
The move
The platform moved to the company’s own GPU server: one 48 GB card, split by HAMi into six 8 GB slices. HAMi sits inside each container and refuses any GPU memory allocation past its slice, so six people can share the card without touching each other’s memory. Time-slicing alone, Kubernetes’ built-in way to share a GPU, can’t enforce that.
| First version (AWS) | GPU server | |
|---|---|---|
| A new session | may wait for a machine to boot | takes a ready, pre-started pod |
| Sharing a GPU | up to 3 users, no isolation | 6 users, 8 GB each, enforced |
| Machine lifecycle | create, use, terminate | assign, use, release |
| Your files | JuiceFS on S3 | a volume on the server |
How a launch works now
The design the migration followed. Drawn from the post-prototype design document. Which details reached production is not yet confirmed.
The slow step from the first version is gone: pods are already running on the GPU server with their 8 GB slice, so a launch only assigns one and starts the editor inside it. The same workflow then watches the session until the timer ends, the user stops it, or the heartbeat stops.
- Web app → API serverLaunch with an editor and a duration.
- API server → TemporalStarts one workflow for the whole session.
- Temporal ↔ PostgreSQLChecks and deducts credits, then finds a free pre-started pod.
- Temporal → GPU serverAssigns the pod, starts the editor inside it, and checks it's ready.
- Workspace readyThe session is saved and the link returns to the browser.
- While you workThe browser sends a heartbeat every 5 minutes; the workflow waits for the timer, a stop, or silence.
- At the endThe editor stops, the pod goes back to the pool, unused credits are refunded.
What I did
| Built | The Next.js frontend where users choose their editor, memory and GPU for each session. |
| With the team | The move from AWS to the company GPU server. Bringing the backend services, database and workflow engine up on the new cluster, each with its own access controls. |
| Maintained | The first version on AWS, which I kept running and extended. |
Result
Launch times as seen in the workflow history, not a formal benchmark. The design expected launches in seconds; what still takes minutes in practice isn’t written up here yet.
Open questions in the design
- Files live on one server’s volume, so a user moved to another server wouldn’t find them there.
- A fixed pool of slices means a full server has to say “no capacity”, or queue, or fall back to the cloud.
- Both the database and HAMi know which slices are in use; keeping two records in step is its own problem.