Hugging Face has updated its Jobs infrastructure service, allowing developers to deploy private, OpenAI-compatible vLLM inference servers using a single command with pay-per-second billing starting at $1.50 per hour. The feature runs on containerized Hugging Face infrastructure via the vllm/vllm-openai Docker image, eliminating the need to provision Kubernetes clusters or manage dedicated servers. Developers can now spin up endpoints ranging from 4B parameter models to massive 122B mixture-of-experts models and query them directly via standard HTTP or Python clients.
How One-Command vLLM Deployment Works
To use the new deployment capabilities, developers require the huggingface_hub Python package (version 1.20.0 or higher) and a logged-in local environment. Executing the hf jobs run command launches the container, routes the specified port through Hugging Face's public jobs proxy, and exposes an OpenAI-compatible endpoint.
Deployments are billed per second based on hardware usage, such as an Nvidia A10G instance priced at $1.50 per hour, with optional hard timeouts to prevent unexpected charges. Users can monitor job status or cancel running tasks at any time using hf jobs cancel <job_id>.
Scaling Models and Agent Integration
The deployment method easily scales to larger models by adjusting hardware flavors and runtime parameters. According to Hugging Face, serving larger models like the 122B parameter Qwen 3.5 on a dual Nvidia H200 instance (h200x2) requires matching the --tensor-parallel-size flag to the GPU count and capping context length with --max-model-len 32768 to prevent memory errors.
The service also supports multiple interactive developer workflows:
- Gradio Interfaces: Developers can launch local web interfaces that stream model responses and reasoning steps side by side.
- Interactive SSH Debugging: Running jobs with the
--sshflag allows users to connect directly into the container usinghf jobs ssh <job_id>to monitor GPU memory or inspect processes. - Coding Agent Backends: Launching vLLM with
--enable-auto-tool-choiceenables terminal agent harnesses like Pi to drive file reading, editing, and execution.
HF Jobs vs. Inference Endpoints: Honest Caveats
While HF Jobs offers maximum flexibility for quick tests and batch generation, it comes with important operational trade-offs compared to dedicated managed services. Every exposed port on HF Jobs is gated by default, meaning every request must carry a Hugging Face API token with read access rather than serving as an open public API.