FAQ

Questions about running compute jobs with ON? You’ll find the answers here.
Don’t see yours? Reach out!
Can I extend my session length later, or do I have to commit upfront?

You set a session length at launch (one hour by default), but you're not locked in, you can extend it later from the running service without interrupting it.

Can I take a Curated package and still tune it myself, different context length, quantization, or GPU count?

Yes, via Customize at the bottom of the package panel. It hands off into the Custom Models flow, carrying the model and the package's launch parameters over as a starting point, and keeps the package's resource minimums as a floor on whatever you size yourself.

What happens if I pick a second model after already configuring one?

The second model replaces the first, and any launch parameters you'd already set for the first one are discarded, not carried over. Selection is single-model, so switching means reconfiguring from scratch.

Which inference engines are supported?

vLLM and llama.cpp. vLLM needs a CUDA GPU and is the high-throughput option. llama.cpp runs on CPU or GPU and serves quantized (GGUF) models, which is what lets you run something on a node with no free GPU.

Can I run a gated or private Hugging Face model?

Yes, supply your Hugging Face access token at launch. It's held in your browser tab only, encrypted before it reaches the node, and never written into a URL.

Is the API OpenAI-compatible?

Yes. Both vLLM and llama.cpp expose an OpenAI-compatible API, so anything that calls POST /v1/chat/completions works against your service unchanged.

Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.