Skip to main content

Using Spark Fuse™ through the API

Spark Fuse is a REST API: you submit a Docker image and a command, Spark runs it on GPU or CPU compute you do not manage, streams logs while it runs, and writes everything your job puts in /output/ into ShareSync™.


How a job works​

  1. You authenticate with POST /auth/login and reuse the bearer token on every call (tokens last 30 days).
  2. You submit POST /api/compute/jobs with an image, a command, and an instance SKU.
  3. Spark Fuse picks compute, pulls your image, and runs the command.
  4. /output/ is mounted to a ShareSync folder. Optional /input/ is mounted read-only from files you uploaded or already stored in ShareSync.
  5. Logs stream over SSE while the job runs.
  6. When the container exits, compute is released (InstantCompute can keep a short idle-hold so the next job can reuse warm compute).

Production base URL:

export SPARK_HOST="https://api.prod.aapse1.sparkcloud.studio"

InstantCompute and SmartCompute​

Set mode on each submit:

API valueProduct nameBest forBehaviour
"instant" (default)InstantComputeIteration, debugging, inference testsLands on warm GPU in about 1–2 seconds when capacity is free; otherwise a short cold start. Jobs are not interrupted mid-run. Idle-hold / warm-pool is InstantCompute only.
"smart"SmartComputeLarger, interrupt-tolerant batchesTypically cheaper (~60%). Compute can be reclaimed mid-job with a short warning. Every submit is a fresh start — no warm pool.

Guide pricing is on the Spark Fuse product page. Rates vary by SKU, region, demand, and availability.


1. Get a bearer token​

POST /auth/login is the only supported auth endpoint. Do not call identity-provider endpoints directly.

export SPARK_TOKEN=$(curl -sX POST "$SPARK_HOST/auth/login" \
-H "Content-Type: application/json" \
-d "{
\"email\": \"$YOUR_SPARK_EMAIL\",
\"password\": \"$YOUR_SPARK_PASSWORD\"
}" | jq -r '.token')

Successful body:

{
"resp": "Login Successful",
"success": true,
"token": "eyJhbGciOiJSUzI1NiIsInR5cCIgOi..."
}

HTTP 200 can still mean a failed login. Branch on success === true before using token. Failed logins return success: false and token: null.

Send the token on every compute call:

curl -s "$SPARK_HOST/api/compute/jobs" \
-H "Authorization: Bearer $SPARK_TOKEN"

The same token authenticates ShareSync WebDAV (inputs and outputs). There is no token exchange.

When a compute call returns 401, call /auth/login again and retry. There is no separate refresh endpoint.


2. List SKUs you can submit​

curl -s "$SPARK_HOST/api/compute/skus" \
-H "Authorization: Bearer $SPARK_TOKEN" | jq '.'

Use one of the returned names as instanceType on submit. The list is what Fuse accepts; it is not a guarantee that that SKU has free capacity at that moment.


3. Submit a job​

POST /api/compute/jobs

Minimum body: image, command, instanceType.

curl -sX POST "$SPARK_HOST/api/compute/jobs" \
-H "Authorization: Bearer $SPARK_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"image": "pytorch/pytorch:2.7.1-cuda12.8-cudnn9-runtime",
"command": ["bash", "-c", "nvidia-smi && echo hello-from-spark"],
"instanceType": "g7e.2xlarge"
}' | jq '.'

Typical response:

{
"jobId": "16fb6d80-0ac3-4d42-9c5d-80c02a26e5ae",
"status": "queued",
"output": {
"shareSyncPath": "/Spark Fuse Job Outputs/16fb6d80-0ac3-4d42-9c5d-80c02a26e5ae/",
"shareSyncBaseUrl": "https://your-org.files.sparkcloud.studio/dav/spaces/<id>/Spark%20Fuse%20Job%20Outputs/16fb6d80-.../"
}
}

output.shareSyncBaseUrl is the WebDAV URL for anything written to /output/.

Useful optional fields:

FieldNotes
mode"instant" (default) or "smart"
envCustomer environment variables. Keys starting with SPARK_ are reserved.
outputShareSyncPathShareSync folder for outputs. Default: /Spark Fuse Job Outputs/{jobId}/
inputShareSyncPathExisting ShareSync folder mounted read-only at /input/
inputPushMode"auto-prepare" — Fuse allocates an input folder and returns an upload URL
tagsUp to 32 tags, ^[a-z0-9_\-:\.]+$, 1–64 characters

Public linux/amd64 images work from common registries. Private images on Docker Hub and GHCR need a registry credential created through the credentials API.


4. Send input files​

Push (inputPushMode: "auto-prepare"). Submit with auto-prepare, then PUT a tar/gzip stream to input.uploadUrl from the submit response. Fuse extracts it to /input/ before your command runs.

Mount (inputShareSyncPath). Point at a ShareSync path you already filled (desktop app, WebDAV, or previous job outputs). Fuse checks the path at submit time and rejects the job if it cannot see it.

For large, stable libraries (weights, datasets) reused across jobs, use assetsShareSyncPath — mounted read-only at /assets and cached on the node.


5. Watch logs and detect completion​

Logs (not completion):

curl -N "$SPARK_HOST/api/compute/jobs/$JOB_ID/logs/stream" \
-H "Authorization: Bearer $SPARK_TOKEN"

This is Server-Sent Events. Connect as soon as you submit if you want logs from the start (there is no replay).

Do not treat the stream closing as “job finished.” Idle-hold can keep the stream open after the container exits. Poll status instead.

Status (source of truth):

curl -s "$SPARK_HOST/api/compute/jobs/$JOB_ID" \
-H "Authorization: Bearer $SPARK_TOKEN" | jq '.status'
statusMeaning
queuedWaiting for compute
provisioningCompute starting, image pulling
runningYour command is executing
succeededContainer exited 0; outputs uploaded
failedSee error_code / error_message
cancelledYou cancelled the job
smartcompute-interruptedSmartCompute reclaimed the machine

succeeded, failed, cancelled, and smartcompute-interrupted are terminal. Poll about every 5 seconds until you hit one of those.


6. List and cancel jobs​

List jobs for your organisation:

curl -s "$SPARK_HOST/api/compute/jobs" \
-H "Authorization: Bearer $SPARK_TOKEN" | jq '.'

Filter with ?tag= (AND, repeatable) or ?tagsAny=a,b (OR). List responses omit resolved shareSyncBaseUrl; call GET /api/compute/jobs/:jobId for that URL.

Cancel:

curl -sX POST "$SPARK_HOST/api/compute/jobs/$JOB_ID/cancel" \
-H "Authorization: Bearer $SPARK_TOKEN"

Cancel is idempotent. A queued job is removed before compute starts. A running job gets SIGTERM (then SIGKILL after a short grace). Cancel skips any InstantCompute idle-hold.


7. Download outputs from ShareSync​

Use the same bearer token against output.shareSyncBaseUrl from the job GET.

List:

curl -X PROPFIND \
-H "Authorization: Bearer $SPARK_TOKEN" \
-H "Depth: 1" \
"$OUTPUT_BASE_URL"

Download one file:

curl -O \
-H "Authorization: Bearer $SPARK_TOKEN" \
"$OUTPUT_BASE_URL/results.tar.gz"

After a job is terminal, the full log is also archived on ShareSync (log_archive_share_sync_path on the job row).


8. First-job script​

#!/usr/bin/env bash
set -euo pipefail

export SPARK_HOST="https://api.prod.aapse1.sparkcloud.studio"

LOGIN_RESPONSE=$(curl -sX POST "$SPARK_HOST/auth/login" \
-H "Content-Type: application/json" \
-d "{\"email\": \"$SPARK_EMAIL\", \"password\": \"$SPARK_PASSWORD\"}")

if [[ "$(echo "$LOGIN_RESPONSE" | jq -r '.success')" != "true" ]]; then
echo "Login failed: $(echo "$LOGIN_RESPONSE" | jq -r '.resp')" >&2
exit 1
fi
export SPARK_TOKEN=$(echo "$LOGIN_RESPONSE" | jq -r '.token')

SUBMIT_RESPONSE=$(curl -sX POST "$SPARK_HOST/api/compute/jobs" \
-H "Authorization: Bearer $SPARK_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"image": "pytorch/pytorch:2.7.1-cuda12.8-cudnn9-runtime",
"command": ["bash", "-c", "nvidia-smi && echo hello-from-spark"],
"instanceType": "g7e.2xlarge",
"mode": "instant"
}')

JOB_ID=$(echo "$SUBMIT_RESPONSE" | jq -r '.jobId')
echo "Submitted $JOB_ID"

while :; do
STATUS=$(curl -s "$SPARK_HOST/api/compute/jobs/$JOB_ID" \
-H "Authorization: Bearer $SPARK_TOKEN" | jq -r '.status')
echo "status=$STATUS"
case "$STATUS" in
succeeded|failed|cancelled|smartcompute-interrupted) break ;;
esac
sleep 5
done

Endpoint cheat sheet​

MethodPathPurpose
POST/auth/loginBearer token
GET/api/compute/skusAllowed instance types
POST/api/compute/jobsSubmit a job
GET/api/compute/jobsList jobs
GET/api/compute/jobs/:jobIdJob status and ShareSync URLs
GET/api/compute/jobs/:jobId/logs/streamLive logs (SSE)
POST/api/compute/jobs/:jobId/cancelCancel

info icon
Prefer ComfyUI?

You can also submit Fuse jobs from ComfyUI with the spark-fuse-comfyui-node. The node uses the same API described here.