QVAC

Add streaming chat or a shopping assistant that answers questions about your app's catalog.

How it works

QVAC loads a model and performs inference on the device running the SDK. Your application supplies prompts and decides which actions a model may request.

The Node example uses the SDK's event stream. It downloads a model on first use, prints generated text and unloads the model while retaining its cached files.

Desktop scaffolds use a worker adapter and private lifecycle helpers. The renderer should call the worker rather than import native inference packages.

Run an example

Use Node 22.17+ and npm for installation. Start in a new directory for this example so its ./data storage cannot collide with another example. Save each file below with the exact filename shown, then run the commands in order.

mkdir qvac-demo
cd qvac-demo
npm init -y
npm install @qvac/sdk@'0.21.0'

Save as example.mjs:

import { loadModel, completion, unloadModel, QWEN3_600M_INST_Q4 } from '@qvac/sdk'
let modelId
try {
  modelId = await loadModel({ modelSrc: QWEN3_600M_INST_Q4 })
  const run = completion({
    modelId,
    history: [{ role: 'user', content: 'Explain local-first apps in one sentence.' }],
    stream: true
  })
  for await (const event of run.events) {
    if (event.type === 'contentDelta') process.stdout.write(event.text)
  }
  await run.final
  console.log()
} finally {
  if (modelId !== undefined) await unloadModel({ modelId, clearStorage: false })
}

Run from this example directory:

node example.mjs

Expected result

A generated sentence about local-first apps. Text varies by model; the first run downloads model files.

API essentials

• loadModel({ modelSrc }): prepare a model and obtain its identifier.
• completion({ modelId, history, stream }): start generation; consume events for incremental content and await final for completion.
• unloadModel({ modelId, clearStorage: false }): release the model while preserving downloaded files.

Troubleshooting and limits

• A first run needs model-download connectivity, disk space and compatible hardware. No universal memory requirement is implied.
• Check the installed SDK version before copying upstream snippets; this example uses the repository's Node SDK profile.
• Mock output does not establish native inference, cancellation or cached offline startup. Validate those independently on your target.

Next steps