The provided code snippet outlines a strategy for integrating the on-device Prompt API with a hosted fallback solution, ensuring that your application can leverage local AI capabilities when available and fall back to cloud-based services otherwise. Here's a breakdown of how this works:
-
Type Definition: The
PromptFntype is defined as an asynchronous iterable function that takes a string (the prompt) and yields strings (responses in chunks). -
Function to Get Prompt Function (
getPromptFn):- This function attempts to use the on-device Prompt API if it's available.
- If the device supports the Prompt API, it creates a session with a system prompt ("You are a concise assistant.") and returns an asynchronous generator that yields chunks of text as they come from the
promptStreamingmethod.
-
Hosted Fallback:
- If the on-device solution isn't available, the function falls back to a hosted API.
- The hosted API is invoked via a POST request with JSON data containing the prompt and an instruction to stream the response.
- The response from the hosted API is read in chunks using
fetchandTextDecoder, which are then yielded by the asynchronous generator.
-
**Usage
Read the full article at DEV Community
Want to create content about this topic? Use Nemati AI tools to generate articles, social posts, and more.

![[AINews] The Unreasonable Effectiveness of Closing the Loop](/_next/image?url=https%3A%2F%2Fmedia.nemati.ai%2Fmedia%2Fblog%2Fimages%2Farticles%2F600e22851bc7453b.webp&w=3840&q=75)



