Chrome's Gemini Nano Prompt API: A Step-by-Step Guide

The provided code snippet outlines a strategy for integrating the on-device Prompt API with a hosted fallback solution, ensuring that your application can leverage local AI capabilities when available and fall back to cloud-based services otherwise. Here's a breakdown of how this works:

1. Type Definition: The PromptFn type is defined as an asynchronous iterable function that takes a string (the prompt) and yields strings (responses in chunks).

2. Function to Get Prompt Function (getPromptFn):

   • This function attempts to use the on-device Prompt API if it's available.
   • If the device supports the Prompt API, it creates a session with a system prompt ("You are a concise assistant.") and returns an asynchronous generator that yields chunks of text as they come from the promptStreaming method.

3. Hosted Fallback:

   • If the on-device solution isn't available, the function falls back to a hosted API.
   • The hosted API is invoked via a POST request with JSON data containing the prompt and an instruction to stream the response.
   • The response from the hosted API is read in chunks using fetch and TextDecoder, which are then yielded by the asynchronous generator.

4. **Usage

Read the full article at DEV Community

---

Want to create content about this topic? Use Nemati AI tools to generate articles, social posts, and more.