gemini-omni-flash-api – Coding skill for AI agents

Gemini Omni Flash Skill

When to Use

Use this skill for generative video editing, text-to-video, image-referenced video generation, and first-frame-to-video transition animations using the official google-genai SDK. Includes workflows for pre-processing/optimizing high-resolution or long source videos with ffmpeg.

This skill uses the Gemini Omni Flash model (gemini-omni-flash-preview) to perform text to video generation, image to video generation and video editing.

> [!WARNING]

> Important Regional Restrictions: Uploading videos to use for video edits is NOT available in the EEA, Switzerland, the United Kingdom, and some US states. If a video-to-video edit completes quickly with empty outputs (total_output_tokens: 0 or no video content), check the current availability documentation and API error; empty output alone does not identify the cause.

Core capabilities

1. Video editing and refinement: Editing existing videos (maximum duration 10 seconds), applying stylistic changes, or performing inpainting/outpainting.

2. Text to video: Generating videos from a text prompt.

3. First-frame to video: Generating videos from a single input image.

4. Image-referenced generation: Using style, character, or object references from images to guide video generation.

Data and execution boundaries

These scripts upload media and prompts to Google and can incur API charges. Confirm the selected media, remote processing and batch size are within the user request before calling. Use GEMINI_API_KEY from the environment; avoid command-line keys in shell history. Work from an isolated project directory; choose new output paths because existing files are refused. Inspect generated media and report missing or partial output honestly. Never infer a regional restriction solely from empty output; inspect API errors and current official availability. Model access, SDK version requirements, limits and pricing must be checked against official documentation before use; this static bundle does not establish them.

Workflow

1. Analyze request: Determine the target task (e.g., first-frame-to-video, reference-guided editing) and identify any input media assets.

2. Run SDK scripts:

• Directly run the appropriate utility (scripts/video/generate_video.py or scripts/upload_file.py).
• Configure settings like --aspect-ratio (e.g. 16:9, 9:16) and --duration (any integer between 3 and 10 seconds, e.g. 3, 5, 10).

3. Retrieve and process output: Outputs are saved to the local filesystem (e.g. media/). Report back the completed media path to the user.

Reference Documentation

• Interactions API: All operations and state management for the Gemini Omni Flash model (gemini-omni-flash-preview) are handled via the [Interactions API](https://ai.google.dev/gemini-api/docs/interactions-overview).
• Files API: Input media files (such as reference images and videos) must be uploaded via the [Files API](https://ai.google.dev/gemini-api/docs/interactions/files) first before being referenced in generations. The uploaded file URI and MIME type are then included in the interactions.create input parts array.
• [Interactions API Skill Reference](https://github.com/google-gemini/gemini-skills/blob/main/skills/gemini-interactions-api/SKILL.md): Platform-wide guidelines, current model specifications, and SDK usage rules for the Interactions API.

Dependencies and Prerequisites

• Python SDK (google-genai): Requires google-genai >= 2.10.0 (Python) to support the new interactions client attribute. Install or upgrade using:

  pip install -U google-genai
  

• Python Runtime: Requires Python >= 3.10 (for compatibility with modern google-genai SDK types and methods).
• ffmpeg & ffprobe: prep_video.py, inspect_video.py, and generate_video.py (when stripping audio via --strip-audio) require ffmpeg and ffprobe binaries installed and available in your system PATH.

Available scripts

Use the following Python scripts to upload media with the Files API, prepare input videos with ffmpeg, and generate video outputs using the Interactions API.

1. [upload_file.py](scripts/upload_file.py): Uploads local media (images and videos) to the Files API and polls until ACTIVE. If uploading a video larger than 25MB, it prints an informative warning/tip highlighting that Gemini Omni Flash is optimized for editing 10s videos at 720p/24fps, and recommends pre-processing with prep_video.py first to speed up the upload.

   python3 scripts/upload_file.py path/to/image.png
   

2. [generate_video.py](scripts/video/generate_video.py): Performs end-to-end video generation and downloads the output video. It detects and uploads local media references (images or videos) before calling the Interactions API. Large video assets (>25MB) will trigger informative pre-processing recommendations without blocking the upload.

• Text to video:

     python3 scripts/video/generate_video.py "A close-up of a cat drinking tea" --output media/cat_tea.mp4
     

• Image to video (first frame and reference):

     python3 scripts/video/generate_video.py "The waves crash against the shore." --image reference.png --output media/waves.mp4
     

• Video interpolation:

Provide exactly two images as keyframes to generate a transition video between them:

     python3 scripts/video/generate_video.py "A smooth timelapse from sunrise to sunset" --image start.png --image end.png --output media/interpolation.mp4
     

• Video editing (keep original audio):

     python3 scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --output media/anime_style.mp4
     

• Video editing (regenerate all audio from scratch):

     python3 scripts/video/generate_video.py "Transform the style to Japanese anime" --video input.mp4 --strip-audio --output media/anime_style_new_audio.mp4
     

• Turn-by-turn video editing (edit previous interaction):

Edit a prior video generation without re-uploading assets by passing the interaction ID:

     python3 scripts/video/generate_video.py "Change the setting to a snowy winter wonderland." --previous-interaction-id "abc123xyz..." --output media/winter_wonderland.mp4
     

• Parallel batch execution (prompts file): Run multiple prompts from a line-by-line text file concurrently:

     python3 scripts/video/generate_video.py --prompts-file prompts.txt --concurrency 3
     

• Parallel batch execution (JSON config): Execute fully configured, distinct generation and editing jobs in parallel:

     python3 scripts/video/generate_video.py --batch jobs.json --concurrency 3
     

*Example jobs.json:*

     [
       {
         "prompt": "Transform the style to Japanese anime.",
         "video": "input.mp4",
         "output": "media/anime_style.mp4",
         "strip_audio": false,
         "aspect_ratio": "16:9"
       },
       {
         "prompt": "A smooth timelapse from sunrise to sunset.",
         "image": ["start.png", "end.png"],
         "output": "media/interpolation.mp4"
       }
     ]
     

3. [inspect_video.py](scripts/video/inspect_video.py): Inspects a local video file (using ffprobe) to check its duration, resolution, frame rate (FPS), audio stream presence, and format details.

   python3 scripts/video/inspect_video.py media/output.mp4
   

• To get a pre-parsed, structured JSON summary:

     python3 scripts/video/inspect_video.py media/output.mp4 --json
     

• To get the complete, unmodified ffprobe raw JSON dump:

     python3 scripts/video/inspect_video.py media/output.mp4 --raw
     

4. [prep_video.py](scripts/video/prep_video.py): Normalizes, trims, and formats any video file to fit standard Gemini Omni Flash generation and editing limits. It handles timecode-based trimming, optional frame rate conversion, and proportional scaling of large videos (max 1280x720 for landscape, 720x1280 for portrait) to optimize upload times without stretching. If the video is longer than 10 seconds and the script is run interactively (in a TTY), it prompts the user to select the first 10s, last 10s, or enter a custom timecode (defaulting to the first 10s).

• Trim first 10s (default):

     python3 scripts/video/prep_video.py path/to/source.mp4
     

or explicitly specify the start and duration:

     python3 scripts/video/prep_video.py path/to/source.mp4 --start 0 --duration 10
     

• Trim last 10s (automatically calculates starting point based on source length):

     python3 scripts/video/prep_video.py path/to/source.mp4 --start last
     

• **Tri

… (full content is in the download package)