Create Runway videos like a Pro

3 min read • August 12, 2024

Introduction
Preparing Prompts
Executing Bash script
Results
Conclusion

Introduction

In this article, we will show how to use an experimental Runway API for RunwayML to batch-generate videos. If you are an advanced user or perhaps want to utilize your time and efforts efficiently, and are a happy owner of Runway’s Unlimited subscription, this article is for you!

As you probably already figured out, it takes many attempts before you get your perfect 10-second video. After all, there are so many ways to hint Runway. This trial and error task can be very tedious and time-consuming. But it does not have to be, enter the bright and shiny world of Automation. With a simple bash script provided below and some basic file editing, you can cut out all the waiting time and get straight to picking the winners.

Preparing Prompts

Suppose you have a few images you want to use for your video project. The first step will be to create a file prompts.json as shown below and edit it so it contains your images along with desired Runway prompts. You can put as many images as you wish and as many prompt variations as you can think of. No need to hold back, as you will not be running them manually, not anymore.

It may help to feed your images to ChatGPT, Claude or perhaps Gemini along with the Gen-3 Alpha Prompting Guide and ask them to build prompts for you. This way, you can build a lot of prompts very quickly and get it going. Later, you can see what is working and what’s not, and refine them.

[
    {
        "filename": "<Your file name #1 goes here, eg sample.jpg>",
        "prompts": [
            {
                "id": 1,
                "prompt": "<Your RunwayML Gen-3 Alpha prompt #1 goes here>"
            },
            {
                "id": 2,
                "prompt": "<Your RunwayML Gen-3 Alpha prompt #2 goes here>"
            }
        ]
    },
    {
        "filename": "<Your file name #2 goes here>",
        "prompts": [
            {
                "id": 1,
                "prompt": "<Your RunwayML Gen-3 Alpha prompt #1 goes here>"
            },
            {
                "id": 2,
                "prompt": "<Your RunwayML Gen-3 Alpha prompt #2 goes here>"
            }
        ]
    }
]

Executing Bash script

The real magic starts here. We assume that you subscribed to useapi.net. Our subscription is essentially free when you consider the time you will save. It takes only a few minutes to set up Runway with our API. Finally, you can glance over the very detailed documentation we provide. Each endpoint provides the ability to Try It right from the browser. If you don’t feel like reading the docs, that is fine too, the script below is all you need.

We will be using bash, which is built in to most *nix, Mac OS, and is also available on the Windows WSL subsystem.

Create a file runway_generate.sh with the code provided below. Execute from the terminal chmod +x runway_generate.sh to allow execution of this script.

Your images and prompts.json should be in the same folder as runway_generate.sh.

Finally, execute the script ./runway_generate.sh API_TOKEN EMAIL [NUM_GENERATE] where API_TOKEN is your useapi.net API token, and email is the email you used to configure Runway. The optional parameter NUM_GENERATE sets how many times the same prompt should be executed. If not provided, it will be set to 3.

The script will upload all the images from prompts.json to Runway and execute Gen-3 Alpha and Gen-3 Alpha Turbo with the provided prompts for each image using ExtendedMode to save your credits. This script uses Gen-3 Alpha Turbo by default to ensure faster generation times. You can set turbo: false if you wish to use the slower but more elaborate Gen-3 Alpha instead of Gen-3 Alpha Turbo.

All generated videos will be downloaded locally to your drive, so you can view them once they are ready.

#!/bin/bash

# Usage: ./runway_generate.sh API_TOKEN EMAIL [NUM_GENERATE]

# NOTE: You may need to install jq, see https://jqlang.github.io/jq/download/

# Get API token from command line
API_TOKEN=$1
EMAIL=$2

# Optional: Number of video generations per prompt, default 3
NUM_GENERATE=${3:-3}

# API endpoints
CREATE_ENDPOINT="https://api.useapi.net/v1/runwayml/gen3/create"
TASK_ENDPOINT="https://api.useapi.net/v1/runwayml/tasks/"
UPLOAD_ENDPOINT="https://api.useapi.net/v1/runwayml/assets/?"

# Reset uploaded.txt and task_ids.txt
>uploaded.txt
>task_ids.txt

# Function to upload an asset
upload_asset() {
    local file_path="$1"
    local filename=$(basename "$file_path")
    local content_type=$(file -b --mime-type "$file_path") # Get MIME type

    echo "Uploading: $filename (Content-Type: $content_type)"

    response=$(curl -s -i \
        -H "Accept: application/json" \
        -H "Content-Type: $content_type" \
        -H "Authorization: Bearer $API_TOKEN" \
        -X POST "${UPLOAD_ENDPOINT}&name=${filename}&email=${EMAIL}" \
        --data-binary @"$file_path")

    http_code=$(echo "$response" | head -n 1 | awk '{print $2}')
    response_body=$(echo "$response" | awk '/^{/ , /^}$/')

    if [[ "$http_code" == "200" ]]; then
        asset_id=$(echo "$response_body" | jq -r '.assetId')
        echo "$filename,$asset_id" >>uploaded.txt
        echo "Uploaded successfully: $filename, assetId: $asset_id"
    else
        echo "Error uploading $filename. HTTP code: $http_code, Response: $response_body"
    fi
}

# Read filenames from prompts.json and upload the associated files
jq -r '.[].filename' prompts.json | while IFS= read -r filename; do
    if [[ -f "$filename" ]]; then
        upload_asset "$filename"
    else
        echo "File not found: $filename"
    fi
done

echo "All required sample images uploaded. Asset IDs stored in 'uploaded.txt'."

# Function to create a video with retry logic for 429 errors
create_video() {
    local asset_id="$1"
    local prompt="$2"

    # 10 seconds long using exploreMode + Gen-3 Alpha Turbo 
    local payload=$(jq -n \
        --arg image_assetId "$asset_id" \
        --arg text_prompt "$prompt" \        
        '{exploreMode: true, turbo: true, image_assetId: $image_assetId, text_prompt: $text_prompt, seconds: 10}') 

    local retry_count=0
    local max_retries=10

    while [[ "$retry_count" -le "$max_retries" ]]; do
        response=$(curl -s -i \
            -H "Accept: application/json" \
            -H "Content-Type: application/json" \
            -H "Authorization: Bearer $API_TOKEN" \
            -X POST "$CREATE_ENDPOINT" \
            -d "$payload")

        http_code=$(echo "$response" | head -n 1 | awk '{print $2}')
        response_body=$(echo "$response" | awk '/^{/ , /^}$/')

        if [[ "$http_code" == "200" ]]; then
            task_id=$(echo "$response_body" | jq -r '.taskId')
            seed=$(echo "$response_body" | jq -r '.options.seed')

            if [[ "$task_id" != "null" ]]; then
                echo "Video creation started for prompt: '$prompt' with task ID: $task_id (seed: $seed)"
                echo "$asset_id,$prompt,$task_id,$seed" >>task_ids.txt
                break
            else
                echo "Error creating video for prompt: '$prompt'. Could not extract 'taskId'. Response: $response_body"
                ((retry_count++))
                sleep 30
            fi
        elif [[ "$http_code" == "429" ]]; then
            echo "Rate limit hit (429). Retrying in 30 seconds..."
            ((retry_count++))
            sleep 30
        else
            echo "Error creating video for prompt: '$prompt'. HTTP code: $http_code, Response: $response_body"
            ((retry_count++))
            sleep 30
        fi
    done

    if [[ "$retry_count" -gt "$max_retries" ]]; then
        echo "Max retries exceeded for prompt: '$prompt'"
    fi
}

# Execute video generation
while IFS=',' read -r filename asset_id; do
    jq -c '.[]' prompts.json | while IFS= read -r item; do
        current_file=$(echo "$item" | jq -r '.filename')
        if [[ "$filename" == "$current_file" ]]; then
            echo "$item" | jq -c '.prompts[]' | while IFS= read -r prompt; do
                text_prompt=$(echo "$prompt" | jq -r '.prompt')
                for ((i = 1; i <= NUM_GENERATE; i++)); do
                    create_video "$asset_id" "$text_prompt"
                done
            done
        fi
    done
done <uploaded.txt

echo "Video generation requests submitted."

# Extract task_id and seed, retrieve video URLs, and download video to local file
while IFS=',' read -r line; do
    # Extract the full task_id using regex
    task_id=$(echo "$line" | grep -oP 'user:\d+-runwayml:[^ ]+-task:[\w-]+')
    # Extract seed which is the last field in the line
    seed=$(echo "$line" | awk -F ',' '{print $NF}')

    while true; do
        task_response=$(curl -s -i \
            -H "Accept: application/json" \
            -H "Authorization: Bearer $API_TOKEN" \
            "$TASK_ENDPOINT$task_id")

        # Extract HTTP status code and response body
        http_code=$(echo "$task_response" | head -n 1 | awk '{print $2}')
        task_response_body=$(echo "$task_response" | awk '/^{/ , /^}$/')

        if [[ "$http_code" == "200" ]]; then
            task_status=$(echo "$task_response_body" | jq -r '.status')

            if [[ "$task_status" == "SUCCEEDED" ]]; then
                video_url=$(echo "$task_response_body" | jq -r '.artifacts[].url')
                # Assumed prompt is the same for all (since it's repeating), using fixed filename structure
                video_filename="${task_id}_${seed}.mp4"
                echo "Downloading: $video_url to $video_filename"
                curl -o "$video_filename" "$video_url"
                break
            elif [[ "$task_status" == "FAILED" ]]; then
                echo "Warning: Task $task_id FAILED."
                break
            else
                echo "Task $task_id is still in progress. Waiting..."
                sleep 30
            fi
        else
            echo "Error retrieving job status for $task_id. HTTP code: $http_code, Response: $task_response_body"
        fi
    done
done <task_ids.txt

echo "Video retrieval and download complete."