Create Runway videos like a Pro
3 min read • August 12, 2024 (October 22, 2024)
Table of contents
Introduction
In this article, we will show how to use an experimental Runway API for RunwayML to batch-generate videos. If you are an advanced user or perhaps want to utilize your time and efforts efficiently, and are a happy owner of Runway’s Unlimited subscription, this article is for you!
As you probably already figured out, it takes many attempts before you get your perfect 10-second video. After all, there are so many ways to hint Runway. This trial and error task can be very tedious and time-consuming. But it does not have to be, enter the bright and shiny world of Automation. With a simple script provided below and some basic file editing, you can cut out all the waiting time and get straight to picking the winners.
Preparing Prompts
Suppose you have a few text prompts and/or images you want to use for your video project. The first step will be to create a file prompts.json as shown below and edit it so it contains your images along with desired Runway prompts. You can put as many images as you wish and as many prompt variations as you can think of. No need to hold back, as you will not be running them manually, not anymore.
It may help to feed your images to ChatGPT, Claude or perhaps Gemini along with the Gen-3 Alpha Prompting Guide and ask them to build prompts for you. This way, you can build a lot of prompts very quickly and get it going. Later, you can see what is working and what’s not, and refine them.
If you are not using images as a starting point, you can still use the above approach to generate text prompts.
You can provide an first and last images (frames) in addition to your text prompt. Currently .png, .gif and .jpeg are supported. You can also use .webp images, just change the extension to .jpeg.
Give our experimental Midjourney API a try to get initial images generated from your prompt. This can help drive your video generation in a more precise way.
Please note that the format of the provided below prompts.json changed in script v2.0 (October 18, 2024).
Expand prompts.json
[
{
"text_prompt": "By default the Gen-3 Alpha Turbo with exploreMode ON and 5-second settings will be used."
},
{
"turbo": true,
"exploreMode": true,
"seconds": 10,
"aspect_ratio": "portrait",
"firstImage": "./first_image.jpeg",
"lastImage": "./last_image.jpeg",
"text_prompt": "You can explicitly specify all parameters (use first/lastImage for first/lastImage_assetId) for the Gen-3 Alpha Turbo. For more details, see https://useapi.net/docs/api-runwayml-v1/post-runwayml-gen3turbo-create"
},
{
"turbo": false,
"exploreMode": true,
"seconds": 10,
"image": "./image.jpeg",
"image_as_end_frame": true,
"text_prompt": "You can explicitly specify all parameters (use image for image_assetId) for the Gen-3 Alpha. For more details, see https://useapi.net/docs/api-runwayml-v1/post-runwayml-gen3-create"
}
]
Executing script
The real magic starts here. We assume that you subscribed to useapi.net. Our subscription is essentially free when you consider the time you will save. It takes only a few minutes to set up Runway with our API. Finally, you can glance over the very detailed API documentation we provide. Each endpoint provides the ability to Try It right from the browser, for example Gen-3 Alpha » Try It or Gen-3 Alpha Turbo » Try It. If you don’t feel like reading the docs, that is fine too, the script below is all you need.
We will be using Node.js to execute JavaScript script provided below. Please download and install version 21 or older.
Create a file runwayml.mjs with the code provided below.
Your prompts.json should be in the same folder as runwayml.mjs.
Finally, execute the script node ./runwayml.mjs API_TOKEN EMAIL where API_TOKEN is your useapi.net API token and EMAIL is configured Runway account email you want to use.
All generated videos will be downloaded locally to your drive, so you can view them once they are ready.
The script will upload all the images from prompts.json to Runway and execute Gen-3 Alpha or Gen-3 Alpha Turbo with the provided prompts for each image using ExtendedMode to save your credits. This script uses Gen-3 Alpha Turbo by default to ensure faster generation times. You can set "turbo": false if you wish to use the slower but more elaborate Gen-3 Alpha instead of Gen-3 Alpha Turbo.
All generated videos will be downloaded locally to your drive, so you can view them once they are ready.
Expand runwayml.mjs script
/*
Script version 2.1, October 22, 2024
Script to generate videos using prompts with Runway API v1 by useapi.net 🚀
For more details visit https://useapi.net/docs/api-runwayml-v1
Installation Instructions:
==========================
You need Node.js v21 or newer installed to run this script. Download and install Node.js from:
- Windows, macOS, Linux: https://nodejs.org/
After installation, verify by running the following commands in a terminal:
node -v
Running the Script:
===================
Usage: node runwayml.mjs <API_TOKEN> <EMAIL> [PROMPTS_FILE]
Replace API_TOKEN with your actual useapi.net API token, see https://useapi.net/docs/start-here/setup-useapi
Replace EMAIL with configured Runway email account, see https://useapi.net/docs/start-here/setup-runwayml
If optional PROMPTS_FILE not provided prompts.json will be used.
Example #1:
--------
node runwayml.mjs user:1234-abcdefhijklmnopqrstuv [email protected]
This command executes the script using API token user:1234-abcdefhijklmnopqrstuv with [email protected] Runway account email.
Example #2:
--------
node runwayml.mjs user:1234-abcdefhijklmnopqrstuv [email protected] myprompts.json
This command executes the script using API token user:1234-abcdefhijklmnopqrstuv with [email protected] Runway account email and load prompts from myprompts.json file.
Changelog:
==========
- October 22, 2024: Small bug fix with parameter validation.
*/
import readline from 'node:readline';
import fs from 'fs/promises';
import path from 'path';
import { writeFile } from 'node:fs/promises';
import { Readable } from 'node:stream';
// Constants
const RESULTS_FILE = 'runwayml_results.txt';
const ERRORS_FILE = 'runwayml_errors.txt';
const DEFAULT_PROMPTS_FILE = 'prompts.json';
const SLEEP_429 = 10 * 1000; // in milliseconds
const SLEEP_DOWNLOAD = 20 * 1000; // in milliseconds
const urlAccounts = 'https://api.useapi.net/v1/runwayml/accounts';
const urlGen3Create = 'https://api.useapi.net/v1/runwayml/gen3/create';
const urlGen3CreateTurbo = 'https://api.useapi.net/v1/runwayml/gen3turbo/create';
const urlDownload = 'https://api.useapi.net/v1/runwayml/tasks/';
const urlUploadAsset = 'https://api.useapi.net/v1/runwayml/assets/?email=';
// To upload .webp rename it to .jpeg
const supportedFileExtensions = ['png', 'jpeg', 'gif']
// { filename: assetId }
const uploadedFiles = {};
// Utility to sleep for given milliseconds
const sleep = (ms) => new Promise((resolve) => setTimeout(resolve, ms));
// Function to fetch configured Runway API accounts
async function fetchAccounts(apiToken) {
const response = await fetch(urlAccounts, {
headers: {
'Accept': 'application/json',
'Authorization': `Bearer ${apiToken}`
}
});
if (!response.ok) {
console.error(`⛔ Error fetching accounts (HTTP ${response.status}): ${response.statusText}`);
process.exit(1);
}
return response.json();
}
const elapsedTimeSec = (start) => (Date.now() - start) / 1000;
async function uploadAsset(apiToken, email, filename) {
// Check if already uploaded
if (uploadedFiles.hasOwnProperty(filename))
return uploadedFiles[filename];
const startTime = Date.now();
console.log(`⬆️ Account ${email} uploading file…`, filename);
const body = new Blob([await fs.readFile(filename)]);
const name = path.basename(filename);
const fileExt = filename.split('.').pop();
const response = await fetch(`${urlUploadAsset}${email}&name=${name}`, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Authorization': `Bearer ${apiToken}`,
'Content-Type': `image/${fileExt}`
},
body
});
if (response.ok) {
const json = await response.json();
const { assetId } = json;
console.log(`🆗 assetId (${elapsedTimeSec(startTime)} sec)`, assetId);
uploadedFiles[filename] = assetId;
}
else {
console.error(`❗ Unable to upload file HTTP ${response.status} (${elapsedTimeSec(startTime)} sec)`, await response.text());
// Do not attempt to upload failed file again
uploadedFiles[filename] = undefined;
}
return uploadedFiles[filename];
}
async function submit(apiToken, url, body, index, text_prompt) {
const createResponse = await fetch(url, {
method: 'POST',
headers: {
'Accept': 'application/json',
'Content-Type': 'application/json',
'Authorization': `Bearer ${apiToken}`
},
body
});
const createBody = await createResponse.text();
if (createResponse.status == 200) {
const json = JSON.parse(createBody);
const { taskId } = json;
if (taskId) {
await fs.appendFile(RESULTS_FILE, `${taskId},#${index}:${text_prompt}\n`);
console.log(`✅ taskId`, taskId);
return 200;
} else {
const error = `No taskId found in HTTP 200 response`;
console.log(`❓ ${error}`, createBody);
await fs.appendFile(ERRORS_FILE, `${error},#${index}:${text_prompt}\n`);
return 500;
}
} else {
switch (createResponse.status) {
case 429:
console.log(`🔄️ Retry on HTTP ${createResponse.status}`);
break;
case 422:
console.log(`🛑 MODERATED prompt`, createBody);
await fs.appendFile(ERRORS_FILE, `${createResponse.status},#${index}:${text_prompt}\n`);
break;
case 412:
console.log(`🛑 account run out of credits`, createBody);
break;
default:
console.log(`❗ FAILED with HTTP ${createResponse.status}`, createBody);
await fs.appendFile(ERRORS_FILE, `${createResponse.status},#${index}:${text_prompt}\n`);
}
return createResponse.status;
}
}
async function submitGen3Turbo(apiToken, email, prompt, index) {
const { firstImage, lastImage, text_prompt, aspect_ratio, seed } = prompt;
const exploreMode = prompt?.exploreMode ?? true;
const seconds = prompt?.seconds ?? 5;
console.log(`🚀 Gent-3 Alpha Turbo » Prompt #${index} • account ${email} • exploreMode ${exploreMode ? 'ON' : 'OFF'} • ${seconds} secs …`);
const firstImage_assetId = firstImage ? await uploadAsset(apiToken, email, firstImage) : undefined;
const lastImage_assetId = lastImage ? await uploadAsset(apiToken, email, lastImage) : undefined;
const body = JSON.stringify({ firstImage_assetId, lastImage_assetId, text_prompt, aspect_ratio, seconds, seed, exploreMode })
return await submit(apiToken, urlGen3CreateTurbo, body, index, text_prompt);
}
async function submitGen3(apiToken, email, prompt, index) {
const { image, image_as_end_frame, text_prompt, seed } = prompt;
const exploreMode = prompt?.exploreMode ?? true;
const seconds = prompt?.seconds ?? 5;
console.log(`✈️ Gent-3 Alpha » Prompt #${index} • account ${email} • exploreMode ${exploreMode ? 'ON' : 'OFF'} • ${seconds} secs …`);
const image_assetId = image ? await uploadAsset(apiToken, email, image) : undefined;
const body = JSON.stringify({ email, image_assetId, image_as_end_frame, text_prompt, seconds, seed, exploreMode })
return await submit(apiToken, urlGen3Create, body, index, text_prompt);
}
// Function to download videos
async function download(apiToken) {
if (! await fileExists(RESULTS_FILE)) return;
try {
const resultsContent = await fs.readFile(RESULTS_FILE, 'utf8');
const lines = resultsContent.trim().split('\n');
for (const line of lines) {
const [taskId, prompt] = line.split(',');
const videoFilename = `${taskId.replace(/:/g, '_')}.mp4`;
console.log(`👉 ${taskId}`);
try {
await fs.access(videoFilename);
console.log(`⚠️ ${videoFilename} already exists. Skipping download.`);
continue;
} catch {
// File does not exist, proceed with downloading
}
while (true) {
const response = await fetch(`${urlDownload}${taskId}`, {
headers: {
'Accept': 'application/json',
'Authorization': `Bearer ${apiToken}`
}
});
if (!response.ok) {
console.log(`🛑 Download failed ${taskId} (HTTP ${response.status}):\n${prompt}\n`, await response.text());
break;
}
const taskResponseBody = await response.json();
const { status, artifacts, error, progressRatio, estimatedTimeToStartSeconds } = taskResponseBody;
if (status == 'FAILED') {
console.error(`🛑 FAILED ${taskId} (${error}):\n${prompt}\n`);
break;
}
if (status == 'SUCCEEDED') {
const url = artifacts.at(0).url;
if (url) {
console.log(`✅ Downloading ${url} to ${videoFilename}`);
try {
const videoResponse = await fetch(url);
if (!videoResponse.ok) {
console.error(`⛔ Unable to download ${taskId} (HTTP ${videoResponse.status}):\n${prompt}\n`, url);
break;
}
const stream = Readable.fromWeb(videoResponse.body);
await writeFile(videoFilename, stream);
} catch (err) {
console.error(`⛔ Error during download: ${err}`);
}
} else
console.error(`🛑 Unable to download ${taskId} status (${status} ${error}):\n${prompt}\n`);
break;
}
console.log(`⌛ ${taskId} status (${status}) and is still in progress (${progressRatio * 100}%, seconds to start ${estimatedTimeToStartSeconds}), waiting…`);
await sleep(SLEEP_DOWNLOAD);
}
}
} catch (error) {
console.log(`⛔ Error during download:`, error.stack || error);
}
}
// Main function
async function main() {
const apiToken = process.argv[2];
const email = process.argv[3];
const promptFile = process.argv[4] || DEFAULT_PROMPTS_FILE;
if (!apiToken || !email) {
console.error('Usage: node runwayml.mjs <API_TOKEN> <EMAIL> [PROMPTS_FILE]');
process.exit(1);
}
console.info('Script v2.1');
console.info('Node version is: ' + process.version);
try {
if (await fileExists(RESULTS_FILE)) {
let user_input;
while (!['y', 'n'].includes(user_input)) {
user_input = (await promptUser(`❔ ${RESULTS_FILE} file detected. Do you want to download the results now? (y/n): `))?.toLowerCase();
if (user_input == 'y') {
await download(apiToken);
await fs.unlink(RESULTS_FILE);
}
}
}
const start = new Date();
try {
console.info('START EXECUTION', start);
await execute(apiToken, email, promptFile); // Pass the promptFile to execute function
}
finally {
console.info('COMPLETED', new Date());
console.info('EXECUTION ELAPSED', diffInMinutesAndSeconds(start, new Date()));
}
try {
console.info('START DOWNLOAD', start);
await download(apiToken);
}
finally {
console.info('TOTAL ELAPSED', diffInMinutesAndSeconds(start, new Date()));
}
} catch (error) {
console.error('⛔ Error during execution:', error.stack || error);
}
}
// Modify the execute function to accept promptFile as a parameter
async function execute(apiToken, email, promptFile) {
const accounts = await fetchAccounts(apiToken);
console.info(`Configured Runway API accounts (${Object.values(accounts).length}):`, Object.values(accounts).map(a => a.email).join(', '));
if (Object.values(accounts).length <= 0) {
console.error(`⛔ No configured Runway accounts found. Please refer to https://useapi.net/docs/start-here/setup-runwayml`);
process.exit(1);
}
if (!accounts[email]) {
console.error(`⛔ Accounts ${email} not found. Please refer to https://useapi.net/docs/start-here/setup-runwayml`);
process.exit(1);
}
if (accounts[email].error) {
console.error(`⛔ Accounts ${email} has pending error. Please resolve and update account at https://useapi.net/docs/api-runwayml-v1/post-runwayml-accounts-email`);
process.exit(1);
}
const promptData = await fs.readFile(promptFile, 'utf8');
const prompts = JSON.parse(promptData);
console.log(`Total number of prompts to process`, prompts.length);
let warnings = [];
const paramsGen3AlphaTurbo = ['turbo', 'firstImage', 'lastImage', 'text_prompt', 'aspect_ratio', 'seconds', 'seed', 'exploreMode'];
const paramsGen3Alpha = ['turbo', 'text_prompt', 'seconds', 'seed', 'image', 'image_as_end_frame', 'exploreMode'];
const invalidKeys = (supported, prompt) => Object.keys(prompt).filter(key => !key.startsWith('__') && !supported.includes(key))
for (let i = 1; i <= prompts.length; i++) {
const prompt = prompts[i - 1];
const { turbo, text_prompt, firstImage, lastImage, image } = prompt;
const validateImage = async (file) => {
if (file) {
try {
await fs.access(file);
} catch {
warnings.push(`⚠️ Image '${file}' does not exist. Prompt ${i}`);
}
const ext = file.split('.').pop();
if (!supportedFileExtensions.includes(ext))
warnings.push(`⚠️ Image ${file} extension ${ext} now supported. Prompt ${i}`);
}
};
if (turbo === false) {
const notSupported = invalidKeys(paramsGen3Alpha, prompt);
if (notSupported.length)
warnings.push(`⚠️ Gen-3 Alpha » following params not supported: ${notSupported.join(',')}. Prompt ${i}`);
if (!text_prompt && !image)
warnings.push(`⚠️ Gen-3 Alpha » please specify text_prompt or image or both. Prompt ${i}`);
}
else {
const notSupported = invalidKeys(paramsGen3AlphaTurbo, prompt);
if (notSupported.length)
warnings.push(`⚠️ Gen-3 Alpha Turbo » following params not supported: ${notSupported.join(',')}. Prompt ${i}`);
if (!firstImage && !lastImage)
warnings.push(`⚠️ Gen-3 Alpha Turbo » please specify firstImage or lastImage or both. Prompt ${i}`);
}
await Promise.all([validateImage(firstImage), validateImage(lastImage), validateImage(image)]);
}
if (warnings.length > 0) {
warnings.forEach(warning => console.warn(warning));
console.error(`⛔ Execution stopped due to warnings.`);
process.exit(1);
}
for (let i = 0; i < prompts.length; i++) {
const prompt = prompts[i];
while (true) {
const responseCode = prompt?.turbo === false ?
await submitGen3(apiToken, email, prompt, i + 1) :
await submitGen3Turbo(apiToken, email, prompt, i + 1);
if (responseCode == 429)
await sleep(SLEEP_429);
else
if (responseCode == 412) {
process.exit(1);
} else
break;
}
}
}
// Utility function to check if a file exists
async function fileExists(path) {
try {
await fs.access(path);
return true;
} catch {
return false;
}
}
// Function to prompt user input
async function promptUser(query) {
const rl = readline.createInterface({
input: process.stdin,
output: process.stdout
});
return new Promise((resolve) => rl.question(query, answer => {
rl.close();
resolve(answer);
}));
}
function diffInMinutesAndSeconds(date1, date2) {
const diffInSeconds = Math.floor((date2 - date1) / 1000);
return `${Math.floor(diffInSeconds / 60)} minutes ${diffInSeconds % 60} seconds`;
};
main();
Examples
Gen-3 Alpha Turbo » A beautiful blonde woman with striking blue eyes and an hourglass figure, posing confidently as if for a magazine cover.
Gen-3 Alpha Turbo » Ghost Rider riding chrome-detailed motorcycle.
Gen-3 Alpha
Conclusion
Visit our 
Discord Server or 
Telegram Channel for any support questions and concerns.
We regularly post guides and tutorials on the 
YouTube Channel.
Check our GitHub repo with code examples.