YouTube Transcript to JSON With Python: Fetch, Clean and Structure Captions

To convert a YouTube transcript to JSON with Python, install youtube-transcript-api, create a YouTubeTranscriptApi() instance, call .fetch(video_id), then pass .to_raw_data() to json.dump or use the library's JSONFormatter. That gives you raw caption snippets. For LLMs, search or RAG, add one more step: merge the fragments into sentences, drop tags like [Music] and save everything in a stable schema with the video ID, language and timestamps.

This guide uses version 1.2.4 of the library, the current release on PyPI. Many tutorials still call YouTubeTranscriptApi.get_transcript(). That static method was deprecated in 1.0.0 and removed in 1.2.0, so that code fails today. Every example below uses the current instance API.

What you get from YouTube captions

A fetched transcript is a list of short snippets. Each snippet has three fields:

• text: the caption text shown on screen.
• start: when the snippet appears, in seconds from the beginning of the video.
• duration: how long the snippet stays on screen, in seconds.

The transcript also carries metadata: video_id, language, language_code and is_generated, which tells you whether YouTube created the captions automatically or a person uploaded them.

Two details matter later. First, duration is screen time, not speech time, and the library's own source code warns that snippets can overlap. Second, snippets are display fragments, not sentences. Auto-generated captions in particular split sentences across snippets and can include sound tags such as [Music] or [Applause]. Raw snippets are fine for subtitles, but they are awkward input for a language model.

Install and fetch the transcript

The library supports Python 3.8 through 3.14. Pin the version, because the 1.x line has already shipped breaking changes:

pip install youtube-transcript-api==1.2.4

Pass the video ID, not the full URL. A small helper handles the common link formats:

from urllib.parse import urlparse, parse_qs

from youtube_transcript_api import YouTubeTranscriptApi


def video_id_from_url(url):
    parsed = urlparse(url)
    if parsed.hostname == "youtu.be":
        return parsed.path.lstrip("/")
    if parsed.path == "/watch":
        return parse_qs(parsed.query)["v"][0]
    if parsed.path.startswith(("/shorts/", "/embed/", "/live/")):
        return parsed.path.split("/")[2]
    return url  # already an ID


ytt_api = YouTubeTranscriptApi()
video_id = video_id_from_url("https://www.youtube.com/watch?v=VIDEO_ID")
transcript = ytt_api.fetch(video_id, languages=["en"])

print(transcript.language_code, transcript.is_generated, len(transcript))
for snippet in transcript.snippets[:3]:
    print(snippet.start, snippet.duration, snippet.text)

languages is a priority list: ["de", "en"] tries German first, then English. When a video has both a manual and an auto-generated transcript in the same language, the library picks the manual one. To see what exists, or to choose a type explicitly, use list():

transcript_list = ytt_api.list(video_id)
for t in transcript_list:
    print(t.language_code, t.language, t.is_generated, t.is_translatable)

generated = transcript_list.find_generated_transcript(["en"])
transcript = generated.fetch()

Turn it into JSON

The quickest route is to_raw_data(), which returns a list of dictionaries with text, start and duration keys:

import json

with open(f"{video_id}.raw.json", "w", encoding="utf-8") as f:
    json.dump(transcript.to_raw_data(), f, ensure_ascii=False, indent=2)

The built-in JSONFormatter does the same job and returns a string. It forwards keyword arguments to json.dumps, so indent and ensure_ascii work as usual:

from youtube_transcript_api.formatters import JSONFormatter

json_text = JSONFormatter().format_transcript(transcript, indent=2, ensure_ascii=False)

Both outputs contain only the snippet list. The video ID, language and is_generated flag are left out, so keep the raw file for debugging and build a richer document for anything downstream.

Clean it up for people and LLMs

Cleaning has three jobs: remove noise, normalize whitespace and merge fragments into readable segments without losing timestamps. The library already unescapes HTML entities and strips formatting tags unless you pass preserve_formatting=True, so the remaining work is mostly about structure.

import re

NOISE = re.compile(r"\[[^\]]*\]|♪")  # [Music], [Applause], music notes
SENTENCE_END = (".", "?", "!")


def clean_text(text):
    return " ".join(NOISE.sub(" ", text).split())


def build_segments(snippets, max_chars=400, max_gap=2.0):
    """Merge caption fragments into sentence-sized segments with start and end times."""
    segments, current = [], None
    for i, snip in enumerate(snippets):
        text = clean_text(snip["text"])
        if not text:
            continue
        end = snip["start"] + snip["duration"]
        if i + 1 < len(snippets):
            end = max(snip["start"], min(end, snippets[i + 1]["start"]))  # trim overlaps
        if current and (snip["start"] - current["end"] > max_gap
                        or len(current["text"]) >= max_chars):
            segments.append(current)
            current = None
        if current is None:
            current = {"start": snip["start"], "end": end, "text": text}
        else:
            current["end"] = end
            current["text"] += " " + text
        if current["text"].endswith(SENTENCE_END):
            segments.append(current)
            current = None
    if current:
        segments.append(current)
    for seg in segments:
        seg["start"], seg["end"] = round(seg["start"], 2), round(seg["end"], 2)
    return segments

Call it with build_segments(transcript.to_raw_data()). A segment closes when its text ends a sentence, when the speaker pauses for longer than max_gap seconds, or when it passes max_chars. The last two rules matter for auto-generated captions, which may have little or no punctuation. Trimming each snippet's end to the next snippet's start removes the overlap, and every segment keeps the start of its first fragment and the end of its last one, so you can still link to the exact moment in the video.

A stable JSON schema

Downstream code should never have to guess the shape of a file. A schema like this works well for search, summaries and RAG:

{
  "schema_version": 1,
  "video_id": "VIDEO_ID",
  "language": "en",
  "is_generated": true,
  "fetched_at": "2026-10-14T09:30:00+00:00",
  "segments": [
    { "start": 0.0, "end": 4.12, "text": "Welcome back to the channel." },
    { "start": 4.12, "end": 9.8, "text": "Today we will clean up some caption data." }
  ],
  "full_text": "Welcome back to the channel. Today we will clean up some caption data."
}

Building it from a fetched transcript takes one function:

from datetime import datetime, timezone


def to_document(transcript):
    segments = build_segments(transcript.to_raw_data())
    return {
        "schema_version": 1,
        "video_id": transcript.video_id,
        "language": transcript.language_code,
        "is_generated": transcript.is_generated,
        "fetched_at": datetime.now(timezone.utc).isoformat(timespec="seconds"),
        "segments": segments,
        "full_text": " ".join(s["text"] for s in segments),
    }

The segments list serves citations and chunking, full_text serves summaries, and is_generated lets you treat auto captions with more caution. schema_version lets you change the format later without breaking older consumers.

Processing many videos

For a batch, reuse one YouTubeTranscriptApi instance (it keeps a single HTTP session), cache every result on disk, skip videos that have no captions and retry only errors that can succeed on a second try:

import json
import random
import time
from pathlib import Path

import requests
from youtube_transcript_api import (
    NoTranscriptFound,
    RequestBlocked,
    TranscriptsDisabled,
    VideoUnavailable,
    YouTubeRequestFailed,
    YouTubeTranscriptApi,
)

ytt_api = YouTubeTranscriptApi()
OUT = Path("transcripts")
OUT.mkdir(exist_ok=True)


def process(video_id, retries=3):
    path = OUT / f"{video_id}.json"
    if path.exists():
        return "cached"
    for attempt in range(retries):
        try:
            transcript = ytt_api.fetch(video_id, languages=["en"])
            break
        except (TranscriptsDisabled, NoTranscriptFound, VideoUnavailable) as exc:
            return f"skipped: {type(exc).__name__}"
        except RequestBlocked:
            raise  # the IP is blocked: stop the batch instead of hammering
        except (YouTubeRequestFailed, requests.RequestException):
            if attempt == retries - 1:
                raise
            time.sleep(5 * 2 ** attempt + random.uniform(0, 2))
    doc = to_document(transcript)
    tmp = path.with_suffix(".tmp")
    tmp.write_text(json.dumps(doc, ensure_ascii=False, indent=2), encoding="utf-8")
    tmp.replace(path)
    return "ok"


video_ids = ["VIDEO_ID_1", "VIDEO_ID_2"]
for vid in video_ids:
    print(vid, process(vid))
    time.sleep(random.uniform(2, 5))

IpBlocked is a subclass of RequestBlocked, so one except clause covers both. Writing to a temporary file and renaming it means a crash never leaves a half-written file in the cache, and the cache means a rerun only fetches what is missing. Neither YouTube nor the library publishes a safe request rate for this endpoint, so run a single worker with a randomized pause and log every skip.

When requests get blocked on cloud servers

This is the most common surprise. The library's README says YouTube has started blocking most IPs known to belong to cloud providers such as AWS, Google Cloud Platform and Azure, so code that works on your laptop often raises RequestBlocked or IpBlocked once deployed. A home or office IP can be blocked too if it makes too many requests.

The README's workaround is proxies. Static proxies tend to get banned after extended use, so it recommends rotating residential proxies. The library ships a WebshareProxyConfig for Webshare (the README discloses that its Webshare links are referral links) and a GenericProxyConfig for any HTTP, HTTPS or SOCKS proxy:

import os

from youtube_transcript_api import YouTubeTranscriptApi
from youtube_transcript_api.proxies import GenericProxyConfig

ytt_api = YouTubeTranscriptApi(
    proxy_config=GenericProxyConfig(
        http_url=os.environ["PROXY_HTTP_URL"],
        https_url=os.environ["PROXY_HTTPS_URL"],
    )
)

Keep proxy credentials in environment variables, not in code. The README is also clear that a proxy is no guarantee, because YouTube can block the proxy's IP as well.

The library uses an undocumented part of the YouTube API that the web client calls, and its README warns that it could stop working if YouTube changes how things work. It is not an official API. Review YouTube's Terms of Service, respect creators' rights to their content and fetch only what you need.

Validating the output

A cleaning bug can silently produce empty or mangled files, so check every document before it moves on:

def validate(doc, raw_snippets):
    problems = []
    for key in ("video_id", "language", "is_generated", "segments", "full_text"):
        if key not in doc:
            problems.append(f"missing field: {key}")
    segments = doc.get("segments", [])
    if not segments:
        problems.append("no segments")
    prev_end = 0.0
    for i, seg in enumerate(segments):
        if not seg["text"].strip():
            problems.append(f"segment {i}: empty text")
        if seg["end"] < seg["start"] or seg["start"] < prev_end - 0.01:
            problems.append(f"segment {i}: bad timestamps")
        prev_end = seg["end"]
    raw_chars = sum(len(s["text"]) for s in raw_snippets)
    if raw_chars and len(doc.get("full_text", "")) < 0.5 * raw_chars:
        problems.append("cleaning removed more than half of the text")
    return problems

Call it inside process() right after to_document(), and quarantine any document that fails instead of writing it to the cache. The length check catches an over-eager noise pattern, which is the most likely way to lose real speech. For stricter typing, the same rules fit in a Pydantic model or a JSON Schema file.

Using the JSON for RAG or summaries

Once every transcript shares one schema, the next steps are simple:

• RAG and search. Group consecutive segments into chunks of a few hundred tokens, embed them and store video_id and start as metadata. An answer can then cite a link such as https://www.youtube.com/watch?v=VIDEO_ID&t=95s that opens at the right moment. The RAG architecture blueprint covers chunking, retrieval and cited answers in more detail.
• Summaries. Send full_text to a model, or summarize long videos in chunks and then combine the partial summaries. Before you run hundreds of videos, estimate the token bill with the LLM API cost calculator.

If you want this as a scheduled workflow with a model step at the end, that is the kind of work covered under AI workflow automation.

Wrapping up

The core is small: fetch with the current instance API, clean and merge the fragments, save one stable schema, validate it and plan for IP blocks before you deploy. For a real example of the full flow, the YouTube transcript ETL case study describes a pipeline that extracts captions automatically, merges caption fragments into readable text with NLP techniques and turns them into a consistent JSON schema, from video link to final file, which removed manual transcription completely.