Annotations

Add time-synchronized text annotations to your audio timeline with drag-to-edit functionality. The annotations system provides two approaches: an integrated pattern (recommended) where annotations are managed by the provider, and composable building blocks for custom UIs.

Installation

Install the annotations package:

npm install @waveform-playlist/annotations

Quick Start

The simplest way to add annotations is through the provider's annotationList prop. Wrap your playlist with AnnotationProvider to enable annotation rendering — the <Waveform /> component then renders annotation boxes and text automatically.

import { useState } from 'react';
import {
  WaveformPlaylistProvider,
  Waveform,
  useAudioTracks,
} from '@waveform-playlist/browser';
import { AnnotationProvider } from '@waveform-playlist/annotations';

function AnnotatedPlaylist() {
  const { tracks, loading } = useAudioTracks([
    { src: '/audio/podcast.mp3', name: 'Podcast' },
  ]);

  const [annotations, setAnnotations] = useState([
    { id: '1', start: 0, end: 5, lines: ['Introduction'] },
    { id: '2', start: 5, end: 15, lines: ['Topic Overview'] },
    { id: '3', start: 15, end: 30, lines: ['Main Discussion'] },
  ]);

  if (loading) return <div>Loading...</div>;

  return (
    <WaveformPlaylistProvider
      tracks={tracks}
      timescale
      annotationList={{
        annotations,
        editable: true,
        linkEndpoints: false,
      }}
      onAnnotationsChange={setAnnotations}
    >
      <AnnotationProvider>
        <Waveform />
      </AnnotationProvider>
    </WaveformPlaylistProvider>
  );
}

With this pattern, <Waveform /> internally renders AnnotationBoxesWrapper, AnnotationBox components, and AnnotationText — no manual composition needed. Without AnnotationProvider, annotation data is stored but not rendered.

Annotation Structure

Each annotation has the following properties:

interface AnnotationData {
  id: string;         // Unique identifier
  start: number;      // Start time in seconds
  end: number;        // End time in seconds
  lines: string[];    // Text content as array of lines
  language?: string;  // Optional language code (e.g., 'en', 'es')
}

Provider Configuration

annotationList Prop

Pass annotation data and editing options to the provider:

interface AnnotationList {
  annotations?: AnnotationData[];   // Array of annotations
  editable?: boolean;               // Enable drag editing
  isContinuousPlay?: boolean;       // Auto-play next annotation
  linkEndpoints?: boolean;          // Link adjacent boundaries (default: false)
  controls?: AnnotationAction[];    // Custom action buttons per annotation
}

onAnnotationsChange Callback

Called whenever annotations are modified (drag, text edit, boundary change):

<WaveformPlaylistProvider
  tracks={tracks}
  annotationList={{ annotations, editable: true }}
  onAnnotationsChange={(updatedAnnotations) => {
    setAnnotations(updatedAnnotations);
  }}
>
  <Waveform />
</WaveformPlaylistProvider>

:::note Naming Convention The provider uses onAnnotationsChange (browser package). The AnnotationText component uses onAnnotationUpdate (annotations package). Both receive the full updated annotations array. :::

Waveform Annotation Props

When using the integrated pattern, <Waveform /> accepts these annotation-related props:

Prop	Type	Description
annotationControls	AnnotationAction[]	Custom action buttons for each annotation item
annotationTextHeight	number	Height in pixels for the annotation text list
renderAnnotationItem	(props: RenderAnnotationItemProps) => ReactNode	Custom render function for annotation list items
getAnnotationBoxLabel	(annotation, index) => string	Custom label for annotation boxes on the timeline

<Waveform
  annotationTextHeight={200}
  getAnnotationBoxLabel={(annotation, index) => `${index + 1}. ${annotation.lines[0]}`}
  renderAnnotationItem={({ annotation, index, isActive, onClick, formatTime }) => (
    <div onClick={onClick} style={{ background: isActive ? '#ffe0b2' : 'transparent' }}>
      <strong>{formatTime(annotation.start)} - {formatTime(annotation.end)}</strong>
      <p>{annotation.lines.join(' ')}</p>
    </div>
  )}
/>

Browser Hooks

The browser package provides hooks for annotation interactions. These are used internally by <Waveform /> but can also be used directly for custom UIs.

useAnnotationDragHandlers

Provides @dnd-kit drag handlers for annotation boundary editing.

import { useAnnotationDragHandlers } from '@waveform-playlist/browser';

const { annotations, linkEndpoints } = usePlaylistState();
const { setAnnotations, annotationPreview } = usePlaylistControls();
const { samplesPerPixel, sampleRate, duration } = usePlaylistData();

const { onDragStart, onDragMove, onDragEnd } = useAnnotationDragHandlers({
  annotations,                          // AnnotationData[]
  onAnnotationsChange: setAnnotations,  // called once, on release, when `preview` is passed
  preview: annotationPreview,           // optional: shows the drag without changing your state per move
  samplesPerPixel,                      // number
  sampleRate,                           // number (default 48000; pass the provider's)
  duration,                             // number (total duration in seconds)
  linkEndpoints,                        // boolean
});

Without preview, onAnnotationsChange is called on every pointer move. Escape cancels the drag and restores every annotation it moved. If you draw annotations yourself, read them through useAnnotationsWithPreview(annotations, annotationPreview) so your boxes follow the drag.

Use with @dnd-kit/react's DragDropProvider:

import { DragDropProvider } from '@dnd-kit/react';

<DragDropProvider onDragStart={onDragStart} onDragMove={onDragMove} onDragEnd={onDragEnd}>
  {/* annotation boxes */}
</DragDropProvider>

useAnnotationKeyboardControls

Keyboard navigation and editing for annotations.

import { useAnnotationKeyboardControls } from '@waveform-playlist/browser';

const {
  moveStartBoundary,    // (delta: number) => void
  moveEndBoundary,      // (delta: number) => void
  selectPrevious,       // () => void
  selectNext,           // () => void
  selectFirst,          // () => void
  selectLast,           // () => void
  clearSelection,       // () => void
  scrollToAnnotation,   // (annotationId: string) => void
  playActiveAnnotation, // () => void
} = useAnnotationKeyboardControls({
  annotations,                // AnnotationData[]
  activeAnnotationId,         // string | null
  onAnnotationsChange,        // (annotations) => void
  onActiveAnnotationChange,   // (id: string | null) => void
  duration,                   // number
  linkEndpoints,              // boolean
  continuousPlay,             // boolean (optional)
  enabled,                    // boolean (optional)
  scrollContainerRef,         // RefObject<HTMLDivElement> (optional)
  onPlay,                     // (startTime, duration?) => void (optional)
});

Default Keyboard Shortcuts:

Key	Action
← / →	Move start boundary
Shift+← / Shift+→	Move end boundary
↑ / ↓	Select previous / next annotation
Home / End	Select first / last annotation
Escape	Clear selection
Enter	Play active annotation

Annotation Package Components

The @waveform-playlist/annotations package exports composable building blocks for custom annotation UIs.

AnnotationsTrack

A layout wrapper for annotation content. Use this when building a custom annotation UI outside the integrated <Waveform /> pattern.

interface AnnotationsTrackProps {
  className?: string;
  children?: React.ReactNode;
  height?: number;
  offset?: number;
  width?: number;
}

import { AnnotationsTrack } from '@waveform-playlist/annotations';

<AnnotationsTrack height={30}>
  {/* Custom annotation content */}
</AnnotationsTrack>

tip

AnnotationsTrack is a simple layout container — it does not accept annotations, onAnnotationsChange, or editable props. For the common case, use the integrated pattern (pass annotationList to the provider and use <Waveform />).

AnnotationText

A scrollable list view of annotations with automatic scrolling to the active annotation during playback.

import { AnnotationText } from '@waveform-playlist/annotations';

<AnnotationText
  annotations={annotations}
  activeAnnotationId={currentAnnotation?.id}
  shouldScrollToActive={isPlaying}
  editable={true}
  height={200}
  onAnnotationClick={(annotation) => seekTo(annotation.start)}
  onAnnotationUpdate={setAnnotations}
/>

Custom Annotation Rendering

Use renderAnnotationItem for complete control over how each annotation appears in the list:

<AnnotationText
  annotations={annotations}
  activeAnnotationId={activeId}
  renderAnnotationItem={({ annotation, index, isActive, onClick, formatTime }) => (
    <div
      onClick={onClick}
      style={{
        padding: '12px',
        background: isActive ? '#ffe0b2' : 'transparent',
        borderLeft: isActive ? '4px solid #ff9800' : '4px solid transparent',
        cursor: 'pointer',
      }}
    >
      <div style={{ fontWeight: 'bold' }}>{annotation.id}</div>
      <div style={{ fontSize: '12px', color: '#666' }}>
        {formatTime(annotation.start)} - {formatTime(annotation.end)}
      </div>
      <div>{annotation.lines.join('\n')}</div>
    </div>
  )}
/>

AnnotationBox

Individual draggable annotation box for the timeline. Supports boundary resizing with @dnd-kit.

import { AnnotationBox } from '@waveform-playlist/annotations';

<AnnotationBox
  annotationId={annotation.id}
  annotationIndex={index}
  startPosition={startPixels}
  endPosition={endPixels}
  label={annotation.id}
  color="#ff9800"
  isActive={annotation.id === activeId}
  editable={true}
  onClick={() => selectAnnotation(annotation)}
/>

AnnotationBoxesWrapper

Container that aligns annotation boxes with the waveform, accounting for track controls width.

import { AnnotationBoxesWrapper, AnnotationBox } from '@waveform-playlist/annotations';

<AnnotationBoxesWrapper height={30}>
  {annotations.map((annotation, index) => (
    <AnnotationBox
      key={annotation.id}
      annotationId={annotation.id}
      annotationIndex={index}
      startPosition={annotation.start * pixelsPerSecond}
      endPosition={annotation.end * pixelsPerSecond}
      label={annotation.id}
    />
  ))}
</AnnotationBoxesWrapper>

useAnnotationControls Hook

Manages annotation editing state including continuous play mode and linked endpoints.

import { useAnnotationControls } from '@waveform-playlist/annotations';

function AnnotationEditor() {
  const {
    continuousPlay,
    linkEndpoints,
    setContinuousPlay,
    setLinkEndpoints,
    updateAnnotationBoundaries,
  } = useAnnotationControls({
    initialContinuousPlay: false,
    initialLinkEndpoints: false,
  });

  // Handle drag updates
  const handleDragEnd = (annotationIndex: number, newTime: number, isDraggingStart: boolean) => {
    const updated = updateAnnotationBoundaries({
      annotationIndex,
      newTime,
      isDraggingStart,
      annotations,
      duration,
      linkEndpoints,
    });
    setAnnotations(updated);
  };

  return (
    <div>
      <label>
        <input
          type="checkbox"
          checked={continuousPlay}
          onChange={(e) => setContinuousPlay(e.target.checked)}
        />
        Continuous Play
      </label>
      <label>
        <input
          type="checkbox"
          checked={linkEndpoints}
          onChange={(e) => setLinkEndpoints(e.target.checked)}
        />
        Link Endpoints
      </label>
    </div>
  );
}

Linked Endpoints Behavior

When linkEndpoints is enabled:

• Dragging the end of annotation A moves the start of annotation B if they're adjacent
• Annotations "snap" together when boundaries meet
• Useful for transcription where segments should be contiguous

When disabled:

• Annotations can overlap or have gaps
• Boundary collisions push adjacent annotations

Control Components

Pre-built checkbox and button components for common annotation controls:

import {
  ContinuousPlayCheckbox,
  LinkEndpointsCheckbox,
  EditableCheckbox,
  DownloadAnnotationsButton,
} from '@waveform-playlist/annotations';

function AnnotationControls({ annotations }) {
  const [continuousPlay, setContinuousPlay] = useState(false);
  const [linkEndpoints, setLinkEndpoints] = useState(false);
  const [editable, setEditable] = useState(true);

  return (
    <div style={{ display: 'flex', gap: '1rem' }}>
      <ContinuousPlayCheckbox
        checked={continuousPlay}
        onChange={setContinuousPlay}
      />
      <LinkEndpointsCheckbox
        checked={linkEndpoints}
        onChange={setLinkEndpoints}
      />
      <EditableCheckbox
        checked={editable}
        onChange={setEditable}
      />
      <DownloadAnnotationsButton
        annotations={annotations}
        filename="my-annotations.json"
      />
    </div>
  );
}

Styling Annotations

Customize annotation appearance via theme:

const theme = {
  // Annotation boxes on timeline
  annotationBoxBackground: 'rgba(255, 255, 255, 0.85)',
  annotationBoxActiveBackground: 'rgba(255, 200, 100, 0.95)',
  annotationBoxHoverBackground: 'rgba(255, 255, 255, 0.98)',
  annotationBoxActiveBorder: '#ff9800',
  annotationLabelColor: '#2a2a2a',

  // Resize handles
  annotationResizeHandleColor: 'rgba(0, 0, 0, 0.4)',
  annotationResizeHandleActiveColor: 'rgba(0, 0, 0, 0.8)',

  // Text list items
  annotationTextItemHoverBackground: 'rgba(0, 0, 0, 0.05)',
};

<WaveformPlaylistProvider tracks={tracks} theme={theme}>
  <Waveform />
</WaveformPlaylistProvider>

Import/Export

Aeneas Format

Parse and serialize the Aeneas synchronization format:

import { parseAeneas, serializeAeneas } from '@waveform-playlist/annotations';

// Parse Aeneas JSON
const aeneasData = { fragments: [...] };
const annotations = parseAeneas(aeneasData);

// Serialize back to Aeneas format
const exported = serializeAeneas(annotations);

Download Button

Use the built-in download button:

<DownloadAnnotationsButton
  annotations={annotations}
  filename="annotations.json"
/>

Custom Export

function ExportAnnotations({ annotations }) {
  const handleExport = () => {
    const json = JSON.stringify(annotations, null, 2);
    const blob = new Blob([json], { type: 'application/json' });
    const url = URL.createObjectURL(blob);

    const a = document.createElement('a');
    a.href = url;
    a.download = 'annotations.json';
    a.click();

    URL.revokeObjectURL(url);
  };

  return <button onClick={handleExport}>Export Annotations</button>;
}

Import from JSON

function ImportAnnotations({ onImport }) {
  const handleImport = (event: React.ChangeEvent<HTMLInputElement>) => {
    const file = event.target.files?.[0];
    if (!file) return;

    const reader = new FileReader();
    reader.onload = (e) => {
      const json = e.target?.result as string;
      const imported = JSON.parse(json);
      onImport(imported);
    };
    reader.readAsText(file);
  };

  return <input type="file" accept=".json" onChange={handleImport} />;
}

Complete Example

import { useState } from 'react';
import {
  WaveformPlaylistProvider,
  Waveform,
  PlayButton,
  PauseButton,
  StopButton,
  useAudioTracks,
} from '@waveform-playlist/browser';
import {
  useAnnotationControls,
  ContinuousPlayCheckbox,
  LinkEndpointsCheckbox,
  EditableCheckbox,
  DownloadAnnotationsButton,
} from '@waveform-playlist/annotations';

function AnnotationsExample() {
  const { tracks, loading, error } = useAudioTracks([
    { src: '/audio/podcast.mp3', name: 'Podcast Episode' },
  ], { immediate: true });

  const [annotations, setAnnotations] = useState([
    { id: '1', start: 0, end: 10, lines: ['Introduction'] },
    { id: '2', start: 10, end: 30, lines: ['Main Topic'] },
    { id: '3', start: 30, end: 45, lines: ['Conclusion'] },
  ]);

  const [editable, setEditable] = useState(true);

  const {
    continuousPlay,
    linkEndpoints,
    setContinuousPlay,
    setLinkEndpoints,
  } = useAnnotationControls();

  if (loading) return <div>Loading...</div>;
  if (error) return <div>Error: {error}</div>;

  return (
    <WaveformPlaylistProvider
      tracks={tracks}
      samplesPerPixel={1024}
      waveHeight={100}
      timescale
      annotationList={{
        annotations,
        editable,
        isContinuousPlay: continuousPlay,
        linkEndpoints,
      }}
      onAnnotationsChange={setAnnotations}
    >
      {/* Playback controls */}
      <div style={{ display: 'flex', gap: '0.5rem', marginBottom: '1rem' }}>
        <PlayButton />
        <PauseButton />
        <StopButton />
      </div>

      {/* Annotation controls */}
      <div style={{ display: 'flex', gap: '1rem', marginBottom: '1rem' }}>
        <ContinuousPlayCheckbox checked={continuousPlay} onChange={setContinuousPlay} />
        <LinkEndpointsCheckbox checked={linkEndpoints} onChange={setLinkEndpoints} />
        <EditableCheckbox checked={editable} onChange={setEditable} />
        <DownloadAnnotationsButton annotations={annotations} />
      </div>

      {/* Waveform with integrated annotation boxes and text */}
      <Waveform annotationTextHeight={200} />
    </WaveformPlaylistProvider>
  );
}

export default AnnotationsExample;

Use Cases

Podcast Chapters

Mark sections of a podcast for easy navigation:

const chapters = [
  { id: '1', start: 0, end: 120, lines: ['Intro & Sponsors'] },
  { id: '2', start: 120, end: 600, lines: ['Guest Interview'] },
  { id: '3', start: 600, end: 900, lines: ['Q&A Session'] },
  { id: '4', start: 900, end: 960, lines: ['Outro'] },
];

Transcription Segments

Break down audio into transcribed segments:

const transcription = [
  { id: '1', start: 0, end: 3.5, lines: ['Welcome to the show.'] },
  { id: '2', start: 3.5, end: 7, lines: ["Today we're discussing..."] },
  // ...
];

Music Markers

Mark sections in music:

const musicSections = [
  { id: 'v1', start: 0, end: 16, lines: ['Verse 1'] },
  { id: 'c1', start: 16, end: 32, lines: ['Chorus'] },
  { id: 'v2', start: 32, end: 48, lines: ['Verse 2'] },
  { id: 'c2', start: 48, end: 64, lines: ['Chorus'] },
  { id: 'br', start: 64, end: 80, lines: ['Bridge'] },
  { id: 'c3', start: 80, end: 96, lines: ['Final Chorus'] },
];

Multi-line Annotations

Annotations support multiple lines of text:

const detailedAnnotations = [
  {
    id: '1',
    start: 0,
    end: 30,
    lines: [
      'Speaker: John Smith',
      'Topic: Introduction to the project',
      'Key points: overview, timeline, goals',
    ],
  },
];

Live Example

See the Annotations Example for a full working demo.

Next Steps

• Recording - Record audio with annotations
• Theming - Customize annotation appearance