Mobile Accessibility Bible
DAC Accessibility Specialist
iOSmedia-motionAdvanced

Making Multimedia Content Accessible

This guide explains how to make multimedia—such as videos and product demos—accessible to all users. It covers best practices for adding captions, providing audio descriptions, and ensuring user-controlled playback. Using real SwiftUI and UIKit examples, it shows how to integrate caption tracks with AVPlayerItem, announce playback state changes for VoiceOver, and label controls with accessibility traits and hints.

🎯 Why It Matters

Accessible multimedia ensures that everyone can enjoy and understand your app’s content—regardless of hearing or visual ability. Videos and audio clips often convey critical information or emotions that must be accessible to all users. Providing captions benefits Deaf or hard-of-hearing users, while audio descriptions and accessible playback controls support users who are blind or have low vision.
By integrating captions, ensuring user-initiated playback, and clearly labelling controls, developers can create inclusive multimedia experiences that align with WCAG and Apple’s accessibility guidelines.

đź’» Implementation

Correct Code Example: Adding Captions to an AVPlayerItem

Demo Description

This snippet demonstrates selecting a caption track from an AVAsset, ensuring the captions display alongside the video

Implementation

// SwiftUI  
// ACCESSIBLE: Adds captions dynamically for users who rely on visual text alternatives.  

// MARK: - Caption Integration for Accessibility  
/// Sets up the video player with programmatically added captions  
/// This approach allows for dynamic caption loading and better accessibility control  
private func setupPlayerWithSubtitles() async {
    // Load the main video file from the app bundle
    guard let videoURL = Bundle.main.url(forResource: "guideDogInterview", withExtension: "mp4") else {
        print("Error: Could not find guideDogInterview.mp4 in the bundle.")
        return
    }

    // Load the VTT caption file from the app bundle
    // VTT (Web Video Text Tracks) is the web standard for video captions
    // It provides better accessibility support than SRT format
    guard let vttURL = Bundle.main.url(forResource: "guideDogInterview", withExtension: "vtt") else {
        print("Error: Could not find guideDogInterview.vtt in the bundle.")
        // Fallback: play video without captions if VTT file is missing
        await MainActor.run {
            player.replaceCurrentItem(with: AVPlayerItem(url: videoURL))
        }
        return
    }

    // Create AVURLAssets for both video and caption files
    let videoAsset = AVURLAsset(url: videoURL)
    let subtitleAsset = AVURLAsset(url: vttURL)

    // Retrieve legible media selection group for captions
    let legibleGroup = try? await videoAsset.loadMediaSelectionGroup(for: .legible)
    let playerItem = AVPlayerItem(asset: videoAsset)

    // Select first available caption track, if any
    if let group = legibleGroup, let option = group.options.first {
        playerItem.select(option, in: group)
    }

    await MainActor.run {
        player.replaceCurrentItem(with: playerItem)
    }
}

Code Explanation

In this snippet, the AVAsset specifies a .legible media selection group, which contains available caption or subtitle tracks. The code selects the first available caption track, enabling it so that captions are shown during playback when supported. Additionally, the VideoPlayerView includes both an accessibilityLabel and an accessibilityHint, helping VoiceOver users identify the video content and understand how to interact with it.

Correct Code Example: Disabling Auto-Play and Accessible Playback Controls

Demo Description

Here, we create a SwiftUI view that disables auto-play and relies on the built-in playback controls, allowing the user to manually initiate playback and ensuring an accessible experience

Implementation

// UIKit  
// ACCESSIBLE: User controls playback manually; VoiceOver provides play/pause context.  

import SwiftUI
import AVKit

// MARK: - Accessible Video Player
@objc func buttonTapped() {
    parent.isPlaying.toggle()
    parent.isPlaying ? parent.player.play() : parent.player.pause()
    
    let (icon, announcement, label, hint) = parent.isPlaying
        ? ("pause.fill", "Video is now playing", "Pause video, button", "Double tap to pause the video")
        : ("play.fill", "Video is now paused", "Play video, button", "Double tap to play the video")
    
    playPauseButton?.setImage(UIImage(systemName: icon), for: .normal)
    UIAccessibility.post(notification: .announcement, argument: announcement)
    
    DispatchQueue.main.asyncAfter(deadline: .now() + 0.5) {
        self.playPauseButton?.accessibilityLabel = label
        self.playPauseButton?.accessibilityHint = hint
        UIAccessibility.post(notification: .layoutChanged, argument: self.playPauseButton)
    }
}

Code Explanation

In this snippet, the video player is configured to start paused, so users decide when playback begins. The built-in controls remain fully accessible. The accessibilityLabel and accessibilityHint inform VoiceOver users how to interact with the video. This approach respects user control and supports people with hearing, visual, or cognitive differences.

âś… Best Practices

Do's

  • âś“Always include captions or transcripts for video and audio content
  • âś“Allow users to start playback manually
  • âś“Provide clear accessibilityLabel and accessibilityHint for media controls
  • âś“Use standard system controls whenever possible for built in accessibility

Don'ts

  • âś—Auto-play videos without user interaction
  • âś—Include visual only media without text or audio equivalents
  • âś—Forget to announce state changes to assistive tech
  • âś—Use custom media players that ignore accessibility APIs

🔍 Common Pitfalls

Auto-playing Media

Disorients users relying on screen readers

Missing Captions

Excludes Deaf or hard-of-hearing users

Poorly Labelled Controls

Leaves VoiceOver users unsure what button does what

Overriding System Players

May break built-in accessibility features

VoiceOver

Announces play/pause states and reads captions when available

Switch Control

Allows control over playback buttons and caption toggles

Dynamic Type

Ensures video labels and control text scale with system settings

🧪 Testing Steps

  1. 1Enable VoiceOver: Settings → Accessibility → VoiceOver
  2. 2Play Video: Ensure VoiceOver correctly announces play/pause buttons
  3. 3Check Captions: Turn on subtitles from the system menu and verify they appear
  4. 4Pause/Resume Playback: Confirm VoiceOver announces each state change
  5. 5Screen Rotation: Ensure layout and reading order remain logical