Starting in SDK 17.1, some of AJA’s devices with HDMI inputs support the capture of HDMI auxiliary data using AJA’s inserter/extractor firmware technology.

Note: Although HDMI Aux playback is documented here, the feature is not currently implemented. AJA plans to complete this feature in a future firmware and SDK release.

CONTENTS:

Custom Aux Packet Capture

The only way to capture any/all Auxiliary data packets is by way of the Aux Extractor firmware.

Check that CNTV2Card::features ⟹ DeviceCapabilities::CanDoHDMIAuxCapture returns true if this feature is supported.
When properly configured and enabled, HDMI Aux data packets are automatically extracted from the HDMI input video stream and placed into a separate area of the device frame buffer.
With AutoCirculate :
- SDK & driver version 17.1 or later required.
- Call CNTV2Card::AutoCirculateInitForInput with the AUTOCIRCULATE_WITH_HDMIAUX option.
- Call AUTOCIRCULATE_TRANSFER::SetAncBuffers to specify the destination host Aux buffer(s).
- After CNTV2Card::AutoCirculateTransfer returns, parse the contents of the host Aux buffer(s).
  - AUTOCIRCULATE_TRANSFER::GetCapturedAuxByteCount reports the number of bytes recorded/written into the device frame’s anc buffer region.
  - Manually parse the anc data that was recorded and transferred — see HDMI Aux Buffer Data Format.
  - Or use the Ancillary Data Classes — call AJAAuxiliaryList::SetFromDeviceAncBuffers to retrieve the ::AJAAuxiliary packets.
Without AutoCirculate :
- Call CNTV2Card::AuxExtractGetField1Size and CNTV2Card::AuxExtractGetField2Size to note how many HDMI Aux data bytes were recorded/written into the device frame buffer’s Aux region for Field1 and Field2, respectively.
- Call CNTV2Card::DMAReadAnc to transfer the captured HDMI Aux data to the host.
- Parse the contents of the host Aux buffer(s):
  - Manually parse the aux data that was recorded/transferred — see HDMI Aux Buffer Data Format (below).
  - Or use the Ancillary Data Classes — call AJAAuxiliaryList::SetFromDeviceAuxBuffers to retrieve the AJAAuxiliaryData packets.
Each Aux Extractor is capable of filtering certain types of auxiliary data packets:
- Upon driver startup, all Aux Extractors are configured to exclude nothing (include everything).
- Call CNTV2Card::AuxExtractGetPacketFilters to discover which packet DIDs are currently being excluded.
- Call CNTV2Card::AuxExtractSetPacketFilters to change which packet DIDs will be excluded.
- Use the NTV2DIDSet returned from CNTV2Card::AuxExtractGetDefaultPacketFilters to restore the default packet exclusion list.
The default maximum per-field auxiliary data buffer capacity is 8K by default, or 16K per-frame, including overhead.

TIP: The “NTV2Watcher” tool’s Ancillary Data Inspector is useful for diagnosing issues with auxiliary data capture.

Auxiliary Packet Filtering

For devices that support HDMI Aux capture, each HDMI input has an “Aux Extractor” widget associated with it. Each of these widgets have a filter that can exclude unwanted packets based on their Packet Type. Each filter is configured using five 32-bit registers, each register accommodating four Packet Type values, resulting in the ability to filter (exclude) up to 20 different types of packets.

A non-zero byte value in a filter register will cause the Extractor to skip packets whose Packet Type matches that byte value.
A zero byte value is always ignored by the Extractor.
It is not an error to have the same Packet Type value in multiple byte positions in the filter registers.

By default, the HDMI input’s Aux extractor widget is configured when CNTV2Card::AutoCirculateInitForInput (or CNTV2Card::AncExtractInit) are called. By default, the extractor is configured to NOT filter any packets.

Disabling All Filtering

To disable all filtering of incoming auxiliary data packets, pass an empty NTV2DIDSet to CNTV2Card::AuxExtractSetPacketFilters.

CNTV2Card device;
. . .
const UWord hdmiInput(UWord(::GetIndexForNTV2InputSource(NTV2_INPUTSOURCE_HDMI1)));
device.AuxExtractSetPacketFilters(hdmiInput, NTV2DIDSet());   // Disable all filtering

Note: Changing packet filtering takes effect at the next captured frame.

Filtering Specific Packet Types

To exclude packets having specific HDMI Packet Types, build an NTV2DIDSet that contains the Packet Types you wish to exclude, then pass the set to CNTV2Card::AuxExtractSetPacketFilters.

Restoring Default Filtering

You can also reset the ANC filtering to the default filter set:

CNTV2Card device;
const UWord hdmiInput(UWord(::GetIndexForNTV2InputSource(NTV2_INPUTSOURCE_HDMI1)));
. . .
NTV2VideoFormat vfmt(device.GetInputVideoFormat(NTV2_INPUTSOURCE_HDMI1));
device.AuxExtractSetPacketFilters(hdmiInput, AuxExtractGetDefaultDIDs());

It’s a good practice to save the existing filter settings, change them to what’s needed for whatever processing is necessary, and then restore them afterward:

CNTV2Card device;
. . .
const UWord hdmiInput(UWord(::GetIndexForNTV2InputSource(NTV2_INPUTSOURCE_HDMI1)));
NTV2DIDSet savedDIDs;
device.AuxExtractGetPacketFilters(hdmiInput, savedDIDs);   // Save current anc filter (so it can be restored later)
{
    // Change the anc filter to include some audio packets...
    NTV2DIDSet dids;
    dids.insert(0x01);   // Audio Clock Regeneration
    dids.insert(0x02);   // Audio Sample (L-PCM & IEC-61937 compressed)
    dids.insert(0x84);   // Audio InfoFrame
    device.AuxExtractSetPacketFilters(hdmiInput, dids);   // Set new anc filtering
 
    //  . . . do other processing . . .
}
device.AuxExtractSetPacketFilters(hdmiInput, savedDIDs);   // Restore prior anc filtering

Auxiliary Data Space Limitations

Rarely does the auxiliary data transmitted with a field of video exceed 5KB (or 10KB for a frame of interlaced video) … but it’s possible. If your application must accommodate more than 72KB of auxiliary data, you’ll need to enlarge the driver’s auxiliary data region.

The NTV2 driver uses two virtual registers (kVRegAncField1Offset and kVRegAncField2Offset) to control where the Aux Extractors start writing extracted packet data into (current) frame memory. When the driver starts up, it’s configured to use 0x12000 bytes (~72k) of space per field at the very bottom of each 8MB/16MB/… frame on the device (depending on video format and pixel format).

If you know the largest byte count you’ll encounter for each field, add some padding, then call CNTV2Card::AuxSetFrameBufferSize with those values.

Warning: Itʼs possible to reserve a very large auxiliary data space that actually runs into the video in the frame buffer.

To programmatically check if the Aux space intersects video in the frame buffer:

CNTV2Card device;
. . .
ULWord byteOffset(0), byteCount(0);
NTV2PixelFormat pixelFormat;
const NTV2VideoFormat vidFormat(device.GetInputVideoFormat(NTV2_INPUTSOURCE_HDMI1));
device.GetFrameBufferFormat(NTV2_CHANNEL1, pixelFormat);
const NTV2FormatDescriptor fd(vidFormat, pixelFormat);
device.GetAncRegionOffsetAndSize(byteOffset, byteCount, NTV2_AncRgn_All);
if (fd.GetTotalBytes() > byteOffset)
    FAIL("HDMI Aux in video!");

Maximizing Auxiliary Data Capture Capacity

Determine the largest raster your application is expected to handle.
Determine the most “expensive” (in terms of memory) NTV2PixelFormat your application will need to use.
Determine the “intrinsic” device frame buffer size that will be used for the video format and frame buffer format (8MB/16MB/etc.).
Calculate the maximum available space for auxiliary data in the device frame buffer. This is the distance, in bytes, from just past the last line of the raster to the bottom of the device frame buffer.
Call CNTV2Card::AuxSetFrameBufferSize, passing half the byte distance to each of the two parameters.

For example, 1080i video with a 10-bit YCbCr frame buffer format is easily handled by an 8MB device frame buffer. The 1920x1080 raster occupies the top 5,529,600 bytes of the device frame buffer, which leaves 2,859,008 bytes of space for auxiliary data. If a 16MB frame buffer is used (see CNTV2Card::SetFrameBufferSize), the maximum possible anc space jumps to 11,247,616 bytes — a LOT of space!

The Frame Buffer Inspector of “NTV2Watcher” readily illustrates how this works.

Set it to the Raw view mode.
Change the units used for the vertical ruler to Bytes From Top.
Change the units used for the horizontal ruler to Bytes From Left Edge.
Scroll down to the end of active video (EAV). This is where a noticeable boundary between illuminated and dimmed values indicate where changing pixel data ends. That offset – 0x00546000 – is the first safe (unchanging) byte of available HDMI auxiliary buffer space.
Scroll down further until the data is displayed in a yellow color. That is the HDMI auxiliary data region.

HDMI Auxiliary Data Playout

To output HDMI Auxiliary data packets, use HDMI Aux Inserter firmware:

Check CNTV2Card::features ⟹ DeviceCapabilities::CanDoHDMIAuxPlayback returns true to see if HDMI Aux playback is supported.
Auxiliary data packets are automatically inserted into the outgoing video stream when they’ve been properly formatted and placed into a separate area of the device frame buffer.
With AutoCirculate :
- Call CNTV2Card::AutoCirculateInitForOutput with the AUTOCIRCULATE_WITH_HDMIAUX option.
- Call AUTOCIRCULATE_TRANSFER::SetAncBuffers to specify the source host Aux buffer(s).
- Set the contents of the host Aux buffer(s)…
  - Manually write the data — see HDMI Aux Buffer Data Format (below).
  - Or use the Ancillary Data Classes — call AJAAuxiliaryList::GetTransmitData.
- Call CNTV2Card::AutoCirculateTransfer.
Without AutoCirculate :
- Configure the HDMI output’s Aux Inserter (see CNTV2Card::AuxInsertInit, CNTV2Card::AuxInsertSetEnable, CNTV2Card::AuxInsertSetReadParams, etc.)
- Set the contents of the host Aux buffer(s)…
  - Manually write the data — see HDMI Aux Buffer Data Format (below).
  - Or use the Ancillary Data Classes to put together an AJAAuxiliaryList of packets to send — then call AJAAuxiliaryList::GetHDMITransmitData.
  - Call CNTV2Card::DMAWriteAnc.
Default per-field auxiliary data buffer capacity is 8K by default, or 16K per-frame, including overhead.

TIP: The “NTV2Watcher” tool’s Ancillary Data Inspector is useful for diagnosing issues with auxiliary data playout.

HDMI Aux Buffer Data Format

The data bytes in the auxiliary data buffer consist of zero or more packets. Each packet should have a fixed size of 32 bytes and match this format:

The packet bytes are:

Byte 0: HDMI Packet Type:
- InfoFrame Packet:
  - 0x80: – General InfoFrame Packet (might be invalid)
  - 0x81: – Vendor-Specific InfoFrame
  - 0x82: – AVI InfoFrame
  - 0x83: – Source Product Descriptor InfoFrame
  - 0x84: – Audio InfoFrame
  - 0x85: – MPEG Source InfoFrame
- Other Packet:
  - 0x01: – Audio Clock Regeneration (N/CTS)
  - 0x02: – Audio Sample (L-PCM & IEC-61937 compressed)
  - 0x03: – General Control
  - 0x04: – ACP Packet
  - 0x05: – ISRC1 Packet
  - 0x06: – ISRC2 Packet
  - 0x07: – One Bit Audio Sample Packet
  - 0x08: – DST Audio Packet
  - 0x09: – High Bitrate Audio Stream Packet (IEC-61937)
  - 0x0A: – Gamut Metadata Packet
Byte 1: Header Byte 1 (HB1)
Byte 2: Header Byte 2 (HB2)
- For InfoFrames, the least-significant 5 bits of this byte contains the actual packet payload size, in bytes.
Byte 3:
- For InfoFrames, this byte stores the packet checksum;
- For all other packets, this is the first payload data byte.
Byte 4:
- For InfoFrames, this byte stores the first payload data byte.
- For all other packets, this is the 2nd payload data byte.
Bytes 5 … 31: Remaining payload data bytes

Note: During playback/output, before transferring the F1 (or F2 for interlace) aux buffer from the host, it’s a good practice to truncate (or zero) the buffer at (or past) the last packet written into the buffer.; During capture/input, after transferring the F1 (or F2 for interlace) aux buffer to the host, it’s a good practice to query the actual number of bytes written by the HDMI Aux Extractor, then truncate (or zero) the remainder of the host aux buffer with zeroes, to prevent parsing of stale/invalid aux packet data.

For Capture:

Clients can parse the data themselves, using the above specification.
AJAAuxiliaryList::SetFromDeviceAncBuffers can be called to transform the buffer contents into an AJAAuxiliaryList containing one AJAAuxiliaryData instance per packet.

For Playout:

Clients can write the data themselves per the above specification.
AJAAuxiliaryList::GetTransmitData can be called to fill your Aux playout buffers from an AJAAuxiliaryList.
- If using AutoCirculate , these buffers can be transferred to the AJA device by first calling AUTOCIRCULATE_TRANSFER::SetAncBuffers , then calling CNTV2Card::AutoCirculateTransfer.
- If not using AutoCirculate, the buffers can be transferred to the AJA device using CNTV2Card::DMAWriteAnc.