Building a broadcast AIT

There are several tools that can be used to build broadcast AIT. This tutorial uses TSDuck.

TSDuck uses an XML configuration file to define a broadcast AIT, this should not be confused with an XML AIT which is different purpose and has a different schema.

The user manual for TSDuck can be found here.

TSDuck AIT Parameter Definition

An example TSDuck AIT configuration file is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<tsduck>
  <AIT version="1" current="true" test_application_flag="false" application_type="0x0010">
    <application control_code="0x01">
      <application_identifier organization_id="0x00000070" application_id="0x0001"/>
      <transport_protocol_descriptor transport_protocol_label="0x01">
        <http>
          <url base="https://hbbtv-tests.s3.eu-west-1.amazonaws.com/Tutorials"/>
        </http>
      </transport_protocol_descriptor>
      <application_descriptor service_bound="true" visibility="3" application_priority="1">
        <profile application_profile="0x0000" version="1.3.1"/>
        <transport_protocol label="0x01"/>
      </application_descriptor>
      <application_name_descriptor>
        <language code="eng" application_name="Test"/>
      </application_name_descriptor>
      <simple_application_location_descriptor initial_path="rc-interaction/rc-interaction.html"/>
    </application>

  </AIT>
</tsduck>

The definition of the parameters used in the TSDuck AIT definition file are as follows, more details on the elements of a AIT are here:

VParameter	Type	Description
version	integer	Version of the AIT table, should be incremented as changes are made.
current	bool	Should be set to ‘true’.
test_application_flag	bool	Should be set to ‘false’.
application_type	integer	Should be 0x0010 for HbbTV.
metadata PID	integer	This is the PID of the AIT. This will appear in an extraction but is not require for a source configuration.
control_code	integer	Should be set to 0x01 to autostart the application.
organiszation_id	integer	ID of the organisation that is responsible for the application. These are allocated by DVB.
application_id	integer	Uniquely identifies the application along with the organisation id. These will normally be allocated by the developer.
transport_protocol_label	integer	Identifies the transport protocol if more than one is used. Most HbbTV applications will have a single transport protocol. Some might have a fall-back broadcast application to use when no broadband connection is available.
url base	string	Base URL for the application.
service_bound	bool	Indicates if the application is bound (‘true’) to a service or is unbound (‘false’).
visibility	integer	Must be set to ‘3’.
application_priority	integer	Sets the priority of the applications if multiple are signalled.
application_profile	integer	For most applications this will be ‘0x0000‘.
version	string	Minimum ETSI HbbTV version number the application is built for, in the format major.minor.micro.
transport_protocol label	integer	Links the application definition to the transport protocol definition.
application_name	string	This is a name for the application and can be sent in different languages.
simple_application_location_descriptor initial_path	string	The initial path of the application to be added to the base URL.

Replacing an existing AIT

The simplest way to create a test transport stream for an HbbTV application is to replace an existing AIT in a transport stream rather than creating one from scratch.

To do this you will need to understand the basics about MPEG program specific information (PSI) and in particular the program map table (PMT), and DVB Service Information and in particular the service description table (SDT).

The steps to replace an existing AIT are:

• Obtain a baseline transport stream
• Determine the PID of the AIT you wish to replace in the transport stream
• Extract the existing AIT from the transport stream
• Update the AIT to match your needs
• Replace the updated AIT in the original transport stream
• Test the result

Obtain a baseline transport stream

The TSDuck website has a repository with a large number of transport streams captured from across the world, from Australia to the UK. These can be used as a baseline from which components can be replaced, removed and added.

It is a good idea to test the source transport stream works in your test environment to know that you have a good baseline.

Determine the AIT PID

There several ways of doing this. One approach is to extract the PSI and SI from the transport stream using TSDuck. To find the PID of the AIT you want to change will require you to extract the SDT and the PMT. The SDT will tell you the PMT you want to access, to find the AIT you want to change.

The following TSDuck command will extract the SDT in XML form from a transport stream:

tsp -I file tsfile.ts -P tables --tid 0x42 --xml-output sdt.xml -O drop

Where:

• The original transport stream file is: tsfile.ts
• 0x42 is the table id of the SDT from the actual network, that is the transport stream you are working with
• The file into which the SDT tables are to be extracted as xml is: sdt.xml

The XML output should list all the services in the transport stream and the service name and service ID of channel you want to modify the AIT for. Search for the the service name of the channel and look up the service ID for it. You should have any entry like:

<service service_id="0x0231" EIT_schedule="true" EIT_present_following="true" CA_mode="false" running_status="running">
    <service_descriptor service_type="0x01" service_provider_name="ABC" service_name="ABC TV"/>
    <default_authority_descriptor fqdn="CRID://melbourne2.abc.net.au"/>
</service>

In the above the service ID for the service ‘ABC TV’ is 0x231.

The following TSDuck command will extract the all the PMTs in XML form from a transport stream:

tsp -I file tsfile.ts -P tables --tid 0x02 --xml-output pmt.xml -O drop

Where:

• The original transport stream file is: tsfile.ts
• 0x02 is the table id of PMTs
• The file into which the PMT tables are to be extracted as xml is: pmt.xml

You want to search the file for the service_id for the service you are looking for. In the example above it is 0x0231. The service_id is also known as the program number, but in the TSDuck XML file for a PMT it is listed as service_id.

An example PMT definition from the TSDuck XML output is:

<PMT version="24" current="true" service_id="0x0231" PCR_PID="0x0080">
    <metadata PID="256"/>
    <maximum_bitrate_descriptor maximum_bitrate="44,000"/>
    <component elementary_PID="0x0200" stream_type="0x02">
      <stream_identifier_descriptor component_tag="0x01"/>
      <video_stream_descriptor multiple_frame_rate="false" frame_rate_code="3" MPEG_1_only="false" constrained_parameter="true" still_picture="false" profile_and_level_indication="0x48" chroma_format="1" frame_rate_extension="false"/>
      <data_stream_alignment_descriptor alignment_type="0x02"/>
      <maximum_bitrate_descriptor maximum_bitrate="4,235,600"/>
    </component>
    <component elementary_PID="0x028A" stream_type="0x03">
      <stream_identifier_descriptor component_tag="0x02"/>
      <maximum_bitrate_descriptor maximum_bitrate="272,400"/>
      <ISO_639_language_descriptor>
        <language code="eng" audio_type="0x00"/>
      </ISO_639_language_descriptor>
    </component>
    <component elementary_PID="0x157C" stream_type="0x05">
      <stream_identifier_descriptor component_tag="0x10"/>
      <maximum_bitrate_descriptor maximum_bitrate="0"/>
      <application_signalling_descriptor>
        <application application_type="0x0010" AIT_version_number="0x01"/>
      </application_signalling_descriptor>
    </component>
</PMT>

For the service (PMT) that you want to replace the AIT, you want to look for the PID with an application_signalling_descriptor. This will be the AIT. Take a note of this PID in order to extract just the AIT.

In the example above the AIT PID is 0x157C.

The AIT PID could also be determined using the transport stream analysis tools that come with your modulator.

Extract the existing AIT

The following TSDuck command will extract a specific AIT on a known PID, from a transport stream:

tsp -I file tsfile.ts -P tables -p <PID> --xml-output ait.xml -O drop

Where:

• The original transport stream file is: tsfile.ts
• The PID on which the AIT is to be extracted is: <PID> (e.g. 0x157C)
• The file into which the AIT is to be extracted as xml is: ait.xml

Update the AIT to match your needs

Use a text editor to edit the ait.xml extracted in the previous step.

The main change will be to update the url base and the simple_application_location_descriptor initial_path to match those of your application.

Replace the AIT in the transport stream

The following TSDuck command will replace an AIT on a specific PID:

tsp -I file tsfile.ts -P inject -p <PID> -r ait.xml -O file tsfile-new.ts

Where:

• The original transport stream file is: tsfile.ts
• The PID on which the AIT to be replaced is: <PID> (e.g. 0x157C)
• The new AIT which you have created is in file: ait.xml
• The new transport stream file with the replacement will be: tsfile-new.ts

Test the result

Once you have created the new transport stream file you will need to test it using your chosen test environment.

Adding an AIT to a transport stream with out one

If the baseline transport stream you are working with does not have an AIT then you will need to add it in. Not only will you need to inject the AIT into the transport stream you will also need to update the PMT for the service/channel you want launch you application from with an reference to the AIT.

The steps you need to complete are:

1. Obtain your baseline transport stream
2. Test your transport stream is working in your test environment
3. Determine the PID for the PMT for the service you wish to add the AIT to
4. Extract the PMT
5. Select a PID for the AIT
6. Update the PMT
7. Create the AIT
8. Inject the AIT into the transport stream
9. Test the result

Obtain your baseline transport stream

The TSDuck website has a repository with a large number of transport streams captured from across the world, from Australia to the UK. These can be used as a baseline from which components can be replaced, removed and added. Many of these examples will already have an AIT and this should be replaced following the instructions above.

It is a good idea to test the source transport stream works in your test environment to know that you have a good baseline.

Determine the PID for the PMT for the service you wish to add an AIT

This is a two step process, first you need to determine the service ID. This is done by looking at the SDT.

The following TSDuck command will extract the SDT in XML form from a transport stream:

tsp -I file tsfile.ts -P tables --tid 0x42 --xml-output sdt.xml -O drop

Where:

• The original transport stream file is: tsfile.ts
• 0x42 is the table id of the SDT from the actual network, that is the transport stream you are working with
• The file into which the SDT tables are to be extracted as xml is: sdt.xml

Search the SDT XML file for the name of the service you want to launch your application from, then determine the service_id from that service’s descriptors.

The second step is to determine the PID used for the PMT from the PAT.

The following TSDuck command will extract the PAT in XML form from a transport stream:

tsp -I file tsfile.ts -P tables --tid 0x00 --xml-output pat.xml -O drop

Where:

• The original transport stream file is: tsfile.ts
• 0x00 is the table id of the PAT
• The file into which the PAT tables are to be extracted as xml is: pat.xml

Sample PAT is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<tsduck>
  <PAT version="0" current="true" transport_stream_id="0x0231" network_PID="0x0010">
    <metadata PID="0"/>
    <service service_id="0x0230" program_map_PID="0x0102"/>
    <service service_id="0x0231" program_map_PID="0x0100"/>
    <service service_id="0x0232" program_map_PID="0x0101"/>
    <service service_id="0x0233" program_map_PID="0x0103"/>
    <service service_id="0x0234" program_map_PID="0x0106"/>
    <service service_id="0x0235" program_map_PID="0x0107"/>
    <service service_id="0x0236" program_map_PID="0x0104"/>
    <service service_id="0x0237" program_map_PID="0x0105"/>
    <service service_id="0x0238" program_map_PID="0x0108"/>
    <service service_id="0x0239" program_map_PID="0x0109"/>
    <service service_id="0x023A" program_map_PID="0x010A"/>
    <service service_id="0x023B" program_map_PID="0x010B"/>
    <service service_id="0x023C" program_map_PID="0x010C"/>
    <service service_id="0x023D" program_map_PID="0x010D"/>
    <service service_id="0x023E" program_map_PID="0x010E"/>
    <service service_id="0x023F" program_map_PID="0x010F"/>
  </PAT>
</tsduck>

Search the PAT for the service_id you took from the SDT, and locate the required program_map_PID.

Extract the PMT from the transport stream

The following TSDuck command will extract the PMT in XML form on a known PID, from a transport stream:

tsp -I file tsfile.ts -P tables -p <PID> --xml-output pmt.xml -O drop

Where:

• The original transport stream file is: tsfile.ts
• The PID on which the PMT is to be extracted is: <PID> (e.g. 0x1000)
• The file into which the PMT is to be extracted as xml is: pmt.xml

Select a PID for the AIT

The AIT must have a unique unused PID. This must be selected in order to update the PMT. The transport stream needs to be analysed to pick one.

The following TSDuck command will analyse a transport stream and provide a list of all the PIDs in use:

tsp -I file tsfile.ts -P analyze --pid-list -O drop

Where:

• The original transport stream file is: tsfile.ts

Unfortunately the result is in decimal which will require conversion to hex to work with these examples.

For the tests with ffmpeg generated files, I’ve used 0x1100 for the AIT..

Update the extracted PMT

The following should be added into the PMT XML that was previously extracted.

<component elementary_PID="<PID>" stream_type="0x05">
    <stream_identifier_descriptor component_tag="0x10"/>
    <maximum_bitrate_descriptor maximum_bitrate="0"/>
    <application_signalling_descriptor>
        <application application_type="0x0010" AIT_version_number="0x01"/>
    </application_signalling_descriptor>
</component>

In the XML:

• The elementary_PID should be updated to be the PID you have selected for the AIT.
• A stream_type of 0x05 indicates that the stream carries private sections (not defined by MPEG.
• The component tag of 0x10 should be updated to be a unique component number with in the PMT
• The application_type of 0x0010 indicates that the AIT is signalling a HbbTV application
• AIT_version_number of 0x01 gives the version number of the AIT the PMT is referencing, this is used to signal if there are any changes to the AIT

This should be placed within the <PMT> tags. The easy place to place it is as the last <component>.

Replace the PMT in the transport stream

The following TSDuck command will replace a PMT on a specific PID:

tsp -I file tsfile.ts -P inject -p <PID> -r pmt.xml -O file tsfile-int.ts

Where:

• The original transport stream file is: tsfile.ts
• The PID on which the PMT to be replaced is: <PID> (e.g. 0x1000)
• The new PMT which you have created is in file: pmt.xml
• The new transport stream file with the replacement PMT will be: tsfile-int.ts

Create the AIT

With a text editor you need to create an AIT in TSDuck XML format. An example and the meaning of the parameters is given at the beginning of this guide section.

Inject the AIT into the transport stream

The following TSDuck command will inject an AIT into a transport stream:

tsp -I file tsfile-int.ts -P inject -p <PID> ait.xml=1000 -O file tsfile-new.ts

Where:

• The input transport stream file is: tsfile-int.ts
• The PID on which the AIT to be injected is: <PID> (e.g. 0x157C)
• The new AIT which you have created is in file: ait.xml
• The interval (in milliseconds) at which the AIT will be repeated in the stream (useful for signaling purposes) is set after the AIT XML file name. This example is 1000 ms (1 second) which will repeat the table every second.
• The new transport stream file with the replacement will be: tsfile-new.ts

Test the result

Once you have created the new transport stream file you will need to test it using your chosen test environment.

Building a transport stream from scratch

If you don’t have a baseline transport stream then you will want to build one from scratch. You can do this using FFmpeg. The output from FFmpeg will not have all the DVB tables and so will require some addition tables added to it which you can do using TSDuck, in particular the AIT.

The user manual for FFmpeg can be found here.

A classic video to work with is Bigs Bucks Bunny which can be downloaded from here.

The following command will create a transport stream using a source video file:

ffmpeg -i input.mp4 -c:v mpeg2video -b:v 4M -c:a mp2 -b:a 192k -vf "scale=720:576" -muxrate 5M -f mpegts tsfile.ts

The command break is as follows:

• -i input_video.mp4: Specifies the input file (it could be any format supported by FFmpeg).
• -c:v mpeg2video: Encodes the video as MPEG-2 (a common codec for DVB systems).
• -b:v 4M: Specifies the video bitrate as 4 Mbps.
• -c:a mp2: Encodes the audio as MPEG-1 Layer 2 (standard for DVB audio).
• -b:a 192k: Specifies the audio bitrate as 192 kbps.
• -vf "scale=720:576": Specifies that the output resolution should be SD
• -muxrate 5M: Specifies the total mux rate (bitrate of the entire transport stream) as 5 Mbps.
• -f mpegts: Specifies the output format as MPEG Transport Stream.
• tsfile.ts: The output file will be a transport stream suitable for DVB.

If you want to encode an HD service you could use:

ffmpeg -i input.mp4 -c:v libx264 -b:v 4M -c:a aac -b:a 192k -vf "scale=1920:1080" -muxrate 5M -f mpegts tsfile-HD.ts

The files created by FFmpeg will have a SDT but no other DVB tables.

You can rename the service in the SDT from the default values given by FFmpeg with the following TSDuck command:

tsp -I file tsfile.ts -P svrename Service01 -n "Test Service 01" -p "Test Provider" --lcn 1 -O file tsfile-int.ts

With this transport stream you can now following the instructions given to add an AIT to transport stream. By default FFmpeg will have placed the PMT on PID 0x1000.

The other DVB table you may want to add to the transport stream is a network information table (NIT). This will make it easier for your television is find your test services. The descriptors in the NIT will vary due to the type of the network (e.g. terrestrial, satellite) and the country you are building your transport stream for. TSDuck will build a generic NIT from the information in the PAT and SDT, but it needs a blank NIT to do this.

The following is a empty NIT TSDuck definition file for an NIT:

<?xml version="1.0" encoding="UTF-8"?>
<tsduck>
  <NIT version="1" current="true" network_id="0x3033" actual="true">
  </NIT>
</tsduck>

To add this empty NIT to a transport stream and update it to include services in the SDT and PAT use the following TSDuck command:

tsp -I file tsfile.ts -P inject -p 0x0010  nit.xml=1000 -P nit --build-service-list-descriptors -O file tsfile-nit.ts

This command strings to TSDuck processes together.

Where:

• The input transport stream file is: tsfile.ts
• The PID on which the PMT to be injected is: 0x0010 this is the standard PID for an NIT.
• The new NIT which you have created is in file: nit.xml
• The interval (in milliseconds) at which the NIT will be repeated in the stream (useful for signaling purposes) is set after the AIT XML file name. This example is 1000 ms (1 second) which will repeat the table every second.
• The new transport stream file with the replacement will be: tsfile-nit.ts

To check the NIT that has been created, use the following TSDuck command:

tsp -I file tsfile-nit.ts -P tables -p 0x0010 --xml-output nit-new.xml -O drop

You can modify this NIT XML file to add additional descriptors and then replace it in the transport stream file. The following file has had a logical channel number added for DTG based networks.

<?xml version="1.0" encoding="UTF-8"?>
<tsduck>
  <NIT version="4" current="true" network_id="0x3033" actual="true">
    <metadata PID="16"/>
    <network_name_descriptor network_name="London"/>
    <target_region_name_descriptor country_code="GBR" ISO_639_language_code="eng">
        <region region_name="England" primary_region_code="0x01"/>
        <region region_name="London" primary_region_code="0x01" secondary_region_code="0x12"/>
      </target_region_name_descriptor>
      <target_region_descriptor country_code="GBR">
        <region primary_region_code="0x01" secondary_region_code="0x12"/>
      </target_region_descriptor>
    <transport_stream transport_stream_id="0x0001" original_network_id="0xFF01" preferred_section="0">
      <service_list_descriptor>
        <service service_id="0x0001" service_type="0x01"/>
      </service_list_descriptor>
      <terrestrial_delivery_system_descriptor centre_frequency="490,000,000" bandwidth="8MHz" priority="HP" no_time_slicing="true" no_MPE_FEC="true" constellation="64-QAM" hierarchy_information="0" code_rate_HP_stream="2/3" code_rate_LP_stream="1/2" guard_interval="1/32" transmission_mode="8k" other_frequency="true"/>
      <private_data_specifier_descriptor private_data_specifier="OFCOM"/>
      <dtg_logical_channel_descriptor>
        <service service_id="0x0001" logical_channel_number="1" visible_service="true"/>
      </dtg_logical_channel_descriptor>
    </transport_stream>
  </NIT>
</tsduck>

Replacing the NIT in the transport stream can be done with the following TSDuck command:

tsp -I file tsfile-nit.ts -P inject -p 0x0010 -r nit-new.xml -O file tsfile-nit-new.ts