Introduction
What is streaming?
Streaming or media streaming is a form of transferring data so it can be treated as a continuous stream. This method is preferred to downloading because the client can display the video before the entire file has been received.
The client must also be able to store data in a buffer in case it is received quicker than required.
What are the types of streaming?
While the past video streaming technologies relied on RTP with RTSP, today’s technologies are almost exclusively based on HTTP.
We can totally use the default HTTP streaming technology without anything else!
However, there is SmoothStreaming. This is a feature of Internet Information Services (IIS) Media Services, which is a HTTP-based platform. SmoothStreaming clients get minimal buffering, because this technology adapts the quality of the video stream in real-time with regards to the client’s bandwidth and CPU usage. This technology is known as adaptive streaming.
Choosing the platform and a framework
Picking the platform
Streaming can be implemented on any mobile/tablet platform, such as: iOS, Android, and WinRT (Windows Runtime). Due to the fact that the project we had was already implemented on iOS and Android, we decided to extend it to WinRT.
Selecting a framework that handles encrypted content
After choosing the WinRT platform, the logical choice is to subscribe to Microsoft technologies/products. The Microsoft content protection technology we used is PlayReady.
Licensing management
Handling DRM
Every DRM server has its own custom licensing system. Therefore, in order to handle this, custom authentication must be provided. The following example illustrates a custom authentication request.
public string GetAuthenticationData()
{
string dataString = "{\"loggedInUserId\": \"" + _userID + "\",\"STOKEN\": \"" + _currentSToken + "\"}";
var data = Convert.ToBase64String(Encoding.UTF8.GetBytes(customDataString));
return data;
}In the above example, a SToken and UserId are needed for the authentication process. These are encoded in a JSON format, which is in turn encoded in a Base64 format to be sent to the DRM server in order to obtain the license to play the respective content.
The five pillars of PlayReady
In order to use PlayReady in a WinRT project, one must first install PlayReady and add it as a reference. Afterwards, the following five classes must be used:
public class HttpHelper
{
protected IPlayReadyServiceRequest _serviceRequest = null;
private Uri _uri = null;
public HttpHelper(IPlayReadyServiceRequest serviceRequest)
{
_serviceRequest = serviceRequest;
}
public async Task GenerateChallengeAndProcessResponse()
{
DebugLogger.Log("Generating challenge..");
PlayReadySoapMessage soapMessage = _serviceRequest.GenerateManualEnablingChallenge();
if (_uri == null)
{
_uri = soapMessage.Uri;
}
DebugLogger.Log("Getting message body..");
byte[] messageBytes = soapMessage.GetMessageBody();
HttpContent httpContent = new ByteArrayContent(messageBytes);
IPropertySet propertySetHeaders = soapMessage.MessageHeaders;
DebugLogger.Log("Http Headers:-");
foreach (string strHeaderName in propertySetHeaders.Keys)
{
string strHeaderValue = propertySetHeaders[strHeaderName].ToString();
DebugLogger.Log(strHeaderName + " : " + strHeaderValue);
if (strHeaderName.Equals("Content-Type", StringComparison.OrdinalIgnoreCase))
httpContent.Headers.ContentType = MediaTypeHeaderValue.Parse(strHeaderValue);
else
httpContent.Headers.Add(strHeaderName.ToString(), strHeaderValue);
}
DebugLogger.Log("Http Body:-");
HttpClient httpClient = new HttpClient();
HttpResponseMessage response = await httpClient.PostAsync(_uri, httpContent);
string strResponse = await response.Content.ReadAsStringAsync();
DebugLogger.Log("Http Response: " + strResponse);
DebugLogger.Log("Processing Response..");
Exception exResult = _serviceRequest.ProcessManualEnablingResponse(await response.Content.ReadAsByteArrayAsync());
if (exResult != null)
throw exResult;
DebugLogger.Log("Leave HttpHelper.GenerateChallengeAndProcessResponse()");
}
}public delegate void ReportResultDelegate(bool bResult);
public class Indiv : ServiceRequest
{
protected virtual void IndivServiceRequestCompleted(PlayReadyIndividualizationServiceRequest sender, Exception hrCompletionStatus)
{
}
public void IndivProactively()
{
PlayReadyIndividualizationServiceRequest indivRequest = new PlayReadyIndividualizationServiceRequest();
IndivReactively(indivRequest);
}
private async public void IndivReactively(PlayReadyIndividualizationServiceRequest indivRequest)
{
DebugLogger.Log("Enter Indiv.IndivReactively()");
Exception exception = null;
try
{
_serviceRequest = indivRequest;
DebugLogger.Log("Begin indiv service request...");
await indivRequest.BeginServiceRequest();
}
catch (Exception ex)
{
DebugLogger.Log("Saving exception..");
exception = ex;
}
finally
{
IndivServiceRequestCompleted(indivRequest, exception);
}
DebugLogger.Log("Leave Indiv.IndivReactively()");
}
}
public class IndivAndReportResult : Indiv
{
private ReportResultDelegate _reportResult = null;
public IndivAndReportResult(ReportResultDelegate callback)
{
_reportResult = callback;
}
protected override void IndivServiceRequestCompleted(PlayReadyIndividualizationServiceRequest sender, Exception hrCompletionStatus)
{
DebugLogger.Log("Enter IndivAndReportResult.IndivServiceRequestCompleted()");
if (hrCompletionStatus == null)
{
DebugLogger.Log("********************************************Indiv succeeded**************************************************");
_reportResult(true);
}
else
{
//needed for LA revoke->Re-Indiv->LA sequence
if (!PerformEnablingActionIfRequested(hrCompletionStatus))
{
DebugLogger.Log("IndivServiceRequestCompleted ERROR: " + hrCompletionStatus.ToString());
_reportResult(false);
}
}
DebugLogger.Log("Leave IndivAndReportResult.IndivServiceRequestCompleted()");
}
protected override void EnablingActionCompleted(bool bResult)
{
DebugLogger.Log("Enter IndivAndReportResult.EnablingActionCompleted()");
_reportResult(bResult);
DebugLogger.Log("Leave IndivAndReportResult.EnablingActionCompleted()");
}
}public class LicenseAcquisition : ServiceRequest
{
protected virtual void LAServiceRequestCompleted(PlayReadyLicenseAcquisitionServiceRequest sender, Exception hrCompletionStatus)
{
}
private void HandleIndivServiceRequest_Finished(bool bResult)
{
DebugLogger.Log("Enter LicenseAcquisition.HandleIndivServiceRequest_Finished()");
DebugLogger.Log("HandleIndivServiceRequest_Finished(): " + bResult.ToString());
if (bResult)
{
AcquireLicenseProactively();
}
DebugLogger.Log("Leave LicenseAcquisition.HandleIndivServiceRequest_Finished()");
}
public void AcquireLicenseProactively()
{
try
{
PlayReadyContentHeader contentHeader = new PlayReadyContentHeader(
RequestConfigData.KeyId,
RequestConfigData.KeyIdString,
RequestConfigData.EncryptionAlgorithm,
RequestConfigData.Uri,
RequestConfigData.Uri,
String.Empty,
RequestConfigData.DomainServiceId);
DebugLogger.Log("Creating license acquisition service request...");
PlayReadyLicenseAcquisitionServiceRequest licenseRequest = new PlayReadyLicenseAcquisitionServiceRequest();
licenseRequest.ContentHeader = contentHeader;
AcquireLicenseReactively(licenseRequest);
}
catch (Exception ex)
{
if (ex.HResult == ServiceRequest.MSPR_E_NEEDS_INDIVIDUALIZATION)
{
PlayReadyIndividualizationServiceRequest indivServiceRequest = new PlayReadyIndividualizationServiceRequest();
RequestChain requestChain = new RequestChain(indivServiceRequest);
requestChain.FinishAndReportResult(new ReportResultDelegate(HandleIndivServiceRequest_Finished));
}
else
{
DebugLogger.Log("DomainJoinProactively failed:" + ex.HResult);
}
}
}
public static void DumpContentHeaderValues(PlayReadyContentHeader contentHeader)
{
DebugLogger.Log(" ");
DebugLogger.Log("Content header values:");
if (contentHeader == null)
return;
DebugLogger.Log("CustomAttributes :" + contentHeader.CustomAttributes);
DebugLogger.Log("DecryptorSetup :" + contentHeader.DecryptorSetup.ToString());
DebugLogger.Log("DomainServiceId :" + contentHeader.DomainServiceId.ToString());
DebugLogger.Log("EncryptionType :" + contentHeader.EncryptionType.ToString());
DebugLogger.Log("KeyId :" + contentHeader.KeyId.ToString());
DebugLogger.Log("KeyIdString :" + contentHeader.KeyIdString);
DebugLogger.Log("LicenseAcquisitionUrl :" + contentHeader.LicenseAcquisitionUrl.ToString());
}
private void ConfigureServiceRequest()
{
PlayReadyLicenseAcquisitionServiceRequest licenseRequest = _serviceRequest as PlayReadyLicenseAcquisitionServiceRequest;
DumpContentHeaderValues(licenseRequest.ContentHeader);
DebugLogger.Log(" ");
DebugLogger.Log("Configure license request to these values:");
licenseRequest.Uri = "DRM server’s license acquisition URL";
DebugLogger.Log("ChallengeCustomData:" + "Your Custom Data");
//assemble custom challenge for DRM
licenseRequest.ChallengeCustomData = "Your Custom Data";
DebugLogger.Log(" ");
}
public async void AcquireLicenseReactively(PlayReadyLicenseAcquisitionServiceRequest licenseRequest)
{
DebugLogger.Log("Enter LicenseAcquisition.AcquireLicenseReactively()");
Exception exception = null;
try
{
_serviceRequest = licenseRequest;
ConfigureServiceRequest();
DebugLogger.Log("ChallengeCustomData = " + licenseRequest.ChallengeCustomData);
if (RequestConfigData.ManualEnabling)
{
DebugLogger.Log("Manually posting the request...");
HttpHelper httpHelper = new HttpHelper(licenseRequest);
await httpHelper.GenerateChallengeAndProcessResponse();
}
else
{
DebugLogger.Log("Begin license acquisition service request...");
await licenseRequest.BeginServiceRequest();
}
}
catch (Exception ex)
{
DebugLogger.Log("Saving exception..");
exception = ex;
}
finally
{
DebugLogger.Log("Post-LicenseAcquisition Values:");
DebugLogger.Log("DomainServiceId = " + licenseRequest.DomainServiceId.ToString());
if (exception == null)
{
DebugLogger.Log("ResponseCustomData = " + licenseRequest.ResponseCustomData);
}
if (licenseRequest != null)
{
try
{
DumpContentHeaderValues(licenseRequest.ContentHeader);
LAServiceRequestCompleted(licenseRequest, exception);
}
catch (Exception ex)
{
if (ex.Message.Contains("0x8004C036"))
DebugLogger.Log(ex.ToString(), 1);
else
throw;
}
}
}
DebugLogger.Log("Leave LicenseAcquisition.AcquireLicenseReactively()");
}
}
public class LAAndReportResult : LicenseAcquisition
{
private ReportResultDelegate _reportResult = null;
private string _strExpectedError = null;
private ResponseCustomData _customData;
public string ExpectedError
{
set { this._strExpectedError = value; }
get { return this._strExpectedError; }
}
public LAAndReportResult(ReportResultDelegate callback)
{
_reportResult = callback;
}
protected override void LAServiceRequestCompleted(PlayReadyLicenseAcquisitionServiceRequest sender, Exception hrCompletionStatus)
{
DebugLogger.Log("Enter LAAndReportResult.LAServiceRequestCompleted()");
if (hrCompletionStatus == null)
{
DebugLogger.Log("************************************ License acquisition succeeded ****************************************");
_reportResult(true);
if (!string.IsNullOrEmpty(sender.ResponseCustomData))
{
try
{
ResponseCustomData responseCustomData = JsonConvert.DeserializeObject<ResponseCustomData>(sender.ResponseCustomData);
_customData = responseCustomData;
}
catch (Exception ex)
{
}
}
}
else
{
if (!PerformEnablingActionIfRequested(hrCompletionStatus) && !HandleExpectedError(hrCompletionStatus))
{
DebugLogger.Log("LAServiceRequestCompleted ERROR: " + hrCompletionStatus.ToString());
_reportResult(false);
}
}
DebugLogger.Log("Leave LAAndReportResult.LAServiceRequestCompleted()");
}
protected override void EnablingActionCompleted(bool bResult)
{
DebugLogger.Log("Enter LAAndReportResult.EnablingActionCompleted()");
_reportResult(bResult);
DebugLogger.Log("Leave LAAndReportResult.EnablingActionCompleted()");
}
protected override bool HandleExpectedError(Exception ex)
{
DebugLogger.Log("Enter LAAndReportResult.HandleExpectedError()");
if (string.IsNullOrEmpty(_strExpectedError))
{
DebugLogger.Log("Setting error code to " + RequestConfigData.ExpectedLAErrorCode);
_strExpectedError = RequestConfigData.ExpectedLAErrorCode;
}
bool bHandled = false;
if (_strExpectedError != null)
{
if (ex.Message.ToLower().Contains(_strExpectedError.ToLower()))
{
DebugLogger.Log("'" + ex.Message + "' Contains " + _strExpectedError + " as expected");
bHandled = true;
_reportResult(true);
}
}
DebugLogger.Log("Leave LAAndReportResult.HandleExpectedError()");
return bHandled;
}
}public class RequestChain
{
protected IPlayReadyServiceRequest _serviceRequest = null;
private ReportResultDelegate _reportResult = null;
private IndivAndReportResult _indivAndReportResult = null;
private LAAndReportResult _licenseAcquisition = null;
private ServiceRequestConfigData _requestConfigData = null;
public ServiceRequestConfigData RequestConfigData
{
set { this._requestConfigData = value; }
get { return this._requestConfigData; }
}
public RequestChain(IPlayReadyServiceRequest serviceRequest)
{
_serviceRequest = serviceRequest;
}
public void FinishAndReportResult(ReportResultDelegate callback)
{
_reportResult = callback;
HandleServiceRequest();
}
private void HandleServiceRequest()
{
if (_serviceRequest is PlayReadyIndividualizationServiceRequest)
{
HandleIndivServiceRequest((PlayReadyIndividualizationServiceRequest)_serviceRequest);
}
else if (_serviceRequest is PlayReadyLicenseAcquisitionServiceRequest)
{
HandleLicenseAcquisitionServiceRequest((PlayReadyLicenseAcquisitionServiceRequest)_serviceRequest);
}
else
{
DebugLogger.Log("ERROR: Unsupported serviceRequest " + _serviceRequest.GetType());
}
}
private void HandleServiceRequest_Finished(bool bResult)
{
DebugLogger.Log("Enter RequestChain.HandleServiceRequest_Finished()");
_reportResult(bResult);
DebugLogger.Log("Leave RequestChain.HandleServiceRequest_Finished()");
}
private void HandleIndivServiceRequest(PlayReadyIndividualizationServiceRequest serviceRequest)
{
DebugLogger.Log(" ");
DebugLogger.Log("Enter RequestChain.HandleIndivServiceRequest()");
_indivAndReportResult = new IndivAndReportResult(new ReportResultDelegate(HandleServiceRequest_Finished));
_indivAndReportResult.RequestConfigData = _requestConfigData;
_indivAndReportResult.IndivReactively(serviceRequest);
DebugLogger.Log("Leave RequestChain.HandleIndivServiceRequest()");
}
private void HandleLicenseAcquisitionServiceRequest(PlayReadyLicenseAcquisitionServiceRequest serviceRequest)
{
DebugLogger.Log(" ");
DebugLogger.Log("Enter RequestChain.HandleLicenseAcquisitionServiceRequest()");
_licenseAcquisition = new LAAndReportResult(new ReportResultDelegate(HandleServiceRequest_Finished));
_licenseAcquisition.RequestConfigData = _requestConfigData;
_licenseAcquisition.AcquireLicenseReactively(serviceRequest);
DebugLogger.Log("Leave RequestChain.HandleLicenseAcquisitionServiceRequest()");
}
}public class ServiceRequestConfigData
{
#region Fields
private Guid _guidKeyId = Guid.Empty;
private string _strKeyIdString = String.Empty;
private Guid _guidDomainServiceId = Guid.Empty;
private Guid _guidDomainAccountId = Guid.Empty;
Uprivate ri _domainUri = null;
private Uri _Uri = null;
private string _strChallengeCustomData = String.Empty;
private string _strResponseCustomData = String.Empty;
private PlayReadyEncryptionAlgorithm _encryptionAlgorithm;
private string _strExpectedLAErrorCode = String.Empty;
private bool _manualEnabling = false;
#endregion
#region Properties
public bool ManualEnabling
{
set { this._manualEnabling = value; }
get { return this._manualEnabling; }
}
public Guid KeyId
{
set { this._guidKeyId = value; }
get { return this._guidKeyId; }
}
public string KeyIdString
{
set { this._strKeyIdString = value; }
get { return this._strKeyIdString; }
}
public Uri Uri
{
set { this._Uri = value; }
get { return this._Uri; }
}
public string ChallengeCustomData
{
set { this._strChallengeCustomData = value; }
get { return this._strChallengeCustomData; }
}
public string ResponseCustomData
{
set { this._strResponseCustomData = value; }
get { return this._strResponseCustomData; }
}
//
// Domain related config
//
public Guid DomainServiceId
{
set { this._guidDomainServiceId = value; }
get { return this._guidDomainServiceId; }
}
public Guid DomainAccountId
{
set { this._guidDomainAccountId = value; }
get { return this._guidDomainAccountId; }
}
public Uri DomainUri
{
set { this._domainUri = value; }
get { return this._domainUri; }
}
//
// License acquisition related config
//
public PlayReadyEncryptionAlgorithm EncryptionAlgorithm
{
set { this._encryptionAlgorithm = value; }
get { return this._encryptionAlgorithm; }
}
public string ExpectedLAErrorCode
{
set { this._strExpectedLAErrorCode = value; }
get { return this._strExpectedLAErrorCode; }
}
#endregion
}
public class ServiceRequest
{
private ServiceRequestConfigData _requestConfigData = null;
protected IPlayReadyServiceRequest _serviceRequest = null;
private RequestChain _requestChain = null;
public const int MSPR_E_CONTENT_ENABLING_ACTION_REQUIRED = -2147174251;
public const int DRM_E_NOMORE_DATA = -2147024637; //( 0x80070103 )
public const int MSPR_E_NEEDS_INDIVIDUALIZATION = -2147174366; // (0x8004B822)
public ServiceRequestConfigData RequestConfigData
{
set { this._requestConfigData = value; }
get
{
if (this._requestConfigData == null)
return new ServiceRequestConfigData();
else
return this._requestConfigData;
}
}
protected bool IsEnablingActionRequested(Exception ex)
{
bool bRequested = false;
COMException comException = ex as COMException;
if (comException != null && comException.HResult == MSPR_E_CONTENT_ENABLING_ACTION_REQUIRED)
{
bRequested = true;
}
return bRequested;
}
protected virtual void EnablingActionCompleted(bool bResult)
{
}
protected virtual bool HandleExpectedError(Exception ex)
{
return false;
}
protected bool PerformEnablingActionIfRequested(Exception ex)
{
DebugLogger.Log("Enter ServiceRequest.PerformEnablingActionIfRequested()");
bool bPerformed = false;
if (IsEnablingActionRequested(ex))
{
IPlayReadyServiceRequest nextServiceRequest = _serviceRequest.NextServiceRequest();
if (nextServiceRequest != null)
{
DebugLogger.Log("Servicing next request...");
_requestChain = new RequestChain(nextServiceRequest);
_requestChain.RequestConfigData = _requestConfigData;
_requestChain.FinishAndReportResult(new ReportResultDelegate(RequestChain_Finished));
bPerformed = true;
}
}
DebugLogger.Log("Leave ServiceRequest.PerformEnablingActionIfRequested()");
return bPerformed;
}
private void RequestChain_Finished(bool bResult)
{
DebugLogger.Log("Enter ServiceRequest.RequestChain_Finished()");
EnablingActionCompleted(bResult);
DebugLogger.Log("Leave ServiceRequest.RequestChain_Finished()");
}
}
Implementing the player
PlayerFramework
The first logical step after knowing how to handle DRM is obviously to use this in a player. One could choose to use Microsoft’s MediaPlayer in WinRT. In spite of this being a good idea, the MediaPlayer is rather rudimentary.
When using it, we discover that we have to do a lot of grunt work in order to allow it to use adaptive streaming, DRM, and even closed captioning.
The next course of action would be to seek a player designed to work with these technologies.
The search should be very short, as one would discover that Microsoft has already developed and is continuing development for the PlayerFramework.
So what is this PlayerFramework? It’s actually what we were looking for, and even more. Besides the fact that it supports all of the above listed requirements and is in continuous development, it’s open source! Yes, you’ve read that right.
Configuring the player
Assuming that we’ve created a blank page in our application called “PlayerPage”, we should add the player inside the page using XAML. The whole XAML should look like this:
<Page
x:Class="StreamingTest.PlayerPage"
xmlns="//schemas.microsoft.com/winfx/2006/xaml/presentation"
xmlns:x="//schemas.microsoft.com/winfx/2006/xaml"
xmlns:local="using:StreamingTest"
xmlns:d="//schemas.microsoft.com/expression/blend/2008"
xmlns:mc="//schemas.openxmlformats.org/markup-compatibility/2006"
xmlns:ttmlcs="using:Microsoft.PlayerFramework.TTML.CaptionSettings"
xmlns:PlayerFramework="using:Microsoft.PlayerFramework"
mc:Ignorable="d">
<Grid Background="{ThemeResource ApplicationPageBackgroundThemeBrush}">
<PlayerFramework:MediaPlayer x:Name="mediaPlayer"
IsCaptionSelectionEnabled="True"
IsCaptionSelectionVisible="True"
IsFullScreen="True"
IsGoLiveVisible="True"
VerticalAlignment="Center"
HorizontalAlignment="Center"
MediaQuality="HighDefinition"
Stretch="Uniform">
<PlayerFramework:MediaPlayer.Plugins>
<ttmlcs:TTMLCaptionSettingsPlugin/>
</PlayerFramework:MediaPlayer.Plugins>
</PlayerFramework:MediaPlayer>
</Grid>
</Page>Some of these settings could be eluded of course. If one does not wish to use TTML captions, one might skip this particular plugin.
In order to proceed with what you need to write in the code-behind, you will also require a Playback.cs class, responsible for whatever happens with the PlayerFramework Media Player (which includes setting the Source property in order to play streams). Therefore, one must make the class “Playback.cs”, which contains the following code:
public class Playback : IDisposable
{
private MediaPlayer _mediaElement = null;
protected MediaProtectionManager _protectionManager = new MediaProtectionManager();
private MediaProtectionServiceCompletion _serviceCompletionNotifier = null;
private RequestChain _requestChain = null;
private ServiceRequestConfigData _requestConfigData = null;
public static string LAURL = "Custom DRM License Acquisition URL";
public ServiceRequestConfigData RequestConfigData
{
set { this._requestConfigData = value; }
get { return this._requestConfigData; }
}
/// <summary>
/// Initializes a new instance of the <see cref="Playback"/> class.
/// </summary>
public Playback()
{
DebugLogger.Log("********************* Initializes Playback instance");
}
private void HookEventHandlers()
{
_mediaElement.CurrentStateChanged += new RoutedEventHandler(CurrentStateChanged);
_mediaElement.MediaEnded += MediaEnded;
_mediaElement.MediaFailed += new ExceptionRoutedEventHandler(MediaFailed);
_mediaElement.MediaOpened += new RoutedEventHandler(MediaOpened);
}
private void UnhookEventHandlers()
{
_mediaElement.CurrentStateChanged -= new RoutedEventHandler(CurrentStateChanged);
_mediaElement.MediaEnded -= MediaEnded;
_mediaElement.MediaFailed -= new ExceptionRoutedEventHandler(MediaFailed);
_mediaElement.MediaOpened -= new RoutedEventHandler(MediaOpened);
}
private void SetupProtectionManager()
{
_protectionManager.Properties.Clear();
_protectionManager.ComponentLoadFailed += new ComponentLoadFailedEventHandler(ProtectionManager_ComponentLoadFailed);
_protectionManager.ServiceRequested += new ServiceRequestedEventHandler(ProtectionManager_ServiceRequested);
DebugLogger.Log("Creating protection system mappings...");
//Setup PlayReady as the ProtectionSystem to use
//The native ASF media source will use this information to instantiate PlayReady ITA (InputTrustAuthority)
Windows.Foundation.Collections.PropertySet cpSystems = new Windows.Foundation.Collections.PropertySet();
cpSystems.Add("{9a04f079-9840-4286-ab92-e65be0885f95}", "Microsoft.Media.PlayReadyClient.PlayReadyWinRTTrustedInput"); //Playready TrustedInput Class Name
_protectionManager.Properties.Add("Windows.Media.Protection.MediaProtectionSystemIdMapping", cpSystems);
_protectionManager.Properties.Add("Windows.Media.Protection.MediaProtectionSystemId", "{9a04f079-9840-4286-ab92-e65be0885f95}");
_mediaElement.ProtectionManager = _protectionManager;
}
public Playback(MediaPlayer mediaPlayer, LoggedInUserInfo currentUser)
{
DebugLogger.Log("Entered Playback()");
_mediaElement = mediaPlayer;
DebugLogger.Log("Left Playback()");
}
public void Play(string strMediaPath)
{
PlayInternal(strMediaPath);
}
private void PlayInternal(string strMediaPath)
{
DebugLogger.Log("Enter Playback.Play()");
SetupProtectionManager();
HookEventHandlers();
_mediaElement.Source = new Uri(strMediaPath, UriKind.Absolute);
DebugLogger.Log("Leave Playback.Play()");
}
private void ProtectionManager_ComponentLoadFailed(MediaProtectionManager sender, ComponentLoadFailedEventArgs e)
{
DebugLogger.Log("Enter Playback.ProtectionManager_ComponentLoadFailed()");
DebugLogger.Log(e.Information.ToString());
// List the failing components - RevocationAndRenewalInformation
for (int i = 0; i < e.Information.Items.Count; i++)
{
DebugLogger.Log(e.Information.Items[i].Name + "\nReasons=0x" + e.Information.Items[i].Reasons + "\n"
+ "Renewal Id=" + e.Information.Items[i].RenewalId);
}
e.Completion.Complete(false);
DebugLogger.Log("Leave Playback.ProtectionManager_ComponentLoadFailed()");
}
private void ProtectionManager_ServiceRequested(MediaProtectionManager sender, ServiceRequestedEventArgs srEvent)
{
DebugLogger.Log("Enter Playback.ProtectionManager_ServiceRequested()");
_serviceCompletionNotifier = srEvent.Completion;
IPlayReadyServiceRequest serviceRequest = (IPlayReadyServiceRequest)srEvent.Request;
DebugLogger.Log("Service request type = " + serviceRequest.GetType());
_requestChain = new RequestChain(serviceRequest);
_requestChain.RequestConfigData = this.RequestConfigData;
_requestChain.FinishAndReportResult(new ReportResultDelegate(HandleServiceRequest_Finished));
DebugLogger.Log("Leave Playback.ProtectionManager_ServiceRequested()");
}
private void HandleServiceRequest_Finished(bool bResult)
{
DebugLogger.Log("Enter Playback.HandleServiceRequest_Finished()");
DebugLogger.Log("MediaProtectionServiceCompletion.Complete = " + bResult.ToString());
_serviceCompletionNotifier.Complete(bResult);
DebugLogger.Log("Leave Playback.HandleServiceRequest_Finished()");
}
protected void CurrentStateChanged(object sender, RoutedEventArgs e)
{
DebugLogger.Log("CurrentState:" + ((MediaPlayer)sender).CurrentState);
}
protected virtual void MediaFailed(object sender, ExceptionRoutedEventArgs e)
{
UnhookEventHandlers();
DebugLogger.Log("MediaFailed Source: " + ((MediaPlayer)sender).Source);
DebugLogger.Log("Playback Failed");
DebugLogger.Log("MediaFailed: " + e.ErrorMessage);
}
protected virtual async void MediaEnded(object sender, RoutedEventArgs e)
{
UnhookEventHandlers();
DebugLogger.Log("MediaEnded: " + ((MediaPlayer)sender).Source);
DebugLogger.Log("Playback succeeded");
}
protected virtual void MediaOpened(object sender, RoutedEventArgs e)
{
DebugLogger.Log("MediaOpened: " + ((MediaPlayer)sender).Source);
}
public void Dispose()
{
UnhookEventHandlers();
_protectionManager.ComponentLoadFailed -= new ComponentLoadFailedEventHandler(ProtectionManager_ComponentLoadFailed);
_protectionManager.ServiceRequested -= new ServiceRequestedEventHandler(ProtectionManager_ServiceRequested);
if (_mediaElement != null)
{
_mediaElement.Close();
_mediaElement = null;
}
}
}
public class PlaybackAndReportResult : Playback
{
private ReportResultDelegate _reportResult = null;
private string _strExpectedError = null;
public PlaybackAndReportResult(ReportResultDelegate callback, string strExpectedError = null)
{
_reportResult = callback;
_strExpectedError = strExpectedError;
}
override protected void MediaEnded(object sender, RoutedEventArgs e)
{
DebugLogger.Log("Enter PlaybackAndReportResult.MediaEnded()");
base.MediaEnded(sender, e);
_reportResult(true);
DebugLogger.Log("Leave PlaybackAndReportResult.MediaEnded()");
}
override protected void MediaFailed(object sender, ExceptionRoutedEventArgs e)
{
DebugLogger.Log("Enter PlaybackAndReportResult.MediaFailed()");
base.MediaFailed(sender, e);
bool bHandled = false;
if (_strExpectedError != null)
{
if (e.ErrorMessage.ToLower().Contains(_strExpectedError.ToLower()))
{
DebugLogger.Log("'" + e.ErrorMessage + "' Contains " + _strExpectedError + " as expected");
bHandled = true;
}
}
_reportResult(bHandled);
DebugLogger.Log("Leave PlaybackAndReportResult.MediaFailed()");
}
}Now that we have the Playback.cs class, we may proceed with writing the following code in the PlayerPage.xaml.cs:
public sealed partial class PlayerPage : Page
{
#region Statics
private static MediaExtensionManager s_extensions;
private static IAdaptiveSourceManager s_adaptiveSourceManager;
private static PropertySet s_propertySet = new PropertySet();
#endregion
#region Fields
private Manifest _manifestObject;
private AdaptivePlugin _adaptivePlugin;
private AdaptiveStreamingManager _adaptiveStreamingManager;
private Playback _playback;
#endregion
#region Constructors
public PlayerPage()
{
this.InitializeComponent();
InitializeMediaPlayerPlugins();
InitializeAdaptiveStreamingManager();
//You MUST set this to true !
//If you do not, when you will change between
//audio/video/caption streams
//you will get a small fast-forward bug
mediaPlayer.RealTimePlayback = true;
}
static PlayerPage()
{
s_adaptiveSourceManager = AdaptiveSourceManager.GetDefault();
s_propertySet["{A5CE1DE8-1D00-427B-ACEF-FB9A3C93DE2D}"] = s_adaptiveSourceManager;
SetupExtensionManager();
}
#endregion
#region Helpers
private void InitializePlayback()
{
_playback = new Playback(mediaPlayer);
}
private static void SetupExtensionManager()
{
s_extensions = new MediaExtensionManager();
//Register ByteStreamHandler for PIFF
//This can be removed if you don't play PIFF content
//SmoothByteStreamHandler requires Microsoft Smooth Streaming SDK
s_extensions.RegisterByteStreamHandler("Microsoft.Media.AdaptiveStreaming.SmoothByteStreamHandler", ".ism", "text/xml", s_propertySet);
s_extensions.RegisterByteStreamHandler("Microsoft.Media.AdaptiveStreaming.SmoothByteStreamHandler", ".ism", "application/vnd.ms-sstr+xml", s_propertySet);
//Register ByteStreamHander for pyv/pya content
//This can be removed if you don't play pyv/pya content
s_extensions.RegisterByteStreamHandler("Microsoft.Media.PlayReadyClient.PlayReadyByteStreamHandler", ".pyv", "");
s_extensions.RegisterByteStreamHandler("Microsoft.Media.PlayReadyClient.PlayReadyByteStreamHandler", ".pya", "");
}
private void InitializeAdaptiveSourceManager()
{
s_adaptiveSourceManager.AdaptiveSourceOpenedEvent += adaptiveSourceManager_AdaptiveSourceOpenedEvent;
s_adaptiveSourceManager.AdaptiveSourceClosedEvent += adaptiveSourceManager_AdaptiveSourceClosedEvent;
}
private void InitializeAdaptiveStreamingManager()
{
_adaptiveStreamingManager = _adaptivePlugin.Manager;
_adaptiveStreamingManager.SelectingTracks += adaptiveStreamingManager_SelectingTracks;
}
private void InitializeMediaPlayerPlugins()
{
_adaptivePlugin = new AdaptivePlugin { InstreamCaptionsEnabled = true };
mediaPlayer.Plugins.Add(_adaptivePlugin);
var captionPlugin = new CaptionsPlugin();
captionPlugin.CaptionParsed += CaptionPlugin_CaptionParsed;
mediaPlayer.Plugins.Add(captionPlugin);
}
#endregion
#region Adaptive Source Manager Events
private void adaptiveSourceManager_AdaptiveSourceOpenedEvent(AdaptiveSource sender, AdaptiveSourceOpenedEventArgs args)
{
var adaptiveSource = args.AdaptiveSource;
adaptiveSource.ManifestReadyEvent += adaptiveSource_ManifestReadyEvent;
adaptiveSource.AdaptiveSourceFailedEvent += adaptiveSource_AdaptiveSourceFailedEvent;
}
private void adaptiveSourceManager_AdaptiveSourceClosedEvent(AdaptiveSource sender, AdaptiveSourceClosedEventArgs args)
{
var adaptiveSource = args.AdaptiveSource;
adaptiveSource.ManifestReadyEvent -= adaptiveSource_ManifestReadyEvent;
adaptiveSource.AdaptiveSourceFailedEvent -= adaptiveSource_AdaptiveSourceFailedEvent;
}
private void adaptiveSource_AdaptiveSourceFailedEvent(AdaptiveSource sender, AdaptiveSourceFailedEventArgs args)
{
DebugLogger.Log("AdaptiveSource error ! Details : " + args.HttpResponse);
}
private void adaptiveSource_ManifestReadyEvent(AdaptiveSource sender, ManifestReadyEventArgs args)
{
_manifestObject = args.AdaptiveSource.Manifest;
}
#endregion
protected override void OnNavigatedFrom(NavigationEventArgs e)
{
base.OnNavigatedFrom(e);
CleanupAdaptiveSourceManager();
_manifestObject = null;
}
protected override void OnNavigatedTo(NavigationEventArgs e)
{
base.OnNavigatedTo(e);
InitializeAdaptiveSourceManager();
InitializePlayback();
var streamURL = e.Parameter as string;
_playback.Play(streamURL);
}
private void CleanupAdaptiveSourceManager()
{
s_adaptiveSourceManager.AdaptiveSourceOpenedEvent -= adaptiveSourceManager_AdaptiveSourceOpenedEvent;
s_adaptiveSourceManager.AdaptiveSourceClosedEvent -= adaptiveSourceManager_AdaptiveSourceClosedEvent;
}
}The manifest
As we’ve seen above, we can now play a stream protected by DRM. But what if we want to change between streams (if there is more than one video/audio/caption stream)?
When the manifest from the stream was ready, we saved it in a field called “_manifest” in order to use it for these scenarios.
Video stream
The video stream can only be one. It can have more than one tracks (which contain different qualities or bitrates). It can be obtained by creating a property like illustrated below:
public IManifestStream VideoStream
{
get { return _adaptiveStreamingManager.VideoStream; }
}Audio stream
Now that we have a property with the current video stream, we may obtain the different audio streams present inside of it using the following code (making a property in the process):
public List<IManifestStream> AudioStreams
{
get
{
List<IManifestStream> audioStreams = new List<IManifestStream>();
for (int i = 0; i < _manifestObject.AvailableStreams.Count; i++)
{
if (_manifestObject.AvailableStreams[i].Type == MediaStreamType.Audio)
audioStreams.Add(_manifestObject.AvailableStreams[i]);
}
return audioStreams;
}
}Subtitle stream
We can obtain the subtitle (caption) stream from the same video stream, by making another property, like so:
public List<IManifestStream> CaptionStreams
{
get
{
List<IManifestStream> captionStreams = new List<IManifestStream>();
for (int i = 0; i < _manifestObject.AvailableStreams.Count; i++)
{
if (_manifestObject.AvailableStreams[i].Type != MediaStreamType.Audio && _manifestObject.AvailableStreams[i].Type != MediaStreamType.Video)
captionStreams.Add(_manifestObject.AvailableStreams[i]);
}
return captionStreams;
}
}All of these streams can be extracted from the manifest object for further examination and processing.
Changing between streams while playing
What if we want to change the current audio stream from English to French? How do we do that?
How can we change the current caption stream?
These are the questions that one needs to ask herself/himself when attempting to switch between streams while the player is active. There are 2 ways of making this work:
Through the manifest
One can use the VideoStreams and CaptionStreams properties explained above in order to change them from the manifest object itself.
The way to do this is simple, following these steps:
1. In the adaptiveSourceManager_AdaptiveSourceOpenedEvent event, one must also write the following:
adaptiveSource.AdaptiveSourceStatusUpdatedEvent += adaptiveSource_AdaptiveSourceStatusUpdatedEvent;
2. Once attached to the AdaptiveSourceStatusUpdatedEvent, everything is set. This event will fire each time the manifest on the client is updated. Effectively, one can use the method ‘SelectStreamsAsync’ to set the corresponding audio/caption streams. We will not go into this however, since this has a higher time-complexity than the method we will present below.
Through the MediaPlayer’s properties
The MediaPlayer has two properties we can use: SelectedAudioStream and SelectedCaption. In order to obtain values for these two properties, we can use two other properties from the MediaPlayer: AvailableAudioStreams and AvailableCaptions.
For instance, if we want to set the audio stream and the caption stream to the first ones, we will write the following code:
mediaPlayer.SelectedAudioStream = mediaPlayer.AvailableAudioStreams.First(); mediaPlayer.SelectedCaption = mediaPlayer.AvailableCaptions.First();
These two assignments will set the current audio/caption streams accordingly.
Why are these two more effective than the above method?
Mainly it’s because the method presented above in “Through the manifest” has a higher time-complexity. It is a lower level operation granted, but it is handled much more gracefully by PlayerFramework when setting the properties SelectedAudioStream and SelectedCaption.
Challenges and issues
The main challenge we faced was the fast-forward bug that appeared whenever we switched between audio/caption streams. This was solved after hours of work switching between the methods from “Through the manifest” and “Through the MediaPlayer’s properties”, and finding the property RealTimePlayback present on the MediaPlayer which needed to be set to ‘true’.
Another challenge we faced was obviously when we were dealing with the DRM server. Since every DRM server is custom, license acquisition requests could return a status code 500, 504, 200, etc. which needed to be figured out thoroughly, and treated accordingly. This was achievable only because the DRM server API team and our development team communicated extremely efficient.
Closing thoughts
Even if creating a player that actually works and handles multiple streams might seem like a difficult task, because of the prerequisites (the five pillars, Playback.cs, treating DRM), with logical thinking and thoroughness one can get the job done in an extremely fashionable manner.
Despite the fact that this project was much more than what we described here (player customization, subtitle/audio that are remembered for different streams), if one follows the right steps mentioned here, she/he can get a good start on creating a player that relies on streams with DRM protection and adaptive streaming.
Also worth mentioning, are the quirks such as the one found in PlayerPage.xaml.cs which cannot be deduced so easily and require a fair amount of research:
s_propertySet[“{A5CE1DE8-1D00-427B-ACEF-FB9A3C93DE2D}”] = s_adaptiveSourceManager;
In conclusion, we hope that these guidelines will help anyone who wishes to create apps which use players to stream data.
References:
Author:
Ștefan-Gabriel Gavrilaș
Co-Author:
Ionuț-Cătălin Popovici