Multi-device Timing for Web

Abstract

Timing mechanisms allow operations to be executed at the correct time. The Web already has several mechanisms supporting timed operations, including setTimeout and setInterval, as well as controllers for media frameworks and animations. However, the Web lacks support for multi-device timing. A multi-device timing mechanism would allow timed operations across Web pages hosted by different devices. Multi-device timing is particularly important for the broadcasting industry, as it is the key enabler for web-based secondary device offerings. More generally, multi-device timing has wide utility in communication, collaboration and multi-screen presentation. This Community Group aims to define a common, multi-device, timing mechanism and a practical programming model. This will improve the Web as a platform for time-sensitive, multi-device Web applications.

Multi-device Timing

Timing mechanisms allow operations to be executed at the correct time. Timing has many purposes such as correct time ordering, delay, concurrency/synchronization, periodicity, or perhaps rate adjustments in progress or processing.

Of course, the Web already has several mechanisms supporting timed operations. Most famously setTimeout and setInterval allow fairly precise timed execution of JavaScript and is the basis for timed animations. The HTML5MediaElement also counts as timing mechanism, as it provides timed presentation of continuous media and playback controls. Similarly, the HTMLTrackElement offers time alignment of subtitles and chapter information. WebAnimations is a framework for animations with playback control.

However, the Web lacks support for multi-device timing. All the timing mechanisms above are limited in scope to a single Web page. In contrast, a multi-device timing mechanism would allow timed operations across Web pages hosted by different devices.

Goals

The Community Group aims to define a common, multi-device, timing mechanism and a practical programming model for time-sensitive, multi-device Web applications.

Vision

We envision multi-device timing resources, such as clocks, stop-watches, timeouts or controllers, to be explicitly represented as objects of the Web. This means that timing resources are identified by URL’s, and that they are hosted by dedicated Web servers or services. This way, multiple devices may share timing resources essentially by connecting to the same URL. Furthermore, devices may assert control over timing resources by interacting with a server-side object. For example, applications will have the ability to request timing resources to slow down or speed up, and request controllers to pause, resume or skip back to start. Effects of timing control will be equally available to all connected devices. Finally, since multi-device timing resources are true objects of the Web, traditional concepts of ownership and access control apply and may be enforced by servers. This way different permissions (i.e., read-only, read-write) may be given for different end-users and/or different application components.

This vision is in line with the classical client-server architecture of the Web. The vision also enables a programming model, where Web applications can define and use application-level timing resources (e.g., clocks, stop-watches, timeouts or controllers) for application-specific purposes. These timing resources then drive the execution of relevant, time-sensitive aspects of that application. In effect, time-sensitive execution can be remote controlled from anywhere in the world, via the server hosting the timing resources. This approach naturally matches the event driven nature of modern Web applications.

Multi-device timing resources will be modeled after existing timing concepts, i.e., the system clock, stop-watches, timeout-mechanisms, and controllers of media and animation frameworks. In short, the CG aims to provide multi-device implementations of these concepts (with the realization that this may call for modifications to existing concepts and API’s).

Multi-device timing implies that timing resources may easily be shared between distributed components in a multi-device Web application, or even between different applications all together. Note also that multi-device timing resources support the abstraction of being simultaneous (in real time). Essentially they implement distributed agreement in time. So, if an application-specific media controller is shared between, say a 100.000 devices, and they all evaluate the controller currentTime property (during playback), at exactly the same moment in time, they ideally should get the the same media offset. Furthermore, if one device requests the controller to skip ahead by 5 seconds, this ideally affects all connected clients equally and immediately. Implementations of multi-device timing resources must approximate this ideal behaviour as precisely as possible.

Furthermore, precise multi-device timing must be available for any permutation of device, network connection, OS or Web-browser. In short, multi-device timing resources should be available whenever and wherever the Web is available.

Finally, a main goal for a multi-device programming model is to facilitate easy and flexible combination of time-sensitive application components within multi-device applications. In particular, we imagine a common API between Web browsers and providers of multi-device timing resources. A common API would be a great contribution to the interoperability of time-sensitive application components, and would even allow independent systems to share temporal aspects without necessarily sharing any application data. A common API would also work against a development of proprietary timing solutions and technological “islands”.

Scope of Work

The CG will:

1. Define a common API for multi-device timing resources, i.e, clocks, stop-watches, timeouts and controllers.
2. Review existing programming concepts for timed operation, i.e., setTimeout, setInterval, HTML5MediaElement, HTMLTrackElement, WebAnimation, and suggest adjustments to simplify integration with multi-device timing resources.
3. Draft a protocol specification for a common interface between Web browsers and providers of multi-device timing resources.
4. Discuss native support for multi-device timing in Web browsers.

The outcome of this will likely be a combination of bug reports, API proposals and protocol specifications.

The CG will borrow concepts, ideas and solutions from the Shared Motion proposal [SMP] (aka. Media State Vectors [MSV]) as a starting point. Shared Motions is a generic concept for multi-device timing on the Web, supporting clocks, stop-watches, timeouts and a wide variety of controllers. Shared Motion adopts the centralised approach where motions are hosted online by specialised motion services. Research on Shared Motion documents timing errors < 10 ms when Shared Motion is used in distributed synchronization of HTML5 video and audio, by regular, non-optimised Web browsers [SYNQ]. Though the limits of scalability has not yet been formally documented, the light-weight nature of Shared Motion indicates that online timing services may be highly scalable.

Out of Scope

• Timing mechanisms based on assumptions of synchronized system clocks (e.g., NTP) are out of scope, as synchronized system clocks is not a realistic assumption in the Web environment. This is especially true on mobile devices.
• Timing mechanisms based on assumptions about ISP, network carrier, Intranet or other local communication are out of scope. Multi-device timing services should work anywhere the Web works, and can not make any assumptions, except that devices are connected to Internet, have an internal clock, and support TCP communication. In addition, it must continue to work in NAT setups without port forwarding etc.
• Timing mechanisms that can not reliably deliver lip-synch precision or better are out of scope. Although many applications do not require lip-synch, the ability to use a single mechanism independent of precision-requirements is valuable.
• Timing mechanisms based on multicast streaming or pulse propagation are out of scope, as precision is affected by variation in network latency.
• Timing mechanisms that just provide a clock (i.e., read-only) are out of scope. A multi-device timing mechanism for Web must support a programming model allowing application code to interact with and control timing aspects within the application.
• Timing mechanisms that do not scale well are out of scope. For example, multi-device timing in broadcasting scenarios might require individual timing of millions of devices.
• Timing mechanisms that only support one-way asymmetric control are out of scope, as interaction and control generally should not be limited to a single device, but be available from multiple devices. If desirable, asymmetric control may instead be achieved by means of application-specific restrictions layered on top of a symmetric mechanism.
• Data formats for media synchronization and timed operation are out of scope. In this CG we are only concerned with multi-device timing resources. The CG advocates a model where timing resources are cleanly separated from data resources. This implies that timing resources can be used in combination with any type of timed data, and remain relevant for any choice of data delivery mechanism. This model also supports separation of concern, as backend services can be specialized for timing only - essential with respect to high precision and scalability. Data formats for timed data are already covered by other groups within W3C as well as other standards bodies.

Importance

A multi-device timing mechanism would significantly improve the Web as a platform for timing sensitive multi-device applications.

The broadcasting industry in particular provides a host of use-cases for multi-device timing. For example, Web coverage of live sport may require a delay to match latency of broadcasting networks. Live Web content also requires time-shifted presentation to match later on-demand consumption. Web-based companion device offerings should be synchronized with the programs they enrich, both in live and on-demand settings. Accessibility features for TV may include the option of foreign language audio tracks delivered by smart phones, synchronized (lip-synch) with a shared screen. Traveling viewers might prefer the “light” option for TV programs. This could be audio plus light-weight timed HTML5, as an alternative to bandwidth consuming HD signals. Interactive, time-sensitive ads on companion devices may enable new forms of engagement and revenue.

More generally, multi-device timing is key to a number of very important functions in multi-device applications. Timed presentation of same content on different devices constitutes collaborative viewing and may also require collaborative control. For presentation tools, there could be solutions based on remote controlled HTML5 as an alternative to video-based screen sharing. Multi-device timing also enables multi-screen solutions. Visualization of complex timed data models may be split across multiple screens, yet navigated in unison along a common timeline. Multi-device timing may also be used to record data input from distributed sources according to a common clock, and then later to reproduce those timing aspects in multi-device playback.

In summary, multi-device timing comes with profound implications for the Web as common platform for communication, collaboration and presentation. It will likely affect all domains of Web-based activity; in particular broadcasting, online media, education, health, music, industry, and government.

Community and Business Group Process

Terms in this charter that conflict with those of the Community and Business Group Process are void.

Work Limited to Charter Scope

The group will not publish Community Group Reports that are Specifications on topics other than those listed under "Community Group Reports that are Specifications" above. See below for how to modify the charter. The CLA applies to these Community Group Reports.

Contribution Mechanics

For these Reports, Community Group participants agree to send contributions to either the group “contrib” list or to the general group list, with subject line starting "“[short-name-for spec]". When meeting discussion includes contributions, contributors are expected to record those contributions explicitly on the mailing list as described.

Chair Selection

Participants in this group choose their Chair(s) and can replace their Chair(s) at any time using whatever means they prefer. However, if 5 participants — no two from the same organization — call for an election, the group must use the following process to replace any current Chair(s) with a new Chair, consulting the Community Development Lead on election operations (e.g., voting infrastructure and using RFC 2777).

1. Participants announce their candidacies. Participants have 14 days to announce their candidacies, but this period ends as soon as all participants have announced their intentions. If there is only one candidate, that person becomes the Chair. If there are two or more candidates, there is a vote. Otherwise, nothing changes.
2. Participants vote. Participants have 21 days to vote for a single candidate, but this period ends as soon as all participants have voted. The individual who receives the most votes —no two from the same organization— is elected chair. In case of a tie, RFC2777 is used to break the tie. An elected Chair may appoint co-Chairs.

Participants dissatisfied with the outcome of an election may ask the Community Development Lead to intervene. The Community Development Lead, after evaluating the election, may take any action including no action.

Decision Process

This group will seek to make decisions when there is consensus. When the group discusses an issue on the mailing list and there is a call from the group for assessing consensus, after due consideration of different opinions, the Chair should record a decision and any objections. Participants may call for an online vote if they feel the Chair has not accurately determined the consensus of the group or if the Chair refuses to assess consensus. The call for a vote must specify the duration of the vote which must be at least 7 days and should be no more than 14 days. The Chair must start the vote within 7 days of the request. The decision will be based on the majority of the ballots cast. It is the Chair’s responsibility to ensure that the decision process is fair, respects the consensus of the CG, and does not unreasonably favor or discriminate against any group participant or their employer.

Transparency

The group will conduct all of its technical work on its public mailing list. Any decisions reached at any meeting are tentative. Any group participant may object to a decision reached at an online meeting within 7 days of publication of the decision on the mail list. That decision must then be confirmed on the mail list by the Decision Process above.

Amendments to this Charter

The group can decide to work on a proposed amended charter, editing the text using the Decision Process described above. The decision on whether to adopt the amended charter is made by conducting a 30-day vote on the proposed new charter. The new charter, if approved, takes effect on either the proposed date in the charter itself, or 7 days after the result of the election is announced, whichever is later. A new charter must receive 2/3 of the votes cast in the approval vote to pass. The group may make simple corrections to the charter such as deliverable dates by the simpler group decision process rather than this charter amendment process. The group will use the amendment process for any substantive changes to the goals, scope, deliverables, decision process or rules for amending the charter.

References

[SMP] Shared Motion Proposal

• https://lists.w3.org/Archives/Public/public-web-and-tv/2014Dec/0016.html/
• https://lists.w3.org/Archives/Public/public-web-and-tv/2014Dec/0017.html/

[MSV] Ingar M. Arntzen, Njål T. Borch and Christopher P. Needham. 2013. "The Media State Vector: A unifying concept for multi-device media navigation". In Proceedings of the 5th Workshop on Mobile Video (MoVid '13). ACM, New York, NY, USA, 61-66.

[SYNQ] Njål T. Borch and Ingar M. Arntzen, “Distributed Synchronization of HTML5 Media” (15/2014), http://norut.no/nb/node/3041/