Portable Web Publications for the Open Web Platform
W3C First Public Working Draft
- This version:
- Latest published version:
- Latest editor's draft:
- Github Repository
- Issues tracker:
- Github Issue Tracker
This document is also available in this non-normative format: ePub
This document introduces Portable Web Publications, a vision for the future of digital publishing that is based on a fully native representation of documents within the Open Web Platform. Portable Web Publications achieve full convergence between online and offline/portable document publishing: publishers and users won't need to choose one or the other, but can switch between them dynamically, at will.
Status of This Document
This section describes the status of this document at the time of its publication. Other documents may supersede this document. A list of current W3C publications and the latest revision of this technical report can be found in the W3C technical reports index at http://www.w3.org/TR/.
This document outlines a general vision and should not be considered a technical specification. Instead, its goal is to outline that vision and the possible technical directions to achieve it; more detailed technical work should be documented in separate documents.
This document was published by the Digital Publishing Interest Group as a First Public Working Draft. If you wish to make comments regarding this document, please send them to email@example.com (subscribe, archives). All comments are welcome.
Publication as a First Public Working Draft does not imply endorsement by the W3C Membership. This is a draft document and may be updated, replaced or obsoleted by other documents at any time. It is inappropriate to cite this document as other than work in progress.
This document was produced by a group operating under the 5 February 2004 W3C Patent Policy. The group does not expect this document to become a W3C Recommendation. W3C maintains a public list of any patent disclosures made in connection with the deliverables of the group; that page also includes instructions for disclosing a patent. An individual who has actual knowledge of a patent which the individual believes contains Essential Claim(s) must disclose the information in accordance with section 6 of the W3C Patent Policy.
This document is governed by the 1 September 2015 W3C Process Document.
Table of Contents
- 1. Our Vision
- 2. Why work on this now?
- 3. What are the areas of interest?
- 4. Terminology
- 5. Achieving convergence: work areas
- 5.1 General Architecture for Portable Web Publications
- 5.2 General Archive formats
- 5.3 Capturing overall publication structure
- 5.4 Document and fragment identification
- 5.5 Metadata: discovery
- 5.6 Styling and Layout, Pagination
- 5.7 Security Models
- 5.8 Presentation Control and Personalization
- 5.9 Models for embracing domain-specific restrictions and extensions
- 6. Portable Web Publications and EPUB 3
- 7. Conclusions
- A. References
1. Our Vision
Our vision for Portable Web Publications is to define a class of documents on the Web that would be part of the Digital Publishing ecosystem but would also be fully native citizens of the Open Web Platform. In this vision, the current format- and workflow-level separation between offline/portable and online (Web) document publishing is diminished to zero. These are merely two dynamic manifestations of the same publication: content authored with online use as the primary mode can easily be saved by the user for offline reading in portable document form. Content authored primarily for use as a portable document can be put online, without any need for refactoring the content. Publishers can choose to utilize either or both of these publishing modes, and users can choose either or both of these consumption modes. Essential features flow seamlessly between online and offline modes; examples include cross-references, user annotations, access to online databases, as well as licensing and rights management.
The precise definition of Portable Web Publications is provided in the section 4.1 Portable Web Publications. For the sake of this introduction, suffices to say that a Portable Web Publication is a collection of content items (e.g. pages, chapters, modules, articles) whose content is compatible with Web usage, and structured as a single, self-contained logical unit. Individual items can consist of text, images, graphics, possibly interactive mathematical or chemical formulae, as well as audio and video. These documents, by definition, have a default, linear “reading order”, however the user may choose to skip around in the content just as with a book on paper; alternatively, interactive aspects of the content may alter the reading order on behalf of the user.
2. Why work on this now?
Digital Publishing can be considered to be at a tipping point. Digital publishing formats like EPUB have been broadly adopted globally for trade ebooks, and is starting to gain adoption among textbook publishers as well as corporate marketing departments. However, these formats have largely been seen as an “offline” format up until now. Various browser extensions supporting such complex publications exist, and other solutions are available for delivering these publications in browsers. Browser- and cloud-based solutions require relatively complex server and/or client software. In many cases browser- and cloud-based solutions depend on a proprietary transformation of the packaged files into formats more suitable to network delivery. A focused effort to make digital publications first-class Open web Platform citizens will result in a significant reduction in the complexity of deploying publishing content into browsers for both online and offline consumption. Further, this focused effort will increase the momentum of digital publication and associated Web adoption across communities who are looking for an open, non-proprietary, next-generation portable document format.
The broader Web Platform can also be considered to be at a tipping point. Mobile platform web site use is diminishing in favor of native applications. Hybrid applications that use web content alongside native application technology, and web-technology-based system applications are growing. The specific means of delivering hybrid and web-technology-based system applications is currently proprietary to specific applications frameworks and/or browser platforms. The point of Portable Web Publications is to increase problem solving momentum in package, metadata, and offline support applicable to both portable publications and installed applications. Open and native solutions to replace proprietary packaging, metadata, and offline support are intended to ensure the broadest possible general adoption of the Open Web Platform.
In many respects, the convergence is already happening. In a number of areas (e.g., educational publications, travel books, etc.) publishers already exploit the advanced possibilities of packaged publishing formats to produce highly interactive documents whose features are very close to what one is used to on the Web (see the separate section 6. Portable Web Publications and EPUB 3 for some examples). And the converse is also true: tutorial and introductory articles have appeared on the Web that have the quality of traditional publications that one was used to see in a scientific magazine, but combined with the interactive possibilities of the Web (Mike Bostock’s article on visualizing algorithms or Bret Victor’s article on visualization are just two of several possible examples). However, the convergence still has a long way to go, and this topic that this document, and the concept of Portable Web Publications, aim to explore.
3. What are the areas of interest?
The convergence of Digital Publishing and the Open Web Platform provides a common set of solutions and opportunities to various stakeholders:
Book publishers are investing in the development of technical expertise in web technologies. While gaining understanding of technical topics is important to new and future publishing workflows, the lack of communication between the trade publishers and web application developer communities is resulting in unnecessary duplication and investments in effort.
Collaboration between the Web content development and publishing communities will result in major benefits to publishers. Adopting a universal and interoperable format means publishers can concentrate on engaging content authors in the production of high quality content. The web content development community can be relied on to deal with sophisticated technical issues (e.g., CSS, SVG). Potential future web content formats (e.g., 3D rendering) and various interactive web programs (e.g., visualization tools like D3) will naturally flow into the publishing realm through Portable Web Publications, hence increasing publishers' opportunities to sell new content products across the board.
Realizing new opportunities is a reality for publishers traditionally considered to be on the leading edge of technological advances in working with content. These publishers include STM and educational publishing houses, as well as scholarly and journal publishing organizations (see the section 3.2 Scholarly Journal and STM Publishers).
A converged platform will support more tools and services and a much larger population of trained practitioners compared to the current state of working in parallel universes.
3.2 Scholarly Journal and STM Publishers
Scholarly journal publishers also provide articles for download these days. The most popular distribution format for journal articles continues to be [PDF] as a direct reflection of the scholarly community which highly prioritizes linear text and preservation of print typography. Indeed, the original goal for scholarly publisher to make files available online was to enable readers to download and print content directly, instead of borrowing a paper copy of a journal issue and photocopying relevant articles.
But things are changing. First of all, Web-only publications become part of the mainstream (e.g., the multidisciplinary PLOS ONE or the new PeerJ CompSci journals) with the main content being published with traditional Web technologies like HTML and CSS. And there is much more. Scholarly communication increasingly uses additional media such as video, audio, animated graphics, or very large images, and the trend is to consider these as integral parts of the scientific output. (Mike Bostock’s recent article on visualizing algorithms or the “live” presentation of data in a paper published by F1000 Research are good examples for the new possibilities.) Furthermore, publishing the scientific data sources, like the results of a sociological survey or measurement output of biochemical experiments in XML or CSV formats, alongside the “main” publication, is also coming to the fore, with some journals and institutions actually requiring a public access to those. Gaining access to all these various media and contents both online and offline is important for scholars, whether the goal is to read the publication on the Web, or to download the papers for various reasons: reading the article offline, inclusion of the paper into bibliographic management systems like Mendeley or Zotero, or peer-reviewing submissions. Any offline format for scholarly purposes should be adapted to these needs.
Having essentially identical online and offline versions of the same content, including the usage of various media, leads to similar reading experiences whether online or offline (see, e.g., [Sigarchian]). User annotations, formal reviews, etc., performed by the scholar on a small, mobile device while being offline can be automatically synchronized with the online version as soon as there is Internet access. Being based on a general archival format, Portable Web Publications provide an easy way to consistently include video, audio, interactive scripts, any kind of data, and can naturally contain active links to the scientific data published elsewhere on the Web in case the data is too large to be distributed offline. These and other possibilities provided by Portable Web Publications may contribute to fundamentally change the way scholarly publishing works.
3.3 In-house Publishers
A special form of document production is related to technical and/or user documentation of complex products as well as complex administrative documents. Such documents are often akin to STM or scholarly publications edited by traditional trade or scholarly publishers but, often, the sheer quantity and complexity of production, as well as confidentiality requirements, mean that the production are done in-house. In many respects major corporations such as IBM, Intel, Renault, or Boeing, or institutions like the European Commission, the FAO, or the UNESCO have become specialized publishers themselves.
The quantity of documentation makes it infeasible to produce these documents in print (or print-only); instead, publishing them on the public Web, an Intranet, and/or providing them through specialized mobile devices is the viable alternative. The production of these documents has similar challenges to scholarly publications like accessibility issues, portability of annotations, or the possible inclusion of complex media.
Just as for scientific publications, Portable Web Publications will provide new possibilities for these types of documents. Documentation in Portable Web Publications can be used offline in, for example, a cockpit, while being easily updated through the Web when possible. Inclusion of interactive animation, explanations, etc., become easy thanks to the possibilities provided by the Open Web Platform, whether online or offline.
3.4 Reading System developers
Reading system developers will also benefit. It is already true today that, due to the large scale use of the Open Web Platform technologies in publishing formats, reading systems often rely on existing Web browser “cores”. This means that the development of these reading systems already benefit from a level of synergy insofar as they can rely on software developments done elsewhere. Making Portable Web Publications “native” to browsers will mean that an even larger percentage of the necessary software will be available as part of the “core” and developers can concentrate on book-specific issues such as specialized user interfaces or connection to online bookstores.
But the main advantage of Portable Web Publications for reading systems is a vastly larger user base. Whilst, today, reading systems are mainly used to read traditional novels, the introduction of Portable Web Publications will open up new possibilities for, e.g., scholarly and educational use, journals and magazines, governmental usage, etc.
3.5 Web page designers
The synergy between the traditional publishing community and the Web site designers may help in greatly improving the quality of overall Web page design. Indeed, the publishing community has significant experience on issues like ergonomy, complex layout design, paged layout, or user interface problems when consuming, for example, long, elaborate, and mostly linear content. Publishers also have an experience in a proper editorial and curatory workflow in producing content, which can be easily transposed from traditional publishing to Web site production.
Another aspect of Web page design is its adaption to various environments easily. Creating documents on the Web that could be uncompromisingly displayed both on traditional screens and on mobile devices with varying screen sizes is already a growing trend today; with Portable Web Publications, users will be able to create digitally native documents easily, whether the document is viewed online or offline.
3.6 Web browsers
Generation of an offline version of a Web page (mainly in terms of very long and complex content) is an area where browsers will benefit from Portable Web Publications. Such a facility is important: when roaming charging are often high, or when internet access may be of a low quality or not available at all, users need the possibility to create, in an ad-hoc and easy manner, an offline version of the Web page they are reading. Several browsers offer such facilities already, albeit in mutually incompatible formats. Making Portable Web Publications native to a browser means to standardize an archive format that can be used through a suitable user interface by anyone using a browser. Also, some of the facilities required by reading systems are also extremely useful for “traditional” Web content; annotation facilities are an obvious example. A joint development will therefore provide a welcome addition to the core browser facilities.
It must be emphasized, however, that Portable Web Publications are not meant to create an offline version of any Web page; the emphasis is on Web publications and not to, so to say, duplicate the Web. For example, it is not the goal of Portable Web Publications to store the page of a Web-based email client. The exact boundaries and limitations are specified in a separate section 4. Terminology.
Note that, technically, the inclusion of Portable Web Publication capabilities in browsers is a matter of enhancement rather than addition of completely new capabilities: because Portable Web Publications documents are based on core Web Technologies, the “extras” to make them a native feature of the Web is limited to a smaller set of tasks like handling packages, dealing with features like reading order, dynamic pagination, and displaying tables of contents. An important goal of Portable Web Publications will be to further streamline these tasks, all while making sure it is feasible to include Portable Web Publications content handling even in mobile environments, where the computing and memory limits are more demanding.
3.7 Libraries and archival services
Users will benefit, arguably the most, from a convergence of efforts between Portable Web Publications and other uses of Web technologies. Users will have the choice among different reading systems for the same content, ranging from specialized devices to traditional Web Browsers. Beyond the overall qualities of the reading environment the choice can also be made based on the content and usage: whereas a specialized device would work well for reading a novel on the beach, a Web browser or a high-end tablet may be preferred to consume highly interactive educational content in a class room. Publishers do not have to make this decision: users can do that. The same content can also smoothly migrate from one device or system to another, possibly carrying notes, annotations, but also the possibility to fill interactive Web forms offline (and “pushing” the results to its destination when on line again). Features for people with disabilities will also be provided consistently, whether the content is a portable document or a Web page.
This document is based on the following defintions.
4.1 Portable Web Publications
- A Web Resource is a digital resource that can be uniquely addressed by a Unified Resource Identifier (URI)[URI], and whose content can be accessed through standard network protocols like HTTP, FTP, or the File Protocol (i.e., based on the File URI Scheme[file-uri-scheme]), etc.
- Content of a Web Resource: information and sensory experience to be communicated to the user by means of a user agent, including code or markup that defines the content’s structure, presentation, and interactions.
- Essential Content of a Web Resource: content which, if removed, would fundamentally change the information or functionality of the Web Resource.
- Functionality related to a Web Resource: processes and outcomes achievable through user action.
- A Web Publication is a Web Resource which itself is an aggregated set of interrelated Web Resources, and which is intended to be considered as a single Web Resource. Furthermore:
- As a Web Resource, a Web Publication has its own [URI] which identifies the full set of the constituent resources (as opposed to a particular Web Resource within the Web Publication).
- A Web Publication must be constructed of resources whose formats enable (individually or in conjunction with other resources in the same Web Publication) delivery of essential content and functionality when delivered via a variety of technologies or delivery platforms. In particular, the essential constituents of a Web Publication must be based on technologies of the Open Web Platform.
- A Web Publication should provide a gracefully degrading experience when delivered via a variety of technologies or delivery platforms.
- A Web Publication should provide accessible access to content.
- A Web Publication is not an object with a precise technical meaning, e.g., it is not necessarily equivalent to an HTML Document.
- A Web Resource within a Web Publication is Portable if an OWP compliant user agent can render its essential content by relying essentially on the Web Resources within the same Web Publication
- A Portable Web Publication (PWP) is a Web Publication whose constituent Web Resources are all Portable.
Web Resources in a (Portable) Web Publication are based on the core Open Web Platform technologies like [html5], [svg], [css21] and CSS3 modules, or [ECMAScript] API-s. A Web Publication may also include images, audio and video, metadata files, executable code in, for example, iPython or Maple scripts, or datafiles in [CSV]: Web Resources that are needed to render the essential content of the publication.
The differences between the distinguishing characteristics of Web Publications and Portable Web Publications can be viewed as situational and gradual rather than as representative of bright-line distinctions. Consider the following example:
4.2 States of a Portable Web Publication
The states of a Portable Web Publication can be separated into two different axes. These different states require a different behavior from the user agent, while some of the characteristics of the publication may be invariant across states. The different states are as follows:
- States related to the organization of the Web Resources: A Portable Web Publication is in
- Packed State: when all constituents Web Resources are combined into one unit for storage in a file system, network transfer, etc.
- Unpacked State: when all constituent Web Resources can be directly accessed individually through standard network protocols like HTTP, FTP, etc., or through file system access
- States related to the access of Web Resources: A Portable Web Publication is in
- Protocol State: when all constituent Web Resources, as well as the Portable Web Publication itself, are accessed through network protocols like HTTP, HTTPS, or FTP
- File State: when all constituent Web Resources, as well as the Portable Web Publication itself, are accessed as files on a local file system
The table below shows the same publication (PWP) in the most typical states:
|Packed||PWP as one archive file on a Web Server||PWP as one archive file on a local disc|
|Unpacked||PWP spread over several files on a Web Server||PWP spread over several files on a local disc|
The difference between protocol and file states is not identical, although close, to the difference between the commonly used notions of “online” vs. “offline”. A PWP can be on a local disc but accessed through a Web Server running on that machine (i.e., through a
http://localhost URI): the PWP is in a protocol state, though clearly “offline”. Similarly, a remote file system can be mount on a local disc, in which case a PWP can be accessed as a file, i.e., is in a file state, though possibly “online”.
5. Achieving convergence: work areas
This section lists some of the work areas that activities around Portable Web Publications should engage in. The list is not exhaustive and there are only hints at the technical solutions; one of the main goals of the work ahead will be to clarify the requirements and technical details. It must be emphasized that the solutions to these problems may not come from W3C, but possibly from other, external organizations (document identification is a typical example).
5.1 General Architecture for Portable Web Publications
The latest evolution of browser technologies around Web Workers[web-workers] and Service Workers[service-workers] may fundamentally change the way browsers operate in terms of offline/online. Service Workers will provide a flexible and programmable way to efficiently implement local caching of Web Resources. Caching is implemented as a programmable network proxy, meaning that the browser’s rendering engine becomes oblivious on whether a resource originates from the local cache or diretly from the network.
This evolution makes it possible to bring Portable Web Publications under the same abstraction as a Web Resource identified by a URI. A specialized service worker can cache resources, i.e., can bridge the differences between the offline and online access of a document. Furthermore, by adding additional functionalities to the service worker, it can also deal with possible (un)packaging as well as bridging the differences between protocol and file states. In other words, the core rendering engine of a user agent can operate as if the Portable Web Publication was in unpacked and protocol states with all resources available; the differences are handled by the service worker acting as a proxy.