OpenWF - The Standard for building composited windowing systems

OpenWF™ is a royalty-free, cross-platform API that provides a low-level hardware abstraction interface for composited windowing systems to make use of composition and display hardware. OpenWF is targeted primarily at handheld devices that require portable acceleration of composition whilst minimizing memory bandwidth usage and power levels.

APIs for the Windowing System

OpenWF defines separate interfaces for composition and display control, reflecting the traditional model of independent hardware for each function. Figure 1 depicts OpenWF Composition and OpenWF Display being used by the central windowing system as part of a full graphics and multimedia system.

The OpenWF technologies complement the existing Khronos APIs, defining the underlying route to the display for advanced graphics and multimedia content generated using APIs such as OpenGL|ES, OpenVG and OpenMAX.

In a typical system such as that shown above, the application would access the features of OpenWF via either the native window system or through OpenKODE windowing. For instance, this could take the form of the window system offering the ability to give each window a percentage opacity value to make the window translucent.

OpenWF Composition

A wide range of visual scenarios can be implemented in terms of 2D layering. OpenWF Composition offers a clean API for achieving system-wide composition of layered content, such as 3D content rendered through OpenGLES hardware and video content decoded using OpenMAX‑IL hardware.

Key rendering operations are:

• Scaling (with control over filtering)
• Rotations (90-degree increments)
• Mirroring
• Alpha-blending (per-pixel and global)
• Alpha masking
• Solid background color

The scalable nature of the Composition API allows system adaptors to use acceleration hardware from low-end DSP-based implementations all the way through to high-end Graphics Processing Units. Both render-to-memory and render-to-screen (a.k.a. overlay) hardware can be used to process the content. The technology can be used on systems with unified and non-unified graphics memory.

This implementation flexibility enables the driver writer to dynamically optimize the way content gets merged onto the display according to the specific hardware platform being used. Along with performance/throughput gains, memory bandwidth usage and power consumption can be reduced. This is particularly useful for maximizing battery life during long-running use-cases such as viewing HD video with subtitles.

The Composition API supports both user-driven and autonomous rendering. User-driven rendering means that the windowing system decides when to recompose the scene. Autonomous rendering means that content can be rendered, composed and displayed without the windowing system being involved. The Composition API retains the scene structure information provided by the windowing system, meaning the composition driver has all the information it needs to recompose the scene when new content arrives. On platforms with the appropriate hardware, this can be used to allow hardware video playback directly to the display without per-frame CPU intervention. See section 5 for more details on autonomous composition.

OpenWF Display

Embedded devices with multiple displays and connections for external displays are becoming increasingly common. These displays offer various degrees of configurability and commonly require custom routines for set up and mode selection. OpenWF Display provides a consistent way to query and control the state of these displays.

Key features are:

• Dynamic discovery of external displays, e.g. cable attach detection
• Power control
• Mode-setting (resolution, refresh rate)
• Rotation and flipping control
• Pipeline management (scaling, rotation, mirroring, alpha masking, alpha blending)
• Retrieval of standardized display information (EDIDv1, EDIDv2, DisplayID)
• Content protection control, e.g. HDCP

The Display API is designed to allow the windowing system to be the focal point for display management. Support is provided for built-in displays, such as embedded LCD panels, and external displays, such as those connected by HDMI/DVI/S-Video. The cable attach sequence allows the system to tailor the display mode based on information dynamically reported from the display device that was connected. The wide range of display types is abstracted so that the windowing system can easily offer higher-level services, such as spanning of windows across displays.

OpenWF Display is a low-level abstraction allowing it to be compatible with a broad range of display control hardware without the need for costly software fallbacks. The Display API reflects common hardware constraints and behaviors such as limitations on the number of display pipelines a device can offer.

Features and Benefits

Feature	Benefit
Standardized Interface A well-defined standard abstraction for accessing composition and display control functionality.	For implementers: Reduced hardware and software design costs. Increased marketability of features. Maximized opportunity of hardware functionality being utilized. For users: Decreased time to integrate new hardware. Decreased hardware switching costs. Portability: Minimized rework of higher level software through consistent driver interfaces.
Extensive Conformance Tests Khronos provides adopters with a rich set of tests that verify compliance with the specification and provide an assessment of visual quality.	For implementers: Reduced costs associated with developing in-house testing infrastructure. For users: Provides a guarantee of quality. Tests can be reused as a form of acceptance criteria when sourcing implementations.
Conformant Sample Implementations Khronos publishes an open source Sample Implementation of both OpenWF Composition and OpenWF Display that passes the Conformance Tests and can be used to generate reference images for comparison during testing.	For implementers: Provides a concrete example to guide new implementations. Acts as a focal point for resolving questions over intended behavior. For users: Provides a quick way to try out and learn about the technology. Offers a foundation to build a Windowing System in preparation for switching to a hardware-based implementation.
Optimal processing & data paths OpenWF describes the intended visual result rather than making assumptions about how the content gets onto the screen. This enables the driver writer to choose best data path and best processing hardware for the job.	Increased battery life via the ability to use dedicated hardware with fewer gates than full GPUs. Decreased memory bandwidth usage via the use of overlay composition. Better performance via reduced processing overheads and ability to run composition in parallel with rendering.
Optimal control paths OpenWF supports streaming content directly to the display. For example, video data can be generated, composed and displayed without the need for per-frame intervention by the windowing system.	Increased battery life via increased sleep time for the CPU that runs the windowing system; especially during long-running use-cases such as video playback. Decreased latency between content rendering and display.

Memory Bandwidth

OpenWF Composition and OpenWF Display allow driver writers to take advantage of the full range of hardware available for composition. Choosing wisely between the different types of GPU composition and overlay composition enables memory bandwidth saving of over 69% on some use-cases. For full details, please see the OpenWF whitepaper.

Optimized Control Paths

Traditional integration of composition systems relies on the graphics/multimedia driver signaling the windowing system every time a new frame of content is ready to be composed onto the screen. The OpenWF APIs do not have this limitation and permit composition to occur without per-frame windowing system interaction whilst still keeping the windowing system in control of where content is displayed.

Figure 2 depicts the traditional control-path in red. This consists of the signaling that provides notification of new content, transfers ownership of buffers throughout the chain of components and may also communicate audio-visual synchronization information.

Figure 3 depicts the optimized control-path enabled by OpenWF. Using this model, the windowing system need only be active whilst changing the scene geometry. On systems that contain dedicated hardware for graphics and multimedia, the main benefit is that it offers more time for the main CPU to sleep during long-running use-cases with mostly static scene geometry, e.g. video playback with subtitles. Such systems can also benefit from the reduced latency that comes from new frame notifications bypassing the windowing system.

Deployment Possibilities

OpenWF Composition and OpenWF Display are distinct interfaces reflecting the traditional separation of the hardware. The APIs can be used together in the same system or as standalone APIs. This independence enables system builders to take a phased approach to adopting OpenWF, for example by retaining an existing proprietary API for display control whilst migrating to OpenWF Composition for composition activities, or vice versa.

Figure 4 shows a typical data path in a system in which OpenWF Composition is implemented using a simple hardware blitter whilst OpenWF Display is a direct abstraction of a single display controller. Graphical and multimedia data, commonly stored as pixel buffers in video memory, is routed by the windowing system into the compositor which passes the results to the display controller.