Justification

In Xen 3.0.0, a user can only interact with through a single serial console. Average users are not used to serial consoles and have trouble adjusting to this paradigm. There are many potential solutions including developing a special version of screen to show console history. Screen is a pretty old and crufty code base though so that's not very appealing.

Another solution would be to implement a virtual framebuffer and let Linux render the console to the framebuffer. This approach gives the usability experience that most users would expect. Another benefit is that it makes it easier for X to Just Work (as opposed to having to rely on Xvnc). A virtual framebuffer introduces some unique problems though such as supporting copy/paste and remote viewing.

This document attempts to describe an approach for implementing a virtual framebuffer for Xen. It should be considered to be a dynamic document that is subject to change.

Use Cases

Jane creates a paravirtual domain with a Xend configuration file. Without any special parameters, xm console will (as it does today) attach to a console showing boot messages and an initial getty. She will always access the domain remotely and only ever wants SSH-like access. NB: xend will, by default, pass console=ttyS0 to the guest.

Tim is a RedHat certified engineer and wants to use the RedHat server configuration tools in his domain to configure his webserver. To achieve this, he adds sdl=1 to his domain's configuration file. This will allow Tim to have an X session with a max mode of 1024x768@24. Tim also follows the recommended post-configuration for PV domains document to setup X. This involves setting up X to use the fbdev driver, a PS/2 mouse, and the appropriate extensions enabled/disabled. When Tim starts the domain with xm create, a GUI opens up (if his DISPLAY variable is set properly). Tim can close the GUI and come back to it later with the xm display command.

John is a desktop developer and has a box running Xen with each domain running various distributions. The box is located at a co-location facility so remote display is the only option. He follows the same procedure as Tim but also adds the vnc=1 line to each of his domain's configuration files. He can then connect to each domain by using the domain ID as the VNC display number. Alternatively, he could specify a VNC port number with a line like vnc=5901.

Implementation Details

The Frame Buffer

During domain building, a page and an unbound event channel are allocated for use by the frame buffer device and stored in the shared_info_t structure under the fields fbdev_mfn and fbdev_evtchn. The page is initially zeroed. When the frame buffer driver loads, it fills out the information about the frame buffer it is going to create. This includes the width, height, row stride, and depth. Additionally, it fills out the length of the frame buffer memory (which is guarenteed to be at least height * row stride). It then allocates necessary amount of memory plus a page-aligned array of unsigned longs large enough to contain all of the MFNs for the allocated memory. The allocated memory cannot be larger than 8MB. The frame buffer driver then fills out this array with all of the MFNs of the frame buffer memory and records the MFNs of the array in the page pointed to by fbdev_mfn. After all of this information has been filled out, the initialized field is set in the fbdev_mfn page which lets the remote end know that it can begin displaying the frame buffer data. No notification occurs on this event so the remote end must poll this value.

The information stored in the page pointed to by fbdev_mfn is guarenteed to remain under 1k. Starting at the 1k offset in fbdev_mfn is a ring queue of input events. There are currently no input events defined for the frame buffer driver although they are guarenteed to all be 40 bytes in length and start with an 1 byte type identifier. Unknown events can safely be ignored. The input event ring queue is at max 1k.

Following the input event ring queue is the output events ring queue which again consists out output events that are of 40 byte lengths. There are no output events currently defined by future versions will include things like mouse movements and high-level drawing information. Output events are identified by a single type identifier at the beginning of the event. Output events cannot be ignored but will only be delivered if requested. Output events are requested by a special input event that identifies the set of supported output events. The output ring is at max 2k.

Whenever a message is delivered to or received from one of the queues, an event channel notification is sent on fbdev_evtchn.

The Keyboard/Mouse

During domain building, a page and an unbound event channel are allocated for use by the keyboard/mouse device (referred to now simply as the keyboard device) and stored in the shared_info_t structure under the fields kbd_mfn and kbd_evtchn. The page is initially zeroed. When the driver loads, it initializes itself and sets the initialized field in kbd_mfn to signify that it is ready to accept events. At the 1k offset in kbd_mfn, there is an input event ring queue that is 2k and takes 40 byte input events that all contain a type byte as the first field. Currently, there are three event types for pointer motion, pointer button events, and keyboard events. The pointer motion event contains two members for relative x and y motion. The pointer button event contains two fields also. The first field is a boolean specifying whether the button has been pressed or released (1 for pressed; 0 otherwise). The second field is an integer identifying the button that caused the event (0 for the right mouse button, 1 for the middle mouse button, 2 for the left mouse button). The keyboard event signifies that a key has been pressed and consists of a field identifying whether the key is depressed (1 if depressed; 0 otherwise), and a keycode identifying the key. Currently, the values in linux/input.h are used to represent keycodes.

Unknown buttons or keycodes are ignored as are unknown messages. There is a 1k output queue that is currently unused. Output events will only be sent when requested. At a bare minimum, the keyboard/mouse driver should support a 2-button mouse and a US101PC keyboard.

An event channel notification is sent on kbd_evtchn whenever an input event is received or an output event is sent.

Acceleration

Remote display is an important use-case. A common technique for reducing the bandwidth of remote display protocols is to send higher level operations instead of raw framebuffer updates. Three common high level operations are:

Cursor Update - an image corresponding to the cursor is sent and the client simply uses that as the local cursor and sends x,y updates. This has a huge effect on cursor smoothness even on very slow conncetions.
Copy Region - a copy event corresponds to a screen-to-screen copy. This operations are very common for user actions like scrolling and window dragging. These tend to happen to large areas of the screen too so they normally would occupy a lot of bandwidth.
Fill - it is often considerable more efficient to represent pixel data as a solid color region fill. It's rather difficult to determine optimal squares though if trying to determine where the fills are after the fact. If you get that information from the X-server though, it is very simple hence the importance of this acceleration.
Region Update - an important optimization for something like VNC is to only send regions of the screen that have actually been modified instead of sending the entire frame buffer. The Linux frame buffer interface does not have a mechanism for region update. The frame buffer console code only uses high level ops so we can translate this into appropriate updates. This gives us a few options for the X server. We could only use the high level interface or we could add a special ioctl for marking regions as damaged.

All of these operations are supported by the Linux framebuffer interface. The X fbdev driver does not support these operations though which means that we'll have to either enhance fbdev or write a Xen-specific driver. This is going to require engaging the Xorg community.

For cursor acceleration, it's sufficient to just have the X driver enable hardware cursor acceleration. It's important that it also support ARGB cursor acceleration as normal XCursors are almost never used on modern desktops. Of course, not all platforms will support alpha blended cursors so we should make sure we gracefully fallback. However, it would be really neat to extend VNC to have an AlphaBlendedCursor encoding that could be enabled on platforms that support it.

Copy Region and Fill both can be handed by supporting implementing DGA and XAA support in the driver. This is pretty straight forward.

Tasks

Stage 1

Improve the keyboard support - currently, we do not support all of the keys on a standard 101 at keyboard. I'm not entirely clear on how keyboards differ based on i8n. I think if we just support everything that GDK and Linux does, and have the guest keyboard driver just be an uber-keyboard capable of sending out any key i8n will Just Work. Status: Done. On a PC keyboard everything works, whatever the keyboard layout.
Write-Fault Updates - GerdHoffman has written a VFB for UML that handles userspace writes by setting a timer that unmaps paged in pages and issues an update for that entire scanline. Status: Done.
VNC server - this will be the most basic GUI interface to the frame buffer and will use libvncserver for now. Status: Done.
Xend Plumbing - add the necessary guts to Xend so that this all works as described above.
Save/Restore - we need to implement save/restore handlers so that we can canonicalize the framebuffer

Stage 2

Dynamic Modes: the driver should be initialized with a maximum mode (default 1024x768@24). The maximum mode controls the size of the VGA memory. Any other resolution is valid provided it doesn't use more than the size of the VGA memory. The default dynamic mode should be 700x400 which is size of the standard 80x25 character screen (assuming 8x16 fonts). Dynamic modes will then allow the X server to choose a higher resolution.
Implement a smarter cursor - before we support accelerated cursors, we should be able to at least inform the host of what the current Guest cursor x,y positions are. The host can use this information to intelligently enable/disable grab (when the user moves offscreen for instance). This is a big usuability win.
Implement XAA support - if we provide XAA support, we can use high level operations for screen-to-screen copies and solid fills. This is really what we need VNC to work well.

Stage 3

Implement an actual GUI - this is really a separate effort but closely related. A graphical interface that uses the XenDisplay widget and the xm bindings would be extremely cool though.

Test Plan

For stage one, the following areas need to be tested:

Verification that the framebuffer and keyboard rings buffer correctly and are reliable.
Save/Restore/Migrate results in proper canonicalization and restoration of MFN pointers
The entire region of framebuffer memory can be mapped and accessed correctly
The display format exposed in the guest by the FB ioctls is consistent with the information exposed to dom0.
The user can correctly specify different display modes
A sequence of keystrokes results in the correct input occuring on the current tty

The stage two/three test plan will be written once those two stages are more well-defined.

Future Thoughts

Implement a CGA mode (the recent CGA/curses patch to QEMU is really neat)

How to Get

First, clone the Xen repository and apply the bundle from http://www.cs.utexas.edu/users/aliguori/xen-vfb-20060124.bundle

hg clone http://xenbits.xensource.com/xen-unstable.hg xen-vfb-unstable.hg
cd xen-vfb-unstable.hg
wget http://www.cs.utexas.edu/users/aliguori/vfb-20060124.bundle
hg unbundle vfb-20060124.bundle

Hopefully, you won't have any merge conflicts. Build as you would a normal xen tree. Then clone my vnc server code at http://hg.codemonkey.ws/vncfb and build. You'll need libvncserver.

After launching a domain (passing something like "xencons=ttyS0 video=xenfb" as the argument to extra in your config file), you can run the vncfb server with:

# hg clone http://hg.codemonkey.ws/vncfb vncfb
# cd vncfb
# make XENDIR=/path/to/xen-vfb-unstable.hg
# ./vncfb $(xm domid ExampleDomain)

Then connect to the vnc server with:

$ vncviewer localhost:1

NB: libvncserver's support for the Tight encoding is currently broken. If you use tightvnc, you'll want to disable the tight encoding with a line like:

$ xtightvncviewer -encoding "hextile copy raw" localhost:1

Using X

You can configure the domain to have a working X server by changing the Device, Monitor, Screen, InputDevice, and ServerLayout sections in your /etc/X11/xorg.conf to:

Section "Device"
        Identifier      "Para virtual Framebuffer"
        Driver          "fbdev"
        Option          "fbdev"                 "/dev/fb0"
EndSection

Section "Monitor"
        Identifier      "Generic Monitor"
EndSection

Section "Screen"
        Identifier      "Default Screen"
        Device          "Para virtual Framebuffer"
        Monitor         "Generic Monitor"
EndSection

Section "InputDevice"
        Identifier "Xen Mouse"
        Driver  "mouse"
        Option  "Protocol"      "PS/2"
        Option  "Device"        "/dev/input/mouse0"
EndSection


Section "ServerLayout"
        Identifier      "Default Layout"
        Screen          "Default Screen"
        InputDevice     "Xen Mouse"
EndSection

NB: Right now dynamic modes are not supported so you'll be stuck in a pretty low-res X session. The next thing I'm working on is the ability to resize the framebuffer although it's going to require reconnecting with VNC for most clients (unless they support the DesktopResize pseudo-encoding).