When you have a lot of data to display in a Control such as a ListView, you need some way of virtualizing the Nodes created and used. For example, if you have 10 million data items, you don’t want to create 10 million Nodes. So you create enough Nodes to fill the display dynamically. Because of our heritage in Swing, we know how critical this is for real apps. I got an optimization issue reported this morning on “UI Virtualization”. The report included the following link to a Silverlight blog describing what they’re doing in Silverlight 3:
http://bea.stollnitz.com/blog/?p=338

Having spent several weeks on this problem during the Spring and Summer of this year, the blog on how Silverlight is addressing the issue I found very interesting. It’s fun for me as a developer to see my counterparts in other toolkits wrestling with the same issues and coming to many of the same conclusions. In this case I think we have a stronger solution. I’m going to do a little compare and contrast not in the spirit of competition but rather just to describe what we’re doing.

In the 1.2 release of JavaFX we released the ListView (our equivalent of ListBox in WPF/Silverlight or JList in Swing). The ListView wasn’t very customizable. Each list item would only display the “toString” of the associated data item. However, the guts of the ListView were very complex to support UI Virtualization. Following JavaOne I completely rewrote the Control. The current implementation (still being finalized by some great engineers on the UI Controls team) now exposes API for allowing you to completely customize each cell while also being very efficient in terms of handling very large lists.

During the refactoring I created a (currently private) class called VirtualizedFlow which handles most of the dirty work. I found reading the Silverlight blog linked to above that WPF has a similar class, called VirtualizingStackPanel. It’s always fun to see how other people address the same problem, and a lot of fun when you find out you came to the same conclusions.

WPF has supported UI virtualization for a long time. The ListBox and ListView controls use VirtualizingStackPanel as their default panel, and VSP knows to create UI containers (ListBoxItems or ListViewItems) when new items are about to be shown in the UI, and to discard those containers when items are scrolled out of view.

This is a good start, but actually not good enough. Another feature I built into ListView from the start (1.2 release) was the notion of recycling the cells used. In the Silverlight blog, it is mentioned that .NET 3.5 SP1 also supports this:

.NET 3.5 SP1 supports the reuse of UI containers already in memory. For example, imagine that when a ListBox is loaded, 30 ListBoxItems are created to display the visible data. When the user scrolls the ListBox, instead of discarding ListBoxItems that scroll out of view and creating new ones for the data items that scroll into view, WPF reuses the existing ListBoxItems. This results in significant performance improvements compared to previous versions because it decreases the time spent initializing ListBoxItems. And since garbage collection is not instantaneous, it also reduces the number of ListBoxItems in memory at one time.

However, unlike .NET which requires you to take a separate step to enable recycling, we do it always and for free. The JavaFX API is designed such that it is a natural part of how the system works. When designing the JavaFX API, I wanted to avoid some of the odd corner cases that came out of Swing’s cell renderer concept, but keep the insane scalability. I think we’ve managed that. It is a little more complicated than the simple naive approach of creating a new Cell for every item, but not by much.

The basic concept in JavaFX revolves around the concept of a Cell (which is a Node). Cells are used to display data items in a ListView (and in the future, a TreeView and TableView and TreeTableView, etc). Each Cell represents a data item, though which data item is represented may change over time. For example, suppose I had the following ListView:

ListView {
    items: [1..1000]
}

This ListView will display the numbers 1 through 1000. Initially, enough Cells will be created to display each visible row. So you may have a Cell which will represent data item “1”, and another which represents data item “2”, and so forth maybe up to “10”. If the user then scrolls, then we will attempt to reuse Cells. So the Cell which used to represent “1” now represents “33”.

When you look at the Cell API this is apparent:

public class Cell extends CustomNode, Resizable {
    /**
     * The data value associated with this Cell. This value is set by the
     * multiviewed Control when the Cell is created or updated. This represents
     * the raw data value.
     */
    public var item:Object;

    /**
     * Indicates whether or not this cell has been selected.
     */
    public var selected:Boolean;
    
    /**
     * Called by the system whenever the state of the Cell has been updated.
     */
    public var onUpdate:function():Void;

    /**
     * A container for the nodes that represent the content of this cell. Since
     * Cells must be resized to fit within some space, and since their preferred
     * sizes are important for determining the size of a cell, a Container of
     * some kind is required as the root of the content for the cell.
     */
    public var node:Node;
    
    /**
     * The container for the nodes used to represent the background of the cell.
     * If this is left empty, then the Skin for the Control is responsible for
     * creating a default background, if it so chooses. In this way, custom
     * cells can be created that define both the content and background, or that
     * define only the content with the background being provided by the skin.
     */
    public var background:Node;
}

As you can see in this API, each Cell has an item and the item can change over time. The Cell also has a node which represents the cell’s body, as well as a background which is typically used for rendering selection and so forth and if left null will be supplied by the ListView skin implementation. So to create a custom Cell, all you would need to do is provide the node, and wire it up (bind it) to the item. The system then will reuse Cells, simply updating the item over time as the Cell represents something different, and the bindings will then update the node. There is also a procedural approach where we invoke an onUpdate function variable, which allows the Cell author to respond procedurally if it makes more sense (for example, when working with FXD content exported from the JavaFX Production Suite this might make sense).

The last part of the puzzle is some way for the system to create your custom Cell whenever it needs a new one (which it pools up itself). This is actually very trivially done by exposing a function variable callback on ListView which allows you to create and return a new Cell. In this way you can create custom Cells for representing data richly in your JavaFX apps.

ListView {
    items: [1..1000]
    cellFactory: function():ListCell {
        ListCell {
            node: Panel { ... }
        }
    }
}

Since cell factories are simple functions, it is also easy to create shared libraries of static functions which return your own implementations and reuse them across your application. You can also subclass ListCell with your own custom implementations so as to reuse them across your application.

There is one last thing that JavaFX does that I think is quite unique (from the blog it doesn’t look like Silverlight supports it, and I don’t think Flex does, however Android might I’m not sure). First I’ll quote from the Silverlight blog as to the problem:

Currently WPF supports UI virtualization only when scrolling by item. Pixel-based scrolling is also called “physical scrolling” and item-based scrolling is also called “logical scrolling”.
[…]
I’m often asked if there is a way to work around this limitation. Well, anything is possible, but there is no *easy* workaround. You would have to re-implement significant portions of the current virtualization logic to combine pixel-based scrolling with UI virtualization. You would also have to solve some interesting problems that come with it. For example, how do you calculate the size of the thumb when the item containers have different heights? (Remember that you don’t know the height of the virtualized containers – you only know the height of the containers that are currently displayed.) You could assume an average based on the heights you do know, or you could keep a list with the item heights as items are brought into memory (which would increase accuracy of the thumb size as the user interacts with the control). You could also decide that pixel-based scrolling only works with items that are of the same height – this would simplify the solution. So, yes, you could come up with a solution to work around this limitation, but it’s not trivial.

We have the exact same problem space — we support non-homogenous row heights, potentially incredibly large numbers of items, and scrolling. This presents some really interesting problems. We have a solution to this problem which for typical ListView’s combines pixel scrolling with position (or item) scrolling in a way that the user cannot detect that we’re doing anything special behind the scenes. You can “pan” the list by using the mouse and it looks like pixel scrolling, or move by items using the keyboard and it looks like item scrolling or drag the scrollbar and you won’t know if you are pixel scrolling or item scrolling (except in some very, very edge cases) or you can click the “up” and “down” arrows on the scrollbar and get pixel scrolling. We’re still working on the thumb size problem which I think will turn out to be some heuristic that adjusts the thumb size as it finds out more information on the size of the list (typically for large lists you can assume it needs to be the min size, while on small lists you might just create enough cells to get a reasonably accurate thumb. It only goes awry if you have whacked out differences in cell heights).

So since I want to close out the optimization issue report, I decided to write this blog detailing what we’re planning on doing. Each of our multi-valued controls (list, tree, table, etc) will use the same base Cell API (unlike TreeCellRenderer, TableCellRenderer, ListCellRenderer which were all different in some annoying ways). They’ll all support UI virtualization. Initially we won’t support “model virtualization” (ala TableModel, ListModel, TreeModel) though that will be coming in a future release.