This book takes a practical approach in providing real world solutions to real world problems in 3D user interface (UI) development. Although its emphasis is on next-generation business needs, such as mass customization and online sales, much of what this book offers has broad applicability to 3D user interfaces in other pursuits such as scientific visualization and gaming. Applications and their use of 3D are approached realistically. The book assumes an everyday user without a lot of time to learn yet another application much less a complicated one, and an everyday computer system without exotic devices like head mounted displays and data gloves. From this perspective, Java 3D is introduced as a platform for 3D application development, with a series of examples demonstrating how to implement advanced 3D UI techniques. One could say that this book presents a roadmap from Java 3D to "Swing 3D".

Patterns, Techniques, and Framework

The 3D techniques introduced early in the book are general in nature and not specific to a particular software language or 3D graphics platform. They try to take a fresh approach to the problem of highly interactive 3D user interfaces. Some build on trends that have developed in non-3D and web-based applications, like drag-and-drop and web page like navigation. Others try to impart a natural feel to 3D user interfaces, through cues, suggestive dynamics, and copious visual feedback. Many rely on compute intensive techniques such as fine-grained mouse over and collision detection that until recently put them beyond the reach of common personal computers.

The software framework presented later in the book, implemented in Java 3D, demonstrates many of the 3D UI techniques. It offers concrete examples of how to implement 3D user interfaces that form the basis of intuitive easy to use applications. More importantly, it provides readers with the means for incorporating these techniques into their own applications.

Java3D

To make the examples concrete and the techniques immediately usable a 3D platform had to be chosen. The clear choice was Java 3D. Java 3D is an up and coming extension to Java that had its first full release in December of 1998, but was available long before that in a series of rather complete and solid beta versions. The same widespread appeal for using Java also applies to Java 3D, not the least of which is the promise of platform independence.

Java3D offers tight integration with the rest of the Java environment, which is useful for developing real 3D applications, not just 3D toys to amaze your friends. It was also designed with the intent of speed and flexibility. It should have broad appeal and an air of familiarity to many because it evolved out of the better parts of Open Inventor, OpenGL, DirectX, and VRML. The expected long term benefits of Java 3D more than make up for its short-term growing pains, which in the broader picture are surprisingly few.

Although not intended as a full-blown tutorial the book includes a good introduction to programming in Java3D. Emphasis is on the more practical aspects of using Java 3D as a medium for implementing the user interface techniques described in this book. To help the reader get started a set of Java3D classes are provided that serve as an introductory application framework and the foundation for the book's 3D UI examples.

Real Solutions for Real Problems

Many of the 3D techniques described in this book were identified by the author while working on e-commerce and other end-user applications. A common goal in this work was to develop 3D applications for use by the common man and woman rather than by trained specialists. The need for this work grew out of the realization that traditional windows-based CAD-inspired approaches to 3D applications were not readily adaptable for use by the casual user. The advent of the web and its democratic nature further highlighted the shortcomings of conventional 3D user interface approaches such as these: Users were not going to take training courses, try to use a set of controls that would feel at home in the cockpit of a jumbo jet, or even read the manual just to use a web application once every few months. The web, however, also offered a way out of this dilema.

The appeal of web applications is that they are easy to navigate, visually compelling, and generally take a "learn as you go" approach to user training, with lots of dynamic visual feedback. Exploration of the application is fostered by the user's ability to "back up" or "go home" if they find themselves lost, something that is often impossible to do in window-based applications, 3D or otherwise. Applying such concepts as these to 3D applications will hopefully provide the necessary paradigm shift to make 3D viable and user-friendly for next-generation applications.

Some developers have already recognized the benefits of this approach. Games and child education programs were early adopters of visually compelling, easy to use 3D user interfaces. This book tries to apply this philosophy to new business-oriented applications that might not consider 3D as beneficial, and to traditional 3D applications that are seeking broader appeal through more intuitive user interfaces. (The resulting loss of revenue from seminars and training courses, however, is a topic for a different book.)

A Fresh(er) Approach to 3D User Interfaces

The concepts and techniques described in this book try to take a fresh - or at least a fresher - approach to designing 3D user interfaces. Many take advantage of the immense performance available in recent desktop computer systems to impart a more "intuitive" feel to the user interface. One of the major failings of 3D applications has been the lack of intuitive navigation that people use in exploring the real world, and the lack of intuitive feel that people have in manipulating real objects. Devices developed for Virtual Reality (VR) such as force feedback, data gloves, head-mounted displays, and 3D mice address some of these shortcomings, but are not widely available in the "real world". Their general lack of performance and robustness is also an issue.

This book offers techniques that impart some of the natural feel we have in the real world without resorting to exotic interface devices. They rely on "cues" and sensory substitution to suggest tactile feedback. Other techniques provide the user with dynamic in-scene guidance for how to make things happen. These techniques take advantage of compute intensive operations - made possible by the current state of personal computer hardware - such as proximity detection, dynamics modeling, and a lavish use of mouse-over visual feedback.

As in life, too much of a good thing can be bad for you. Too much realism can get in the way of the task at hand, such as having to use the virtual stairs in order to move virtual furniture from one floor of a virtual building to another in a space planning application. The techniques and guidelines described here try to strike a balance between the sometimes competing needs of interactive realism and user effectiveness.

A premise of this book is that a good 3D user interface should make the 3D-ness of the interface fade into the application woodwork while making the user's job more natural and easy. In other words, it takes a lot of hard work on the part of the developer and the computer to make 3D just another humble part of a well-integrated application. The fact that an application uses 3D instead of 2D, video, or the theory of relativity is immaterial to the user so long as they can get their job done with minimal effort and frustration. This is in keeping with the notion that 3D is not an end unto itself. 3D is merely a technology to be used when the benefits are clear and the costs are not prohibitive, to the developer or to the user.

Not Like Other GUI Books

Books on user interfaces tend to be rather broad and general. This can be good but only gets you so far when you are asked by marketing to add a little "3D" to the new product with the beta release only weeks away. The bible on computer graphics by Foley, van Dam, et al. is a bit closer to the mark. It includes a discussion of user interfaces in the specific context of computer graphics; but it lacks specifics about how to design a 3D user interface for a real application.

For better or for worse this book takes a specific approach to designing and implementing user interfaces for a specific but important class of interactive problems, namely direct 3D manipulation. The concrete examples and software framework offered by this book should make the job of incorporating, or at least prototyping, 3D in visually oriented applications easier, even for designers who are relatively new to 3D.

To give the reader a clearer sense of where this book is coming from here are a few words about what was intentionally left out of the book.

This book does not consider the unique aspects of developing 3D applications for a 10-year-old 286 PC running Windows 3.1 or a high-end SGI Onyx in a VR game palace. Instead, it focuses on the current generation of POCS -- Plain Old Computer System -- characterized outwardly by a 2D display, a mouse, a keyboard, and maybe sound output. In terms of performance, however, the POCS is not quite as ordinary as its name states or its humble countenance might lead you to believe. The generation of POCS that is now emerging is sporting greatly accelerated graphics and clock speeds surpassing 500MHz. Such computers are starting to find their way into offices and homes today, and should become ubiquitous in the next few years (perhaps even being reborn in the form of computer appliances).

This book does not plead the case for using 3D. If you think 3D makes sense for your problem then this book provides ideas and hopefully some of the means to help make it happen. For 3D to be successful in applications it needs to fade into the application woodwork, becoming just as unpretentious but essential as the mouse. As applications grow in sophistication to keep pace with new business demands 3D can offer a more natural and intuitive user interface. The whole issue of whether or not 3D is "worth it" will become moot.

This book does not directly address the very large and much written about fields of data visualization and games. Instead, it offers techniques and software examples that could be applied to such applications. The visualization techniques that are provided in the book play a specific supporting role to the central theme of 3D user interfaces. They are not meant to serve as the core of a visualization package. Similarly, this book will not pretend to be about the ever-popular field of game development. It does, however, share with games the common goal of providing intuitive and easy to navigate user interfaces, so there should be plenty of opportunities for cross pollination.

Finally and perhaps most controversially this book does not try to define what is a "good" UI. Such a determination is highly subjective and 3D user interfaces for the masses is still too young a field for such absolutes. (In fact some might claim that such absolutes are ill-advised even for 2D user interfaces.) Instead, this book suggests techniques, describes their advantages, and leaves it to the developer to decide what is best for his or her targeted audience in terms of UI usability and intuitiveness. As such, the book takes a rather utilitarian approach in its examples, providing enough of a UI to demonstrate a technique. It is left to the reader and to the future of 3D to determine the define 3D UI perfection.

This book can be used "top-down", as a reference: It provides a hierarchical structure for classifying 3D user interface problems, designing solutions, and implementing them in a Java3D framework. The classification and layering of the problem and solution spaces can be used as 3D design "patterns", but in a much broader and less formal sense than the popular patterns used for object-oriented software design.

This book can be used "bottom-up", as a toolkit: It provides an introduction to Java3D programming and a cross-reference between the 3D techniques and those aspects of Java 3D that support them. It includes application utilities, a framework of Java3D classes, and examples of how to glue them together into a useful user interface. Readers can "wire up" their own widgets, extend the provided object classes, or even borrow and modify the source code to try out their own ideas.

This book can be used "from the middle-out", as a tutorial: Its practical introduction to Java3D programming should prove invaluable to readers familiar with Java and 3D but who want a jump start on using Java 3D in real applications. The framework make it easy to experiment with Java 3D and the source code provides plenty of examples of how to move beyond the basics and into the realm of practical use. References throughout the book lead the user between conceptual 3D techniques and their concrete UI implementation.

Parts I and II are conceptual in nature. They assume the reader is familiar with 3D concepts but does not discuss programming issues and are not specific to implementation with any particular language, platform, or 3D API. Parts III and IV are concrete in nature. They include an introduction to Java 3D with emphasis on development of 3D user interfaces and some of the more practical aspects of interactive application development. It uses Java and Java3D as the basis for a framework of classes that demonstrate the concepts introduced in the conceptual portion of the book.

Part I: 3D, The Next Generation

Part I (Chapters 1 through 3) establishes the book's view on the future of 3D applications and discusses the essential role that the user interface plays. Applications are the top level of design abstraction presented in the book. Here applications are classified by purpose and functionality, and patterns are noted in their use of user interface techniques. To establish a proper frame of reference for discussion, common 3D terminology and concepts are defined and discussed.

Part II: 3D User Interface Techniques

Part II (Chapters 4 through 9) describes patterns and techniques for developing 3D user interfaces. Techniques are the middle level of design abstraction presented in the book. Concepts, methods, rules of thumb, controls, and widgets that address common user interface needs in 3D applications are classified and discussed. References are made to software examples in Part IV of the book that demonstrate particular techniques.

Part III: Java 3D User Interface Essentials

Part III (Chapters 10 through 14) introduces the Java3D API. It describes those features useful for developing 3D user interfaces and, more generally, 3D applications. It identifies the Java 3D classes that support the various user interface coordinate spaces, provides guidance on the proper care and feeding of scene graphs, and discusses the foundation that Java3D provides for the user interface framework in Part IV.

Part IV: Java 3D Framework and Examples

Part IV (Chapters 15 through 24) describes a lightweight Java/Java3D framework demonstrating many of the 3D user interface techniques described in Part II. The software framework is the bottom level of design abstraction presented in the book. Its architecture and examples parallel the organization of techniques in Part II. Code samples and diagrams describe how the framework and UI pieces fit together, and examples give the reader an opportunity to "kick the tires".

Software and Feedback

This section lists the software provided with the book and discusses the practical aspects of installing and using it. Also, the publisher describes how to obtain the software from their web site, and discusses their program that allows readers to contact the author to provide feedback and to ask questions concerning the book and its software.