Developing Programs
for
Digital Television

Overview

This document presents two digital television programs designed to demonstrate the capabilities of this new medium. The first, "À la Carte" is a how-to cooking show taking advantage of the capability to compress multiple video streams (multicast) as well as data (datacast) into the bandwidth for one program. "À la Carte" demonstrates our techniques for synchronizing the video streams and providing a new viewer navigation paradigm. "Multiple Dimensions" uses these techniques to extend the current concept of music videos into a virtual nightclub providing a new viewer paradigm for navigation of multiple video streams. Both shows use Java as the framework for the digital television program.

Introduction

Computer oriented activities such as browsing the web, processing e-mail, and electronic transactions are not going to be the reason viewers switch to digital TV. We believe television is used as a portal for viewing entertainment. Television certainly presents quite a bit of information but it is not a tool for accessing and processing information. In addition, while the technology makes it possible to create sophisticated advertising with buttons to press allowing the viewer to make impulsive decisions to buy a product, this will not be the reason viewers embrace digital TV either. People will buy digital television only when there is compelling content which can not be viewed through the current television paradigm.

Until there are viewers watching digital television programming, advertisers will be reluctant to make a significant commitment to the new technology. However, once a viewer community is established, advertisers will begin to invest heavily into even more sophisticated methods to reach the new audiences.

We are creating two demonstration applications as examples of our digital TV paradigm. "Multiple Dimensions" presents a model for viewing live entertainment expanding the concept of music videos. Our edu-tainment (educational entertainment) program, "À la Carte," applies our techniques to a "how-to" show. Other shows including sporting events and drama series can also be enhanced with our digital television programming techniques.

With these two programs, we hope to stimulate creativity among producers of current television programs. Once they see the capabilities they will begin to visualize new ideas leading to a dynamic new television viewing experience. The longer it takes to expose television producers to the capabilities of the new technologies, the longer it will be before the digital television revolution begins.

The Eureka! team has been involved with digital television since mid-1995 when we began working with Hyundai's Digital Video Systems division on their digital television settop box project. We have also worked on several projects with OpenTV including development of a Java API to access their settop platform.

Dimension 7 has been involved with digital media since its inception in 1994 specializing in digital media creation with video, lasers, and computer imaging techniques. These media environments have been designed from their outset to enable a multi-camera (multicast) viewing experience for digital television in addition to experimenting with new methods of advertising and electronic commerce.

The rest of this document presents our ideas for these two digital television programs. We begin with a presentation of the components of a digital television program. This is followed by detailed descriptions of the two programs we are developing.

Digital Television Programs

Digital television technology has been around for a long time. Unfortunately, the term digital television means different things to different people. Most consumers, and possibly most producers, associate the following features with digital TV:

• a clearer picture with better sound,
• the ability to display electronic program guides on the television screen allowing the viewer to select programs from an intuitive interface with the opportunity for in-depth programming descriptions - rather than simply selecting a channel number,
• the ability to overlay graphical images on the television video allowing the viewer to "click" on them in much the same way web browsing is done using a computer,
• check email or browse the internet,
• the ability to restrict access to channels at the broadcast end (encrypted pay channels), and
• the ability to restrict access at the receiving end (e.g. parental control).

All these features and many more are available with digital TV technology. However, its the availability of both synchronized and parallel stream multicast broadcasting that will unleash the true potential of the new medium.

Television is a viewing activity rather than an information processing activity. What are commonly presented as new digital TV features either change this process from a viewing experience (e.g. checking email or browsing the web) or are minor improvements over today's viewing experience (e.g. clearer picture and sound). Multicast broadcasting allows a totally new viewing experience; one we believe will be compelling enough to encourage viewers to adopt the new digital television technology.

The technology of television programs can be observed from two perspectives: the broadcaster perspective and the viewer perspective. A standard analog television program is broadcast as a single audio/visual (A/V) stream. There is a single linear flow of information from the beginning to the end. Digital television technology makes it possible to change the way a program is broadcast. Instead of a single linear stream of information, a digital television program can consist of multiple A/V streams broadcast concurrently (multicast). In addition, it is also possible to broadcast data streams (datacast) along with the A/V streams.

An example of parallel stream viewing

The easiest way to visualize this is to consider a live sporting event such as a PGA golf tournament. Some golf fans attending a PGA tournament prefer to walk around the course with their favorite player (e.g. Tiger Woods, or Greg Norman). Others prefer to watch all the players from a specific place on the course (e.g. the 18th green, or the 12th tee). Still others might choose to walk around the course with one player, then switch to another player, and then decide to stay in a particular spot for a while.

A standard television broadcast of a golf tournament misses this interactive aspect of the action. The A/V stream the viewers see is constantly switched among all the players and shows them only when they are actually swinging the club. It misses the interaction the players have with people in the gallery.

A digital television production can present the tournament almost as if the viewer is actually there. The A/V streams from all the cameras may be broadcast simultaneously and the viewer may choose if they want to watch the view from a particular camera, or continually follow a particular player regardless of that players position on the course. The data streams that flow along with the A/V streams could contain various statistics associated with the tournament (e.g. the leader board). The viewer would be able to browse through this information at any time during the program.

DTV Program Framework

A digital television program may consist of multiple audio and video streams as well as a data stream holding information related to the program. There may also be an associated web site that can be presented on the television or accessed using standard computer browser software. Figure 1 shows this framework and demonstrates the temporal relationships between the various components.

For a live event such as the PGA tournament discussed above, the A/V streams are always synchronized. However for produced shows this may not be the case. These programs may consist of many short segments with "synchronization" points making it possible to switch between streams within the context of the program. We call this a synchronized multicast program.

The program represented in the figure above contains four A/V streams and one data stream. The first segment in the main program track provides an introduction for the program. At the end of this segment, the viewer may switch to any of the three alternative streams or remain with the main program track. Streams three and four each consists of two short segments with a synchronization point across both tracks at the end of the first short segment. At the end of the second short segment there is a synchronization point across all four tracks. At any time during the program, the viewer may switch to the information stream and browse the information being broadcast with the program. This information stream may also include graphical images and data for presentation as an overlay synchronized with the video. Finally, the program also includes an integrated internet web site for use with an internet web browser and may be accessed from the digital television or a computer.

The information stream and the integrated web site may be used for many purposes including electronic commerce. The information stream is only active for as long as it is broadcast (i.e. the duration of the program) while the web site is much more static. The web site could include a chat room where viewers of the program could gather to discuss the episode after the completion of the broadcast.

By using technology derived from the technique currently being used to track the hockey puck in a hockey game broadcast, it will be possible to make video hot links on the screen. For example, the jersey of a football player could be set up as a hot-link to information or a web site for the player. In a fashion show broadcast, the wardrobes of the models could be hot links to an electronic commerce site allowing the viewer to order components of the wardrobe.

Demonstration Show Content

We are producing two digital television shows. "Multiple Dimensions" will demonstrate the power of parallel stream broadcasting for live events and "À la Carte" will demonstrate the flexibility of synchronized multicast broadcasting and its applicability to educational programs.

"Multiple Dimensions"

"Multiple Dimensions" draws on television programming we are all familiar with through shows like American Bandstand and Soul Train. However, the digital age equivalent, "Multiple Dimensions," enables digital TV viewers to navigate the dance club environment in a virtual manner.

Presentation and Navigation

The initial production of "Multiple Dimensions" will use four cameras:

• Camera 1- Establishing, wide-angle shot of the dance floor
• Camera 2 -Roving Camera, capable of moving to backstage and other inaccessible areas on the main dance floor, as well as to set-up shots.
• Camera 3 -Performance camera, the performer can be a DJ, VJ(video jockey), musician, or featured dancer.
• Camera 4 This camera can either be a live feed of the video mix being projected on the walls of the dance environment, or a direct feed from the video mixer. Note that the video mix itself which is being created in real-time by the VJ may include images captured from additional cameras in the studio. This would be the default camera for the large screen.

The television screen will be divided into two main areas. On the left side of the screen are three 'thumbnail' size video windows. To the right of these thumbnail views is the large video window occupying the remaining area of the screen. Alternatively, the viewer can select to view a stream with no thumbnail views. Note that the broadcast option with thumbnails must be composed at the head end of the broadcast - and can not rely on multiple decoders on the receiving end. In our example, this would result in 8 broadcast streams: 4 for an unrestricted view from each camera and 4 additional streams showing a composite of each camera view with the additional streams shown as thumbnails.

The viewer has the ability to move the cursor on the universal remote to 'click' on one of the three thumbnail windows and by so doing allow the image which is selected to replace the default (large screen) viewing area. Essentially, the application provides the viewer with their choice of viewpoint from which to interact with the program material. Instead of having the technical director of the show make the decision of what camera feed is sent for broadcast, the viewer can choose and by so doing becomes more of a participant in the action.

The action in this case is the dance club. "Multiple Dimensions" is built around the excitement of a dance club atmosphere where mating rituals, dance, fashion, music, and art all converge. The most important element in execution of the dance environment is that a compelling experience is created for the real space participants in order that the virtual participants can experience the next best thing to being there.

There are variations on how to most effectively use the camera viewpoints. For instance,

• another window could convey data which would be invoked by a mouse click on a "hotlink" imbedded in a broadcast stream.
• another window could be a chat area for virtual participants to discuss the activity and people in the dance environment/club.
• in the future, with even faster processors and greater functionality in the settop there could be a downstream keyer on the box itself which would allow the viewer to do overlays or composites and as such play video jockey him or herself.
• And even at some point in the future, with the development of additional technology, one window could be a computer graphics character that the viewer/user could have control over to simulate dancing or other activity such as game play that incorporates the live action.

Electronic Commerce

E-commerce is an important component to "Multiple Dimensions". The set-up shot described as one of the variable shots for the roving camera (camera #2) may consist of actors/models that are placed in the midst of the dance/club environment who are wearing garments/clothing by manufacturers who choose to be sponsors of the show. The clothing itself would be identifiable as being 'hot' or 'clickable' in order for the viewer to be taken to the respective manufacturer's online storefront. Needless to say, products would include not only clothing but any item the actors/models might be wearing (jewelry, sunglasses,) or using (Cell phones, PDA's).

"À la Carte"

The host of the show is Roger Romano. Roger received his AOS in Culinary Arts from the California Culinary Academy in San Francisco in 1997. He was sous chef for the CCAs Bay-TV cable show from March through July of 1996 and hosted the show on May 22, 1996. As co-owner of By Invitation, an exclusive catering service, he caters fine meals for private parties ranging in size from less than 10 to more than 250 people. Roger also developed and teaches a "Healthy Cooking" series of classes offered through many businesses, fitness centers and other organizations throughout the Silicon Valley and Monterey Bay areas. His culinary career began by serving as a line-cook for several different restaurants.

What makes Roger the ideal host for "À la Carte" is the combination of his culinary experience with his high tech career. Roger is currently Vice President, Internet Security for Coastek Inc., a Scotts Valley, CA startup company specializing in software solutions for internet security. Roger has also worked in engineering, training, and management for several other Silicon Valley startups including MetaWare, Unisys, and Sysorex International.

Presentation and Navigation

• Stream1 The Gourmet stream demonstrates the traditional, hand-made way to prepare the meal, complete with proper techniques for the experienced cook. Learn how to make fresh pasta and a spicy twist on a classic beurre blanc with a fish stock base. The video stream takes the viewer through to presentation and wine pairing.
• Stream 2 Our Quick and Easy stream shows how to prepare the same meal in less than 30 minutes. This stream is for the novice cook wishing to impress in a failsafe manner, using boxed pasta and canned stock. Shortcuts and substitute ingredients are offered to accomplish the task quickly.
• Stream 3 For the health conscious gourmet, we show a low fat alternative for preparing our meal. This shows how to lower the fat content while maintaining flavor with the proper substitution of ingredients.
• Stream 4 Our second low fat stream shows the Quick and Easy method to prepare the meal.

Each stream is broken down into multiple segments allowing the viewer to create their own method of preparing the meal. For example, the viewer could choose to see how to make the shrimp stock (from the gourmet stream) and the low-fat beurre blanc (from the low-fat gourmet stream) while using store bought pasta (from the quick and easy stream). The following table shows the topics covered in each segment on each stream.

	Gourmet	Quick & Easy (< 30 minutes)	Low-Fat Experienced	Low-Fat Novice
	Introduction: Menu and directions for navigation
1.	Choose/describe ingredients*	Choose/describe ingredients*	Choose/describe ingredients*	Choose/describe ingredients*
2.	Make pasta from scratch	Mis en place	Make pasta from scratch	Mis en place
3.	Make stock from scratch	Mis en place	Make stock from scratch	Mis en place
4.	Sear scallops and shrimp	Sear scallops and shrimp	Sear scallops and shrimp	Sear scallops and shrimp
5.	Make sauce*	Make sauce*	Make sauce*	Make sauce*
6.	Cook fresh pasta	Cook store bought pasta	Cook fresh pasta	Cook store bought pasta
7.	Tomato concasse and parsley garnish	Chop tomato and parsley garnish	Tomato concasse and parsley garnish	Chop tomato and parsley garnish
8.	Plate the dish and pair the wine

Notes:
	* denotes different content although same segment title
	mis en place (everything in its place) = preparation of ingredients

Alternate treatments of synchronized multi-stream presentation

• Food and Wine Pairing - Pick 5 wines. Each stream would focus on a single wine. Within the streams different segments would show how the wine could be paired. Segment 1 could focus on what flavors to expect from the wine and the food that goes with it. Segment 2 could illustrate how to make one of the suggested entrees. Segment 3 could focus on how to make one of the suggested desserts. Segment 4 could discuss the wines only: history, production, where grown, where and how to purchase.
• Pick an entre (e.g. protein, vegetable, starch) - Streams: the Quick meal (shortcuts et al), the Low Fat version, the traditional version, and an optional ethnic treatment. Segments: how to buy the ingredients, preparation, discuss possible complimentary dishes to serve for an appetizer, side dish, and desert.
• Pick basic ingredients and teach ethnic treatments - Streams: Italian, French, Asian, Californian, Indian treatment of the same entree. Segments: a no cooking demo discussing the above treatments, where/how to purchase seasonings and ingredients, and actual meal preparation.

E-Commerce

As with the "Multiple Dimensions" show, "À la Carte" can also spawn interesting electronic commerce opportunities. These can include clickable images to assist in the sale of everything from the kitchen tools to services of a local grocery store which delivers the list of ingredients for the current recipe. The show could also include an internet web site with a tremendous wealth of related information

Additional Program Ideas

Synchronized Multicast Programs

Teaching Languages

Custom News