Steps In Time

Steps In Time is the tentative brand name of a Google Glass application under conceptual development by Larry Denenberg and others who wish to participate.

The application functions as a dancers' assistant, something like this:

• A dancer wearing Google Glass activates Steps In Time at the start of a session.

• During the introduction to each song, the application recognizes the music and identifies the dance.

• The application then cues the dancer during the dance, giving the dancer help remembering the steps or learning the dance for the first time.

The remainder of this article addresses the technical and nontechnical challenges in building this product. This page is locked, but if you wish to join the discussion you can edit the talk page; material will be moved from the there to here as appropriate.

(Each page of HoraWiki has an associated talk page for comments. You can see the talk page by clicking the "Discussion" tab on the upper left hand side of the page.)

Technical Issues

Music Recognition

This is one of the easier problems, because Google Glass already provides music recognition. Moreover, the space of songs to be recognized is very small, at most a few thousand pieces of music (though it's true that many are similar, or have similar introductions). It's important that recognition be quick since we need to know the dance before the first step, even when the introduction is short. This doesn't seem problematic.

Question: Can the music recognition software be accessed from third-party apps?

Synchronization

The app needs to not only recognize the music, but to stay in sync over the course of the entire dance. In the ideal case, the app would continue to listen to the music and detect the beat. It probably suffices to measure the tempo accurately during recognition and assume no changes, or just known changes, e.g. for recordings that are known to speed up. Problems can still arise when the markid arbitrarily changes the tempo mid-dance.

Users of Steps In Time will have a grossly unfair advantage during Steppin' Out, but that hardly seems a reason to scuttle the project.

Cueing

This is the central problem, and a science in itself: How do we help the dancer do the dance?

Most people learning a dance do so by watching and following. Should we display a head-up video of the dance? This seems pointless, because the dancer can just look around (assuming that there are other dancers that already know the dance—learning a completely new dance from a video is a different problem). Also, watching a video during a harkada seems too distracting from the dancer's enjoyment.

Cueing the dancer with audio seems a better plan. Dancers are already comfortable with audio cueing both during teaching and also when adjacent dancers help by calling a dance. Google Glass provides bone conduction audio that permits the wearer to listen with almost no sound audible to others.

(Ultimately, of course, it would be best to have the hardware attach directly to the nervous system, controlling the muscles, so that no volitional effort at all is required. Maybe in V2.)

So suppose we try to imitate calling as it would be done for a neighbor, or by a caller cueing a contra dance. We give the starting foot, use standard terminology for familiar sequences ("left yemenite", "na'aleh to the center") and cue sections of the dance ("Part I again") for global understanding. The calls come slightly before the actual steps. Cueing fades out over the course of the dance; the third time through, we need less cueing than the first time through.

It seems likely that we will want to provide multiple styles of cueing—more or less verbose, emphasis on left/right vs use of standard sequences, more or less cueing of styling, adjustable advance timing. We have to provide a way to select cueing options no matter what, because the user must at least select a language and a gender role for partner dances.

Database

The app requires a database consisting of, for every dance, both cueing information and modelling data for music recognition. The total memory footprint must be small enough to fit everything on the device. We also need a simple and probably automatic mechanism for wireless updates, given the high rate of new dance creation.

Building the database initially is a huge problem. Preparing the music recognition is straightforward and can be automated, but how do we build all the cueing without having someone cue each dance into a microphone, in each language we want to support and for each gender role and cueing style?

As a start, we might crowdsource the problem. Suppose we permit people to create their own cueing; call it a "personal style" that you can select when you want to supply your own dance hints. Now we have an audience of people creating cueing, and we acquire the best ones to provide to others.

Alternative Architectures and Platforms

We've described Steps In Time as a Google Glass application because of the capabilities and relative familiarity of that platform. But there are alternatives, especially since we don't seem to need head-up video, just audio input and output, the latter preferably via bone conduction, plus sufficient computing/storage resources for music recognition, syncing, and cueing. And also since Google Glass is, at this writing, on hiatus.

(By the way, is bone-conduction audio audible to a partner who is in physical contact with the user? Or to someone holding hands with the user in a circle, as if that ever happens nowadays?)

We can approach the problem differently by enlisting the cooperation of the markid. If the application is integrated with the music-playing software and broadcasts cues from a central location, some problems disappear: There's no need for music recognition, synchronization is trivial, there are no constraints on the size of the data, and database updating is easier. The receiver worn by the dancers can be a much simpler device: just a ear or bone-conduction bud with wireless connection to the central station. The markid might put a box of receivers (color-coded for gender, language, and style) at the entrance. Grab one on your way in!

In fact, both proposed architectures can be supported simultaneously, and the self-contained version can make use of anything broadcast by the centrally-based version. The latter is much easier to develop and perhaps should be tackled first.

Nontechnical Issues

Name

"Steps In Time" is, of course, a shoutout to the chimney sweeps' dance in the movie Mary Poppins. We should come up with a better name, preferably one that has some connection to Israel or Israeli dance, especially since domain stepsintime.com is, at this writing, listed for sale at $25000. Though perhaps there is merit in a non-Israeli-centric name for marketing to other groups (see below). Here are some suggestions:

• The Dancer's Friend
• Hora Helper
• Dance With Me (though dancewithme.com is already taken too)
• Remez (or Ramzor)
• Debkassist

Several others have been proposed on the discussion page. Some creativity needed here.

Marketing

There's obviously an extremely limited market for a dancer's assistant like the one described here. We can expand it slightly by broadening the repertoire to include international folk dance, though it still isn't going to make anyone rich. It seems unlikely that contra and round dancers would want an automated substitute for a caller. Modern country line dancing is a promising target.

Are there other dance forms, or other sports, that might make use of a similar device? Someone learning Tai Kwon Do, for example, must execute a long sequence of moves in an exact order. Would they be permitted such a device for cueing?

Social Impact

Suppose a whole bunch of people start using this device. What happens? Would choreographers create dances any differently, knowing that people will have help dancing them? For example, maybe a consequence will be the creation of more dances with complex figures or many parts.