Sponsored in part by... Mark/Space, Inc. MARK/SPACE, INC: The Missing Sync provides the very best in
synchronization for Mac users with BlackBerry, Palm OS, or
Windows Mobile devices. Integrates with Address Book, iCal,
Entourage, iPhoto, and iTunes. <http://www.markspace.com/bits>

 [F] TidBITS  / TidBITS  / TidBITS Talk  /

Software for editing and printing multiple timed streams

[derek]derek (apparently) - 07:58am Aug 17, 2005 PST
via email

I'm looking for some software suggestions. What I need is a way to print
out various streams of text, all synchronized to a printed time scale.
This doesn't have to happen in real time, nor in an automated way. I can
manually create the needed alignment. How can I create a printable,
textual/graphic representation of various event streams, as they happened
in time?

The entire concept is complex, but the first layer is simple. I need to
present the English text of a 15 minute lecture, that was presented and
videotaped. If that was all I needed, a text document would do it. The
next step makes it harder. I need to combine some visual representation of
a time scale with the lecture text. The combination would allow the reader
to see what word was being spoken at a given moment, for example, what word
the speaker was pronouncing precisely two minutes and eighteen seconds
after he began the lecture. Alternatively, when the speaker uttered the
word "cumquat," what was the elapsed time?

I'm imagining a display of the text of a lecture, line by line. Below each
line of text, we need some sort of time code, such as vertical tick marks
indicating every second. The tick marks should probably be regular in
their spacing, so we need some way to adjust the text of the lecture, in
order to synchronize the printed words with the time of their delivery.
This is beyond what I know how to do in a word processor, but it gets
worse.

The lecture was interpreted into American Sign Language (ASL). The
interpretation was videotaped. The interpreter, of course, signed the
words with somewhat different syntax, and with a varying delay after the
speakers utterances. The ASL will be transcribed into English text
description, and we would like to place that text stream in synchrony with
the original lecture text. As with the English lecture text, we want to be
able to see in the printout what was being signed at a given time, and vice
versa.

Using the lecture text, the time scale, and the ASL transcription, we could
compare what the lecturer and interpreter where doing at a given moment.
For example, we might measure the average lag time between the speaker and
the interpreter, to see how that varied over time. We are actually
interested in several features of the interaction, but all require a way to
relate the ASL and the lecture to the time scale.

The last layer of complexity is adding the reactions of the deaf audience.
Several deaf participants watched the videotape of the lecture being signed
in ASL. They marked the video for certain key features of the information
presentation. We would like to be able to display those marks- when the
different participants marked the different events- and to be able to see
how these marks related to the ASL, and to each other. The same time-based
relationships needs to be shown. The goal of this research is to improve
interpretation, so that the deaf community can receive more effective
services.

I am hoping that someone might know of software that could be adapted to
this need of displaying multiple time streams. I haven't thought of any
good way to approach it. The only bad way that I have come up with, is to
use a layout program like InDesign, to make a series of text boxes, each
one line high, one for each data stream. Each stream could be linked to
the appropriate box on the next page. Synchronization could be
accomplished by adding spaces in the text.

This is a really ugly idea. I'm hoping someone can give me a better one.

Thanks for your thoughts,

Derek Roff
Language Learning Center
Ortega Hall 129, MSC03-2100
University of New Mexico
Albuquerque, NM 87131-0001
505/277-7368, fax 505/277-3885
Internet: derekunm.edu


Mark as Read
  OutlineAll MessagesOlder MessagesOldest MessagesNewest MessagesNewer Messages

benr (apparently) - Aug 18, 2005 7:49 am (#1 Total: 4)  

Reply to this message
via email  

Photo of Author
Posts: 49
Re: Software for editing and printing multiple timed streams

Derek Roff wrote:
 > I'm looking for some software suggestions. What I need is a way to
 > print out various streams of text, all synchronized to a printed time
 > scale....

Derek,

Sounds like a fascinating project.

My first suggestion would be QuickTime: it's simple, and cheap, and will
accomodate a great deal of what you want.

QuickTime movies can contain multiple tracks of different kinds, and you
can switch them on and off, and position the visual ones in different
places. One of the track types is text. Text tracks can appear in any
position, with any font, style, colour etc - including changing these at
abitrary times.

You can generate a text track from a text specification file - a simple
syntax allows you to mark up time codes, and the change to that time
code - change of text, or changes of font, style, colour etc.

If you opened a single tracks, the English transcript of the lecture, in
QuickTime player, you'd see a running display of the time code in the
controller.

You could trivially generate a second track that was just time codes, to
whatever resolution you wanted, and have that displayed at any location
in relation to the transcript, perhaps in some larger font etc.

You could add another text track, the ASL transcription; then the two
tracks would play side by side, or one above the other - and with the
timecode still visible. Of course you can use the controller to scrub
back and forth.

And then of course you could add the video of the lecture, and the video
of the ASL interpreter, as two separate tracks; again, you could space
them as you wished in relation to each other; in two dimensions, but
also in time.

Finally, you could add the reactions of the participants, as yet another
  text track.

Just using the standard player (in 'Pro' mode), without requiring any
custom software development, you can have many different ways of
presenting the material, all synced. Using the player, you can enable
and disable particular tracks, for example. In addition, QuickTime
movies can reference other movies, so you can easily create multiple
movies (each of which occupy just a few K); each displaying different
combinations of the same tracks.

QuickTime might not do everything you want; but I think it would get a
very long way there, for a very modest investment - not only in terms of
$35, but also in that you'd hardly need to do any work beyond the
timecoding of the transcriptions that you'd have to do anyway, whatever
solution you went for. So even if it turned out that QT was not the
solution you end up with, you wouldn't have wasted too much money or
effort along the way.

(And if in fact you did need custom software, it might well be based
round QuickTime anyway. We've done created apps for example that show
the video from a quicktime movie in one place, but extract different
text tracks and use one to provide a commentary elsewhere on screen, and
another to trigger various other events - QuickTime did 95% of the work,
making our app development simple. You might also venture into the
realms of Interactive QuickTime... though you'd want to take a deep
breath and pack a survival kit before you went in there!)

(If you really wanted output on paper, I'd probably argue that this
would still be a good place to start; with all the material arranged as
above, it would not be very difficult to write some software to run out
a page layout, which might eg have a regular time scale, and on that
time scale place in separate rows or columns the time code, a sample
frame from each video track, and the text from each text track.)

BTW on a trivial aspect of the job, I'm just this evening testing a
service that will take an audio file and a transcript (or for extra
money, they'll do the transcription) and return it in various timecoded
formats (including QT text), using a cunning automated service. It's
not perfect, but it looks like it could save a ton of time for anyone
with a lot of such work to do. If in your case there's just a single 15
minute lecture, and only one English transcript (I wouldn't expect it to
work with the ASL), it's probably marginal - but in case you're
interested, contact Kevin Erler at Automatic Sync <www.automaticsync.com>.

Hope this helps -- I'd be very interested to know what you end up with.

   Ben Rubinstein | Email: benr_mlcogapp.com
   Cognitive Applications Ltd | Phone: +44 (0)1273-821600
   http://www.cogapp.com | Fax : +44 (0)1273-728866

Nigel Stanger (apparently) - Aug 18, 2005 7:49 am (#2 Total: 4)  

Reply to this message
via email - Dunedin, New Zealand  

Photo of Author
Posts: 422
Re: Software for editing and printing multiple timed streams

On 18/8/2005 2:58 AM, "Derek Roff" <derekunm.edu> spake thus:

> I am hoping that someone might know of software that could be adapted to
> this need of displaying multiple time streams.

The closest thing that I can think of is that some most music sequencing
software supports a (possibly multiple) lyrics track that's synchronised
with the audio. The same is probably true of video editing software too, and
there you'd have the bonus of actually having the video track in place to
aid in synchronisation. Whether or not you'd be able to then print the text
streams is beyond my knowledge, however.

--
Nigel Stanger, Dunedin, NEW ZEALAND.
http://public.xdi.org/=nigel.stanger

pbd - Aug 25, 2005 8:39 am (#3 Total: 4)  

Reply to this message
 

Photo of Author
Posts: 1
Re: Software for editing and printing multiple timed streams

Hi Derek,

The process you're describing, to a large degree, is known as "Spotting" within the Film & TV industries.

Our company, Skramsoft, is today releasing a software product called First Cut, for Mac OS X. A great deal of the functionality you seek already exists, uniquely, in First Cut. Other features you seek are either in-development or planned.

We welcome you to take to take a look and assess First Cut's suitability for your needs.

Please feel free to contact us to discuss ways in which you may be able to leverage First Cut for your needs.

Cheers,

Jan Skarbek

Skramsoft LLC

jskarbekskramsoft.com

http://www.skramsoft.com

Farad - Dec 28, 2006 1:21 pm (#4 Total: 4)  

Reply to this message
 

Photo of Author
Posts: 1
Re: Software for editing and printing multiple timed streams

It appears now that Skramsoft are not longer distributing the First Cut product. Skramsoft's web site's content has been entirely replaced and no longer refers to First Cut at all. More details about Skramsoft can be found at http://www.farad.com.au/skramsoft/skramsoft_deregistration.php



  OutlineAll MessagesOlder MessagesOldest MessagesNewest MessagesNewer Messages


 [F] TidBITS  / TidBITS  / TidBITS Talk  / Software for editing and printing multiple timed streams




Add a message

To add a message to this discussion, you must be a registered user. Enter your email address below. If you have an account associated with the email address you enter, you will be prompted for your password. If not, you'll be able to create a new account with no fuss.

Enter your email address:

Submit