[Lumiera] Time, Time-spans, Quatisation and Timecode

[Ichthyostega]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


Hi Stefan,

first of all let me express that I'm really happy seeing your newest
contributions coming in -- as soon as I find some time, I'll start
merging in the first parts!

As your work on the time/timecode display widget includes time handling and
conversions, I thought I might point out how this relates to other parts
(partially written, partially just planned). Probably none of this is
immediately relevant to the code you wrote/committed already, but as you
continue in that direction, these considerations might become relevant
regarding the general direction we go. (and btw. I especially encourage
discussion about those future topics, that's why I bring this up on
the mailinglist and not just in private mail)


As a general note, around the year 2000 there was that discussion, maybe even
hype about weak typed languages. (at that time I myself was also part of that
movement). The general assumption was that weak/dynamic languages free up your
mind and thus make you more productive, because they remove obstacles, while
strong typed languages are "bureaucratic". This general assumption turned out
to be wrong. Rather, it's kind of as if there is some "preservation of energy
law" in effect here: I you don't want to climb a mountain, you'd better get
around it (use low ceremony languages). But *if* you want to (as we do here),
then you'd better start ascending early, so the slope doesn't get too steep.
That means setting up guide-lines and a kind of scaffolding early.


Having said that, the next steps are obvious: We have a lumiera::Time type,
which allows us to control possible conversions. We wouldn't be able to do that
just with basic number data types. Numbers are weakly typed. When we make API
functions accept values of type Time instead of numbers, this sets up a gentle
push towards the paths of righteousness  ;-)

The next thing then will be to introduce a *time span* type, which can be
achieved by *subtracting* two time points. At this point, the strict typing
approach starts to pay off: because now every inventor of an API function is
gently forced to switch on his/her brain and *find out* about the *meaning*,
that is, if it's a *time point* or a *time interval* or *offset* what is
needed for the operations in question.

At that point, we need to introduce *Quantisation*. That means to /align/ a
continuous (or quasi continuous) value to a pre-existing, fixed /time grid./
This operation needs the input of an external information about that time
grid, thus it is *impossible to implement it as stand-alone library* function.
(Mind me, I mean it! I know, a gazillion of clever programmers try again and
again to come up with clever tricks here, and try to trick reality. Sorry,
we can't) At this point, indeed we just need that external information source.

Thus, we get at introducing a facility, which I'd call a *Quantiser* (better
name?): This is a service which gets configured and enabled by setting a
*Grid definition*. After this enablement, the Quantizer is capable of
converting Time points into TimeQuant points and intervals into quantised
intervals. And (second capability) it is able thus to translate time
values into *frame counts*.

After passing through an Quantiser, we get quantised time and time intervals,
which basically is data with an additional tag denoting the grid, so it can't
be mixed up. Of course we can draw the raw time values from quantised time.

It might not be immediately obvious, but a time grid consists of 4 informations:
- - the (absolute) time point /where/ you query the quantisation information
- - the current spacing ("current" means: at that point where you query)
- - the current zero point
- - the past history of grid changes, starting from an absolute zero point.
To make that more clear, think of an "bars/beats" quantisation, where the
BPM might have changed between songs, maybe also the tempi and the time base
(4/4, 3/4, 6/8) might have changed.
The same holds true (just the more) if we have a mixture of media (yes, we
have that for a NLE). There might be a time constraint on the zero point,
there might be a mix of media with various framerates, and (we've discussed
that point numerous times) there are media with variable framerate.

Using that approach of combining a Quantiser and a time grid definition
allows us to transform those more complicated cases into modular additions
to be implemented later.


Now considering the big picture:

- - What about the *Backend* ?
  The backend mostly talks in frame numbers, and besides that, at some areas
  it talks in system time / wall clock time (e.g. for scheduling against a
  deadline). It has never to deal directly with all those complicated matters
  detailed above, so it can be kept agnostic.

- - What about Data-Processors, Plug-ins, Media-handling Libraries ?
  Yes, they need to care, because they get to deal with storing and reading
  external timecode formats. Luckily that isn't our business (doh!).
  That was one of the most fundamental decisions for Lumiera, and I'm still
  glad we did that decision. Use Libraries, and if they're not up to the job,
  then work with the library's authors to improve them or choose another
  library.

- - What about the *Session* ?
  Largely the model talks in terms lumiera::Time values, but it need to be
  aware of quantisation. It has to deal with offsets at places. Thus the
  session is a consumer of quantisation services. The overall sanity of the
  model implementation will largely benefit from using clearly typed values
  to express what "flavour" of time is meant.

- - What about the *Player* subsystem ?
  This is the /one and only place/ where quantisation happens.
  Only the /individual play or render process/ has all the necessary
  informations to establish the time grid. It draws the foundation of those
  from both the session/model and the current output configuration. Because
  the player is capable of putting together a full quantisation, it is also
  capable to build a quantiser "as if there would be a playback right here".
  This "as-if" quantiser can then be consumed by the session e.g. to handle
  grid-aligned placements (in the build process, that is).
  Besides, when playback and rendering is going on, the player feeds current
  time and frame values to the play controler, which relays them to the
  GUI (and to the session for the current playback position marker).
  These are the informations which the time widget receives....


Stefan, what does this mean for your work? Not much, mostly. You concentrated
the actual math of some conversions in generic library functions and this
is a good idea IMHO. You might want to refrain from building cleverness
into the GUI widget which doesn't belong there and won't stay there.
But, after all, we're open to refactoring and we're employing that liberally.
Any good piece of code can be broken out and moved elsewhere later!
Anyway, for starters, it would be sufficient that the GUI widget
is just aware of timecode (display) formats...

Generally, for this topic, I'd like to ask you if it might be helpful for
your ongoing work to address those time and time quantisation handling in
the near future? You know, I am working on multiple "frontiers" and I can
re-order priorities, if that helps the co-workers. Thus, do you think it
would be helpful to build that entities outlined above right now?

Cheers,
Hermann Vosseler
(aka "Ichthyo")





-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org

iEYEARECAAYFAk0DqhUACgkQZbZrB6HelLI3MQCbB1W/NXaNIxfd2fufvMEri6H1
uw8AniWm+oKNyC07ziju9K0GuTS9eDRU
=vUJb
-----END PGP SIGNATURE-----