[Lumiera] New guy trying to understand the proc model

Sat Mar 11 02:42:02 CET 2017

Thanks for the helpful information! It cleared up a few things for me. I’ve also started reading some of your tiddlywiki which has been very interesting. Although I personally don’t need a linux video editor, it is the engineering challenge that lures me to this project. It is fast moving up my priority list now… :) However, most of my programming experience has been in web development so not sure how much help I can be, lol.

Peter

> On Mar 10, 2017, at 10:39 AM, Ichthyostega <prg at ichthyostega.de> wrote:
> 
> 
> 
> Hello Peter,
> 
> first of all -- thanks for your interest in this topic. You are welcome!
> Feel free to ask questions, because posing and answering questions actually
> helps to improve our understanding of the subject matter and is a valuable
> form of contribution.
> 
> On 09.03.2017 18:40, Peter Lama wrote:
>> I can see how Placements would work well for a timeline, but can it also 
>> represent node compositing? I'm wondering how it would handle effects that 
>> could have multiple inputs and/or outputs.
> 
> The interesting question is what it means "to represent"...?
> 
> On a technical level, the answer is, of course, yes. The whole point with
> placements is to cover and unify several somewhat tricky features you'd like to
> use in a more elaborate edit. But if "to represent" means to get it in a shape
> which a human understands and likes to work on, the situation is not so clear.
> It would be easy to cook up something based on the "mechanics" of placements,
> which would behave in a highly puzzling and confusing way. So the point is
> rather, can we come up with a representation in the user interface, which
> makes such a feature usable and fun to work with?
> 
> At this point I should clarify some relations. Lumiera has three Layers, and
> this is so for good reason, because the concerns addressed in each of these
> Layers are quite different. The question you pose here spans all three layers.
> 
> If we ask the question: "Does Lumiera support node editing?", then the answer
> would be "No!". With a special twist: The render engine of Lumiera is *based*
> on nodes, but we do not expose them to the user, like other applications do.
> To use a metaphor, node editing is very similar to assembly programming.
> When you first encounter this kind of programming, it looks very cool, it
> looks like giving you enormous powers. Yet in practice it is quite cumbersome
> even to do the most simple stuff in assembly. It does not scale to large
> projects, because, when working on that low level, everything is explicit.
> 
> This is the reason why we try to build a "high-level-model". To gain the
> ability to handle some things implicitly, akin to what a high level
> programming language does. There is a compiler (in Lumiera, we call it
> "Builder"), which walks this high-level model and builds the corresponding
> node network, which can then be performed by the engine. This is where the
> placements come into play. Incidentally, most of this stuff is not yet
> implemented ;-) We're kind of paving our way towards that area.
> 
> But even the high-level-model is not tangible for the user. It is a data
> structure. To allow working on it, we need the third layer, the UI. We need
> to translate the structures of the high-level-model into the commonly known
> interface metaphors, like boxes you can drag with the mouse, windows with
> scrollbars, areas you can expand, nonmodal property panes where you can
> tweak settings etc. At that level, we /could/ (personally I think, we should)
> create a special UI where you can drag connection lines between boxes to
> setup those properties of placement which are related to signal routing.
> But creating such an UI is nowhere near trivial, since you'd need maybe
> four or five dimensions, but you've got just two dimensions in an UI.
> 
> 
> On 09.03.2017 18:40, Peter Lama wrote:
>> I don't think I fully understand what all a Placement can do. I would 
>> appreciate it if you could explain.
> 
> A placement addresses or links together several different topics.
> 
> (1) memory management. A placement is a ref-counting smart-ptr.
> 
> (2) identity. A placement is like an object instance. It carries
>    an unique hash ID (we call that a LUID) and there is an index,
>    allowing to retrieve any placement in the session by ID.
> 
> (3) scopes. Placements can be attached to another placement. Anything
>    attached at a placement is considered to be /within/ the scope
>    created by that parent placement. Tracks are within the Sequence.
>    Tracks can be within Tracks. Clips are within Tracks. Effects
>    are within Clips, or within Tracks, or within a Pipe. Since
>    scopes are nested, you can fetch a global definition from an
>    outer scope, or you can shadow such a definition with a more
>    local definition.
> 
> (4) constraints. A placement holds an ordered list of constraint definitions.
>    We call them "LocatingPins". What is actually constrained, is essentially
>    open. At a minimum, each placement starts out with a time position and
>    an output designation. There /can/ be LocatingPins in the list to constrain
>    these, e.g. "must be 200 frames after the start of Sequence XYZ", or
>    "sound output should go to music subgroup". But LocatingPins /might/
>    introduce additional variables, which means additional degrees of freedom.
>    Like, a LocatingPin might say "sound position is 20° to the left and 45°
>    elevation" or "stereoscopic window position is at 5m distance", or
>    "colour grading uses profile moonlight"
> 
> (5) query resolution. You can ask a placement to resolve itself for a given
>    parameter / dimension: "find out your absolute time position". The placement
>    first applies its own locating pins. If the result is not determined, it
>    passes the query up to the placement of the enclosing scope. If the result
>    is not determined when we reach the root placement, the query is passed on
>    to generic rules and definitions in the session. Like, by default, we route
>    output to the master bus.
> 
> [side note: (1), (2), (3) is implemented. (4) is drafted, (5) is future work]
> 
> Now, when the Builder is triggered, it collects all placements and creates
> a priority queue of tasks. This way, it finds out at what point to start.
> It then prompts the corresponding placement at that points to resolve
> themselves. Which, of course, creates additional tasks, to be enqueued.
> Especially of interest here is, that, during that process, a placement
> might issue a "WiringPlug". Simply put, when the placement finds out that
> it ought be connected to xyz, say "music subgroup", it issues the demand
> "I want to be connected to music subgroup". At that point we do not know
> what "music subgroup" is. But, as we proceed, at some point (hopefully)
> we'll encounter another placement (likely a pipe), which issues a
> "WiringClaim": this placement (directed by its LocatingPins) demands
> "I am music subgroup". Now this allows the Builder to issue a WiringRequest.
> Probably you've got the idea by now. Progressively we'll order the elements,
> and at some point we're able to build up connection paths, starting from
> the media sources. This is where we begin to drop off processing nodes,
> which are wired backwards (each node knows only its predecessors).
> We end up with a small number of exit nodes, which can be "pulled"
> to start the actual rendering.
> 
> 
> I'd better stop at that point. Peter, I am well aware that I haven't really
> answered your question but hopefully I was able to indicate the direction
> towards which to head for a solution. Probably you've also gotten an intuitive
> understanding, that this system of placements can embody anything a nodal
> editing system can express, and it can even embody more. But the real challenge
> is how to shape an interface to enable users and allow them to use these powers.
> And this is a challenge which lies ahead, is not solved yet.
> 
> Cheers!
> Hermann Vosseler
> (aka "Ichthyo")
> 
> 
> 
> _______________________________________________
> Lumiera mailing list
> Lumiera at lists.lumiera.org
> http://lists.lumiera.org/cgi-bin/mailman/listinfo/lumiera
> http://lumiera.org/donations.html