Moving cars

Figure 1 - the bridge

At reference 1 we looked at how an image processing system – possibly a brain – might look at a moving block. Concluding that the brain was likely to have more success with the horizontal features of a horizontally moving object than the vertical features. All this in the context of thinking some more about the frames of consciousness of reference 2.

Here we go one better, with a car moving across London Bridge while we are trying to concentrate on H.M.S. Belfast, framed by Tower Bridge. With the snap courtesy of Street View.

Figure 2 - the car

So what is the subjective, visual experience of a yellow car moving at speed across the field of vision, say from left to right? Perhaps the car illustrated above.

We suppose that the background is more or less unchanging. We suppose that the car, in itself, is also more or less unchanging, leaving aside the occupants and the wheels for the present. All that is changing is the position of the car against the background.

We suppose that this happens rather suddenly and that during the time in question here, neither head nor eyes of our subject attempt to track this car. The subject continues to look straight ahead, attending to H.M.S. Belfast, anchored in front of Tower Bridge. So what does the brain do – if anything – about updating this Belfast flavoured frame of consciousness while the car speeds across it?

Option 1: update the whole frame fast enough that the subjective experience is spot on, only slightly degraded by the reduction in background input from the areas temporarily occluded by the foreground car. If the viewer maintains attention on the Belfast, the car may not even be noticed. It may not even make layer object status.

All this might work at slow speeds, but will break down at some point, as the speed increases. Our intuition is that consciousness is quite slow, that while the front-end processing on the retinas might be fast enough, the back-end processing in the brain is slow. So which bit of the system breaks down? Which bit of the brain system does not manage what a film camera does with ease?

Option 2: the speed increases to the point that the car is moving so fast that all that gets added, briefly, to the subjective experience is a sense of movement of something from left to right. No further detail is available. This sense of movement would be expressed in a new layer, not containing very much information and not lasting very long. Not a visual layer in the ordinary sense of the word at all.

But something which might well take our attention from the Belfast. We are programmed by evolution to worry about odd movements in the bushes – or anywhere else – and we might need to be holding quite hard onto the Belfast, not to be distracted by the odd movement. From where I associate to the scene in the film ‘Goldfinger’, where James is in danger of being put off his important shot in a game of golf by Auric rattling the change in his pocket.

Option 3: somewhere in between the two options foregoing. The background bridge  and river scene is on layer 1 and the foreground car is on layer 3. H.M.S. Belfast is the object being attended to, with supplementary information about her on layer 2. With whether one can be conscious of both the visual image of the Belfast and supplementary information about the Belfast, say that the object which is responsible for the image, for the layer object being attended to, is a ship and that this ship is called ‘H.M.S. Belfast’, let alone the calibre of its guns and the number of bulkheads, being a matter for debate, although introspection suggests that the answer is ‘yes’.

Figure 3 - the shadow

The brain cannot retain visual images for very long and needs more or less continuous input if the image is to be maintained. So even though the background is not changing, the parts of layer 1 which are temporarily occluded are degraded. Then the brain does what it can with the car. It certainly registers movement, vertebrate brains being good at that. It correctly deduces that some object is moving across the front of the field of vision. It fairly successfully works out the shadow of the car on the background. This is suggested in the figure above.

Figure 4 - the horizontal integration

Figure 5 - the projection

Following the thinking of reference 1, having worked out that the movement is horizontal, the brain tries adding up the signal along horizontal bands, which produces something reasonably stable which it can project onto layer 3, this being suggested in Figures 4 and 5 above. A bit more information is getting to consciousness than the plain shadow of Figure 3.

Under this option, the original bridge scene remains on layers 1 and 2. H.M.S. Belfast, the object of attention, is a composite object spanning those layers. As long as attention is held, there is just the one frame. Then the car comes along, fast enough that the brain does not attempt to integrate it into the existing layers, and it just does what it can with a new layer 3.

The present proposition (to be further developed in a post to come) is that this new layer does not necessarily mean the start of a new frame of consciousness. In our daily lives, a frame is often triggered when the eyes saccade to some substantially different place, perhaps because something of interest has been detected there by peripheral vision. Or when attention falters because the frame has gone on too long. One such trigger might be the car doing something unusual, like suddenly taking off vertically, something which the brain may notice, even when the car is not the object of attention. The trajectory of the car has departed substantially from its predicted trajectory, the car becomes the object of attention and the brain needs to take a short time out to recompile that car and its trajectory into a new frame of consciousness. From where I associate, contrariwise, to the films they show on management courses about how people attending to the basketball do not notice that a gorilla has joined in. See, for example, reference 3. A famous example of what turned out to be the surprisingly common phenomenon of inattentional blindness: all kinds of stuff goes on in front of our very eyes of which we are completely oblivious.

But frame decision apart, what sort of an experience can the brain build? At slow speeds, the brain can cope and what we see is a good rendering of a car moving across a good rendering of the background.

But suppose, for example, that the subjective experience of the car is integrated from a tenth of a second’s worth of input, then the image of the car has potentially stretched by the amount it has moved in a tenth of a second. Say at 50kph,  (50 * 1,000) / (60 * 60 * 10) = a bit more than a metre. That is to say that the distance from the position of the back of the car at the start of the period to that of the front of the car at the end of the period is significantly more than the real length of the car. And following the thinking of reference 1, the upshot seems to be that the horizontal features of the car are strengthened at the expense of vertical features and that the car as a whole ceases to be fully opaque and some of the background which is not occluded for the whole of the input period comes through it. The faint verticals bordering the doors (in Figure 4) vanish, while the strong black vertical between the windows survives in the darkening of the corresponding stripes.

Mechanisms

Figure 6

The brain, unlike an old fashioned film camera or a projector with their frames of short but fixed duration, is very context sensitive. So the amount of time over which it integrates its image of the car will vary with context. Light conditions might be good or bad. The condition of the car – a very dirty car being harder to make out – might be good or bad. And the brain itself might be in a good or bad condition. Perhaps, for the first part of a New Year’s Day in a bad condition. The brain might be in a high or low state of alert. All of these things will affect the integration time.

One way of modelling some of this is sketched in Figure 6 above. We have frames of consciousness of varying duration, during which times data for the frame following are collected and compiled. The compiled frame is then transferred to the delivery area, to the layers of LWS-N, during a short inter frame gap – given that we have just the one LWS-N and the switch from one frame to another cannot be instantaneous, any more than the progression from one frame to another on film stock in a projector.

So while frame preparation goes on in parallel with frame display, the frame preparation itself, like the frames themselves is very serial, which puts a tight constraint on the amount of time available. This seriality following from the assumption, from the subjective experience of a single stream of consciousness.

Figure 7

In this sketch we relax the serial condition, allowing more than one stream of collection and compilation. More time, but also more collections of data to look after, more to do in the data management department. Note that the delivery of the second frame has a slightly shorter duration that that of the first, reflecting its smaller information content. Also that we have supposed elsewhere that the information content of a frame of consciousness is of the same order as that of a snap from a mobile phone, that is to say around 5Mb. But it can be a good deal less, unlike Tononi’s theory of Φ, which seems to require lots of information if there is to be any consciousness of it at all. See, for example, reference 4.

Figure 8

But perhaps integration time is not the best way to look at this. Perhaps we do better with something like the arrangement sketched above. The brain collects information about the visual scene more or less continuously, sorting it into object and feature-flavoured pots. Another part of the brain is sampling these pots, with a view to updating the current frame. In which connection, some workers argue that this sampling is driven by frequency, with all the stuff relating to the current object of attention having the same frequency, in some sense. That aside, if the sample gets too far off-piste, it may elect to go for a new frame. We suppose that the material derived from the retinas has a fairly short half life and that left to themselves the pots will run down fairly quickly, with the result that most of that which is sampled is going to be fairly recent. If the car goes missing, it will very quickly vanish from the visual part of consciousness: maintenance of visual images in consciousness for longer than a second or so – or perhaps some shorter period – requires sustained input from the retinas, the brain not having, in normal circumstances anyway, the ability to generate such stuff from memory.

In any event, given the time constraints and the speed of the car, it may not be possible to maintain a good quality moving image in consciousness, leaving us with something along the lines of Figure 5 above, with what one sees being a collection of somewhat see-through horizontal stripes. Give or take what exactly is done about the car not being a rectangle.

Miscellanea

We mentioned above the subjective experience of a single stream of consciousness, with the common view being that multiple personalities and multiple streams of unconscious processing are one thing – but multiple, concurrent streams of consciousness are quite another. That said, work has no doubt been done, is no doubt being done, on exactly what these last might amount to. Are the two streams mutually aware? To the extent that consciousness is in charge, which stream is in charge of which leg? Or is consciousness more like the chairman’s summing up of the discussion about the current item of the agenda? An agreed, unitary view to be written into the minute and on which to base action? No need to clutter up the minute with all the minutiae (as it were). No need for streams.

We note that there is no way, no movement of eyes, head or body, in which the apparent movement of the foreground car against the background bridge can be supressed. Either the background moves, the car moves or both. The movement of the foreground against the background is not relative, it is real. Although that said, if one ran alongside the car while keeping one’s gaze, through the car windows, firmly fixed on the Belfast, one would be doing quite well - and living dangerously, so not to be encouraged.

Lots of work has been done on testing the subjective experience of things which are briefly flashed up on computer screens, typically both just below and just above the threshold for their conscious perception. Fiddling with the context in which things are briefly flashed to see what difference that makes. Testing for the unconscious perception of things of which subjects are not conscious. An example of this work is to be found at reference 5 – which, inter alia, suggests that visual stimuli maybe short as a thirtieth of a second in duration can be processed and make it to consciousness – although the making it to consciousness will take rather longer, maybe ten times as long. With the stimulus in question there being a single capital letter against a plain background – a good deal less complicated that our yellow car – to which we allowed a tenth of a second. With the particular subject of this particular paper being the exploration of the limits in sensory processing speed which manifest themselves as the psychological refractory period and the attentional blink. And there are plenty more references in the book at reference 6.

It would be easy enough to organise experiments which explored aspects of the subjective experience of the movements of yellow cars. One could, for example, ask questions about the car while varying the speed of the car. Was the driver wearing a hat? One could test the conditions under which the subject held his attention on H.M.S. Belfast, with that attention not disturbed by the car. Not so clear how one probes what exactly it is that the subject experiences – it being hard enough to be clear about one’s own experiences, never mind someone else’s.

That said, as is explained in reference 6, it is possible to detect, without asking the subject, whether a short stimulus makes it to consciousness. The presently difficult bit being detecting what exactly it is that makes it.

Conclusions

We have developed the thinking about moving blocks in reference 1 for the more complicated scenario of a car moving across a bridge. Work in progress.

References

Reference 1: http://psmv4.blogspot.com/2020/01/moving-blocks.html.

Reference 2: http://psmv3.blogspot.com/2017/06/on-scenes.html.

Reference 3: https://www.youtube.com/watch?v=vJG698U2Mvo.

Reference 4: Consciousness: here, there and everywhere - Tononi & Koch – 2015.

Reference 5: A shared cortical bottleneck underlying Attentional Blink and Psychological Refractory Period - Sébastien Marti, Mariano Sigman, Stanislas Dehaene – 2012.

Reference 6: Consciousness and the Brain: Deciphering How the Brain Codes Our Thoughts – Stanislas Dehaene – 2016