Introduction
 |
| Figure 1 |
The purpose of this note is to introduce some concepts and definitions which we will use when we come to describe how a simple scene, for example the aeroplane in a clear blue sky snapped by dreamstime left (see reference 5), is mapped into LWS-N (see reference 4). In particular, how the brain adds value, over and above what a camera achieves. How it marks up the image to make it more useful, to enrichen the subjective experience, to make the image more like a diagram than a photograph.
A mapping process which is rather different from what, for example, is done by the Waymo computer when it drives around the town, a computer which does not aspire to consciousness, although knowledge of the former might well provide useful input to our knowledge of the latter. One difference, for example, being that the computer collects far more data than could possibly be fitted into consciousness - which has to reflect a radical process of pruning, selection and simplification.
With one the questions arising being why is consciousness so slow? For Waymo, see reference 10.
The two important definitions in what follows are those of ensembles of neurons and of their descriptions in words. We are not presently much concerned with how these things might come to be, in evolution, in development or otherwise; just with what they might be.
Note first, that a core proposition of LWS-N is that the cortical precursor of what we see, the subjective experience, has been reassembled into topical form, into something like the digital image produced by the camera in a mobile telephone, from all the bits and pieces into which the topical signal from the retina has been analysed in across the visual and higher areas of the brain.
Note second, that it seems very unlikely that the brain, if it does indeed include these features, would implement them in the way described. Its approach to things is statistical rather than algorithmic and is unlikely to come more than approximately to the algorithmic descriptions which follow.
Images
We use image to mean the sensory input from the eyes integrated over some short period of time, say a second or so. We do not mean the sensory input from the eyes generated in an instant, say a few milliseconds.
 |
| Figure 2 |
We suppose the existence of some function which measures the distance between images. Such functions start with simple bit by bit comparisons across the two image frames and go on to allowing all kinds of image transformations and all kinds of interactions with the content, what the image is about. With the former including various more or less rigid transformations the foreground object, the latter including, for example, the unrolling of the trunk of an elephant.
Bearing in mind that the relationship between two images is far from one dimensional and that we can only capture one facet of that relationship in a single, real valued function, we suppose a middling sort of function which minimises the difference between the two images, subject only to permitted transformations – which might be stretched to allow limited manipulations of the foreground object. But not the content sensitivity – which might have been stretched to include more complex manipulations, like the unrolling of the trunk already mentioned.
Ensembles
Our brain knows about things, in the beginning straightforward things like apples, lions, children and mountains, things which we come to describe by common nouns. We avoid getting philosophical about exactly what kind of things that brains know about by saying that the things that the brain knows about are expressed by, defined by ensembles of neurons, what we call light bulb neurons. An ensemble is a strongly connected set of possibly thousands of neurons, possibly being scattered over some small or large area of the cortex, but which, as a whole, reliably fire for the things they stand for, which fire reliably for the right sort of stimuli. We place no constraints on what sort of things or what sort of stimuli. A sample of things is given in the middle of reference 2. Some steps towards finding such neurons are described at reference 3.
We mean strongly connected in the graph theoretic sense. If E is an ensemble, A and B members of that ensemble, then it is likely that A and B are directly connected, that is to say that A stimulates B, B stimulates A or both. And if E is an ensemble, A and B members of that ensemble, then there is a short route in E by which A stimulates B.
We hope, in time, to be able to put some limits on the number of neurons in an ensemble. But for the moment, the best we can do is a lower limit in the hundreds.
If an ensemble E is active at time T then all its constituent neurons are firing. We expect to be able to make this more precise by saying that all its constituent neurons must have fired at least N times within the previous M milliseconds, with N and M positive integers, yet to be identified. Or perhaps something more probabilistic. We expect activations to endure for at least hundreds of milliseconds.
An elementary stimulus is some continuous segment of sensory input spanning some short period of time, typically less than a second or so. A stimulus is not confined to a single mode, say vision, but that said, many stimuli will be unimodal, often simple images, possibly moving, displayed on a computer screen. The length of an elementary stimulus is one.
A complex stimulus is the orderly presentation of a sequence of elementary stimuli. The length of a complex stimulus is the number of constituent elementary stimuli.
While there might well be dark matter in this universe, we are only presently interested in ensembles that we can know about. That is to say, if E is an ensemble, then there must be a stimulus, presentation of which reliably results in the activation of E. In the case of a complex stimulus, the brain is being prepped by the left hand part of the sequence, to the point that E is then reliably activated by the last member of the sequence. Put another way, the left hand part of the sequence is establishing context. Better still, there is an elementary stimulus which reliably activates E, regardless of the starting state of the brain.
We might add the qualification that we are talking here about a brain belonging to an healthy, adult human who is not under any particular stress. Perhaps in what is called the resting state.
Any one stimulus might activate a number of ensembles. But if E1 and E2 are distinct ensembles, then no neuron is a member of both E1 and E2. Furthermore, there is at least one stimulus which reliably activates E1 but not E2 or at least one stimulus which reliably activates E2 but not E1. Put another way, if E1 and E2 are distinct, they must exist and they must be distinguishable.
No ensemble is activated by more than a small number of the very large number of stimuli which are available. At any one time, all but a very few ensembles just sit there, inactive.
We might particularly value ensembles which can be fired up all by themselves, without a cloud of relatives getting in on the act.
The descriptions introduced below can be used as stimuli, as symbols in place of images drawn more or less directly from the world.
Ensembles are robust. They can lose lots of neurons and lots of internal connections before they cease to function.
We allow degrees of firing up, degrees of activation. There might be a thresholds: an ensemble either fires or not; but given that it does fire, we allow different degrees of firing up.
Inter alia, the ensemble functions as a junction box. If an inbound stimulus fires up an ensemble, that ensemble will usually go on to fire up all kinds of other stuff. So if we fire up ‘golden eagle’, the ensemble goes on to fire up stuff about golden eagles. With exactly which stuff usually depending on context. If, for example, one was a gamekeeper, one’s trigger finger might start twitching.
Given the expense of creating ensembles, and by analogy with databases on computers, there might also be something by way of a management information payoff, with their being a good peg on which to hang such stuff.
We need to remember, in the case of the ensemble being about internal stuff, that they are about that internal stuff, they are not the internal stuff itself. An important example is the idea of pain being different from the pain itself. Ensembles are symbolic, ensembles are signs, symbols of and signs for something other than themselves. Furthermore, while the flashing of the ensemble lights might remind us of the original stimulus, might result in replay of that original stimulus, usually we will know the difference. We might have a problem when the sign becomes merged with the stimulus.
Looking ahead, we see growth of ensembles, with ensembles acquiring more neurons as they acquire more importance in the scheme of things. We also see decay and we also see fission of ensembles; what was one ensemble becoming two or more. Ensembles might wax and wane like the moon, a few even with the moon. Some ensembles might start out, but never really make it before fading away again. Some ensembles may be temporary or transient, a variety of working memory. Some ensembles may grow from scratch, from nothing, in which case the idea of lower limit of size (introduced above) becomes a matter of convention more than of fact. In any event, a regular natural history of ensembles. Or in the jargon of data modellers ‘entity life histories’. See reference 9.
But the rest of this note is mainly about what else we can say about these ensembles.
Individuals and collectives
All ensembles are individuals in the sense that an ensemble is about something as a whole, not about the parts or members.
 |
| Figure 3 |
Regarding which, the summary above is suggestive, but which should not be taken too seriously. The sample is very small and we know nothing about the way that entries were selected for this part of the dictionary. Perhaps there were quotas to fill. Certainly a bias towards the French and France, although we doubt whether that much affects this summary. The suggestion being that most ensembles are not going to correspond to a proper noun of this sort. Most people don’t have a clue what Ricci (Lorenzo) or Ricci (Mateo) (to take two examples from another part of Larousse) looked like, never mind what their distinguishing features were. And one bend in the River Thames probably looks much like another bend in the River Marne. Most people neither know nor can name these things: we have to be sparing with our proper names if they are to be useful.
The most important thing about some individual things is what they are made of: a lump of cheese, a pile of sand, a fall of snow. There might be an associated quantity – either a number or something vaguer – and many quantities have units, like metres or yards. For example, 500 grams of cheese or more than a pound of cheese. While such things will certainly have a role in working memory, we are not clear how that stretches to ensembles. Perhaps there would just be one ensemble for each material that we knew about, with the selection varying a good deal from person to person. Some people might be very interested in different kinds of snow, other people might be very interested in different kinds of cheese.
So, while in any one brain there will be ensembles corresponding to entries in a dictionary such a Larousse, there will be probably be more for individual things which are important to the host but which are not of general interest and which do not have a proper name. For example, a favourite tree with a large ball of mistletoe near the top, on the way to the newsagent.
And we expect that rather more, indeed most, ensembles will correspond to common nouns, things which are common enough to have attracted one or two word labels. For example, the ensemble for cabbage will lead to a jumble of information about cabbages the we have known. Allotment keepers and market gardeners might go so far as to have ensembles for different kinds of cabbage – Brussels sprouts reminding us of the need to let in two word forms of this sort.
These ensembles symbolise unbounded collections of things of the same kind: elephants, cheeses, cabbages and kings. The number of such things in such a collection may not be well defined and is certainly not known.
Other ensembles will symbolise bounded collections. Six railway locomotives. The eleven players in a certain football team at a certain time. The 513 houses on a housing estate. We can sometimes, but not necessarily, enumerate the members of a bounded collection. Note that these wholes are almost always greater than the sum of their constituent members; we can almost always find things to say things about the whole, which we cannot say of any of those constituents. For example, this is the greatest team in the world. We associate to the fact that much of the power of an infantry battalion, at least in the Duke of Wellington’s time, lay in the ability of its members to act as one, to act as a unit, not as a bunch of individuals.
Relations between ensembles
A collection is allowed to be a subset of another collection. For example, elephants are animals. A temporary ensemble might be established by the utterance ‘he had apples and pears in his basket’, in which case these particular apples and pears are subsets of the sets of apples and pears more generally.
A collection may be the union of some other collections and individuals. We gloss over the mathematically important distinction between an individual and the collection consisting of just that individual.
An individual is allowed to be a member of a collection. For example, this particular elephant is a member of the troupe of elephants run by this or that circus. The troupe trainer is likely to have ensembles for both troupe and the individual elephants.
These two relations are sometimes conflated into the isa relation, as in ‘the ant isa insect’. A conflation which means we do not fuss about whether we are talking about a member or a subset relation. The second of the pair is necessarily a collection, while the first might or might not be.
Sometimes what is an individual in one context might be a collection in another. So a football club has members, players and others, might be a member of the premier division and is affiliated to the Football Association. Perhaps we can handle this with the context established with stimuli, mentioned in the previous section. We know in which sense we are presently interested in this ensemble from the context.
One individual can be a part of another individual and one collection can be a part of another collection. For example, trees have trunks. In which case, each member of the collection ‘tree’ can be associated with a member of the collection ‘trunk’. The explosion of things like aeroplanes and power stations into parts – and even the humble zip fastener – possibly involving many layers of parts, is important in engineering and manufacturing applications on computers, probably less so in brains, which likely confine themselves to one or two layers at any one time, in any one frame of consciousness.
If A is a member of the collection B, then we might say that C takes the value A for the property B. Otherwise, B is a plausible property for something like C and the members of B are the values that property can take. Properties in the sense of reference 1.
All of this can get terribly complicated and no-one that we know of has come up with a simple framework of this kind which captures everything; that seems to be the rather messy preserve of natural language. All the same, knowing about these relations is an important ingredient of our knowledge of ourselves and of the world that we live, and the brain must have some way of having and using this knowledge. It is an important ingredient in the construction of what gets into LWS-N. But it is not an important ingredient in LWS-N; its work has been done by then and LWS-N just projects some of the output of that work into consciousness. These relations are not further considered in what follows.
Words
Words are things which can be said, heard or written. For present purposes words are certain, permitted sequences of characters. Characters are letters (for example: A, B, g), numerals (for example: 1, 2, 3), other (for example: @, ¾, &, -). With comma, round brackets and space being further examples of other characters. We do not go into the complications involved in relating the character ‘¾’ to a number, but readers interested in other kinds of complication might like to read the Wikipedia article on escape characters at reference 6.
A simple word will be one of: token (usually mainly letters), number (usually mainly numerals) or Boolean (either yes or no. Equivalently, true or false).
A word will be a simple word or a short, underscore separated sequence of simple words. Words do not contain internal spaces.
We allow upper, lower case letters and accented letters, but just treat them all as separate letters, disregarding the relations between them. But for the present we do not consider case or accent, or any of the other stuff which rightly vexes grammarians. We do not consider alphabets other than the English alphabet, phonetic relations between letters and groups of letters (for example, ‘draft’ and ‘draught’ sound the same), font or size. And finally, we do not consider relations between words – lexical (‘cat’ is like ‘cats’) or otherwise (‘flat’ is like ‘apartment’). All matters which are relevant, but beyond our present scope.
Expressions
An expression is a short, possibly empty, sequence of phrases, with an optional head.
A head, if present, is a word. In the present context, often a common noun, for example ‘cat’.
A phrase is a pair of phrase label and phrase value.
Expressions are often written as follows, with square brackets indicating optionality:
- [<head>][([<label>=]<value>…])
- power_station(coal location=Orford_Ness age=43(years) owner=BlackRock)
In the case that the second part is present, there may be one or more phrases.
Phrase labels are optional. For example, if the value is ‘red’ then it can be assumed, in the absence of an explicit label, that the implicit label is ‘colour’. The knowledge that red is a colour would be somewhere in the works.
Phrase values are mandatory and might be a word or some more complex expression.
Some examples of expressions are:
- cat
- cat(name=Frisky)
- Frisky(cat)
- cat(red)
- cat(colour=red)
- cat(colour=red legs=4)
- cat(colour=red legs=4 tail=no)
- shop(brand=Costa town=Epsom street=High_Street)
- (colour=brown legs=6 tail=yes)
Some phrases express properties of a thing of the kind denoted by the head, in the sense set out in reference 1.
The expressions we are presently interested in are neither particularly long nor particularly complicated, but the form is powerful and expressive and can be used to express stuff of more or less arbitrary complexity. Indeed, they are very similar in organisation to expression in the XML which powers much of the Internet.
We can define a isa relation between expressions, a relation which is lexical in the sense that it only depends on the words and their arrangement, with no necessary connection to the real world – although we would expect, on the whole, pretty good alignment; that is part of the point of having words and expressions. Roughly speaking, an expression E1 isa E2 if the expression E1 is bigger than the expression E2, if everything in E1 is to be found in an appropriate place in E2. So, for example, if a label is present in E1 then it must either be absent or matched in E2. So ‘cat(colour=red)’ isa ‘cat(red)’ but not ‘cat(colour=red)’ isa ‘cat(tail=red)’. Such a relation is simple and natural in conception, but can be made complicated and can be made to do a great deal of work.
We can define distance, or distance like functions, on pairs of expressions, functions which are also lexical, but which depend on spelling rather than structure. So ‘hake’ is like ‘hack’. The general idea is to try and map one string of letters onto another and the distance is the measure of the extent to which one fails. Alternatively, one tries to map the sound signal for one lot of words onto that for another, which tests for sounding the same rather than spelling the same. Which might or might not be more useful, depending on what exactly one is trying to do.
We might try to combine these two very different approaches in some way. We might require that if A, B and C are distinct expressions such that A isa B isa C, then the distance between A and C is greater than either of the distances between A and B or B and C but is less than their sum. We might require that if A, B and C are distinct expressions such A isa B but A is not a C, then the distance between A and B is less than the distance between A and C. With the catch here being that A might be a lot more like C than B.
In any event, the object is to end up with some function which measures the distance between two expressions. A companion to the function which measures the distance between two images with which we started.
Descriptions
Descriptions are short expressions which are applied to ensembles. Remembering that lots of ensembles will not have descriptions and lots of descriptions will not have ensembles – a point to which we will return below. Note that while a description does indeed describe an ensemble, for example by providing some key words, it is more like a title than a glowing description of a sun-lit, bubbling stream that a romantic writer would recognise.
If we know a common noun, and that common noun stands for an ordinary, visible, material object in the world – like a telephone box or an armadillo – then there will be an ensemble for that common noun. Or put the other way around, that common noun is a description for that ensemble. We could not know the word in any useful sense otherwise. These ensembles with their one word descriptions make a good start.
Both cat(red) and red(cat) are valid, slightly longer descriptions of ensembles. The first describes a cat or cats which are red. The second describes a red or reds which are appropriate to cats. The sort of reds that cats are. Some people will have ensembles for both descriptions. Others might have just one ensemble, perhaps better described by ‘(cat red)’, the jumble of stuff we remember about reds and cats that we have known.
We note some tension, between a description which is something like ‘Bacchus and Ariadne’, the title of the painting included above, the title by which it is known in the trade, and something which is more descriptive, something like ‘painting(medium=oils period=Italian_Renaissance subject=(Greek_myth pastoral) content=(people(many mainly(naked)) animals(few)))’. The former might do for a connoisseur while the latter might do for the rest of us. One can combine the two styles to make a rather long description, but that does not get away from the convention that just one common noun is at the front. Hopefully, the most important single bit of information about the thing in question, at the time in question.
There is clear attraction in there being a simple map from description to ensemble and we will suppose for the present that any one ensemble is associated with zero, one or more descriptions, while a description may be associated with at most one ensemble – while recognising that some other rules about this might turn out to work better. We might, for example, let in probability: it is 90% likely that this description should be about that ensemble.
The proposition is that in building the contents of consciousness, in building LWS-N, the brain mainly divides the world up into layer objects and parts corresponding pre-existing ensembles with pre-existing descriptions. And it labels those objects and parts in LWS-N, with a small number of those labels being in or near consciousness. So in the case of the aeroplane above, we might have the description aeroplane(engine=4 movement=fast) or aeroplane(type=jumbo movement=fast). The knowledge that healthy jumbo jets have four engines might be somewhere in the works – although we ourselves had to check.
Note that the set of descriptions in used is finite, say less than a few thousand of them, while the set of expressions is much larger, infinite in the case that we do not put any serious limit on length.
From description to stimulus
Keeping things simple, we say that the stimulus corresponding to a description, is no more than its display on a computer screen, more or less as shown here.
We suppose that descriptions are short enough that they can be absorbed from the screen in a second or so, so not involve much in the way of eye movement and do not involve any conscious thought for comprehension.
In this way, we could provide the stimulus easily enough. What is more difficult is finding the matching ensembles inside the brain and recording their behaviour. Scanning a brain and deciding that it is thinking about playing tennis rather than eating rice balls is not quite the same thing, although a step in the right direction. See reference 7 for such a step.
Getting from a description to a bit of natural language is another matter. As a first attempt one might just strip out the control structure, leaving just words and spaces. So ‘cat(colour=red tail=no)’ might be rendered as ‘cat colour red tail no’. And ‘house-boat(colour=(black white) length=12(metre))’ might be rendered as ‘house boat colour black white length 12 metre’. Which gets us some of the way. But something much more complicated would be needed for any worthwhile second attempt. Nevertheless, machinery which generates a bit of proper, natural language is clearly somewhere in the works.
From stimulus to ensemble
 |
| Figure 4 |
In the beginning, there were ensembles but no descriptions. From where we have moved to a position where many ensembles do have descriptions. Which often means that we can activate the ensemble by hearing, reading or saying the associated description. Or by flashing that description up for a second or so on a computer screen. The description is sufficient stimulus, in the sense of the section on ensembles above, to activate the right ensemble. Which amounts to a very useful bit of kit when it comes to sharing and building knowledge about the world, something that people without words and other animals, pretty much all of which are without words, are not very good at.
If an image of something that we know about is flashed on the screen, say an elephant, then some suitable ensemble E1 and its description D1 are activated. And if D2 is any other description such that D1 isa D2, then any ensemble E2 corresponding to D2 might well be activated, but not as much. We might add relations which go the other way, any other description such that D3 isa D1. This scenario is sketched lower left in the figure above – with the search there being shorthand for some probably quite complicated process. Simply minimising some distance function of the sort mentioned above by clambering over some search space is unlikely to be good enough; something involving both feed forward and feed back is likely to be required, something involving both top down and bottom up, is more likely. Perhaps even something which can understand that a dog with erect tail is still a dog, that a dog which is sitting down is still a dog, even if the host has not seen such a thing before. Such understanding may well involve something like analysis of the image into the sort of elements, layer objects and parts, envisaged for LWS-N. Remembering here that while the skill of human infants at tasks of this sort often impresses, the skills of human adults will vary a good deal.
Second from left, we sketch a good part of the point of descriptions. If the description of something that we know about is flashed on the screen, then the right ensemble gets activated. In a computer this might well be achieved with an index rather than a search, it is likely that the brain does better than a simple search – a much less challenging task than that presented in the first sketch, as the heavy lifting has already been done in coining the word.
Third from left, we start to get more complicated, with their not being an ensemble to match the description. So after searching through both ensembles and descriptions, the brain simply manufactures an appropriate image and then uses it to build an ensemble. Or it may not bother if the description is only of passing interest.
And then on the right we have the artist. We suppose he has seen lots of houses over the years, and lots of things that were pink, but never a pink house, despite their being common in the west country, in the Baltic and parts further north. But he would certainly make a good job of painting a picture of a pink house, entirely convincing to someone who knew pink houses well. We dare say that some people could make a good job of hallucinating a pink house to order, although we do not know any such person – and we note that most naturalistic painters prefer to sketch, if not paint, from life. The point is that in this case, the artist is generating a certain sort of object, a fictitious object on demand, rather than drawing what he sees or drawing on memory resources to supply real pink houses he has known. And to the extent that his painting has been important to him, at least for a while, his brain would have turned his painting into an ensemble. Perhaps an ensemble which gradually sinks back, with its connections, into the generality of houses.
Conclusions
We have speculated about the possibilities for ensembles and descriptions, notions which seem to us to have interesting possibilities, but which despite a simple start, seem to hide all kinds of complexities and complications. This despite sticking with just the one modality, sight. So work in progress.
We also need to give some thought about how the existence in a brain of such things might be tested and verified. No test, no existence.
References
Reference 1: http://psmv4.blogspot.com/2018/12/of-cabbages-and-kings-more.html.
Reference 2: https://psmv3.blogspot.com/2018/08/of-cabbages-and-kings.html.
Reference 3: Gnostic cells in the 21st century - Rodrigo Quian Quiroga – 2013.
Reference 4: http://psmv3.blogspot.com/2018/05/an-update-on-seeing-red-rectangles.html.
Reference 5: https://www.dreamstime.com/. A site which was new to us, but of which we are now non-paying members.
Reference 6: https://en.wikipedia.org/wiki/Escape_character. There are further references at the end of this article for readers wanting more of the same.
Reference 7: Detecting awareness after severe brain injury - Davinia Fernández-Espejo and Adrian M. Owen – 2013.
Reference 8: http://psmv3.blogspot.com/2018/06/measuring-consciousness.html. An earlier use of reference 7.
Reference 9: Event Modelling: Entity Life Histories – G. Watkins – 2003. http://www.cems.uwe.ac.uk/~gwatkins/isdp2/04-05/lec4-5elh.doc. Or ask Bing; plenty of other stuff out there.
Reference 10: https://waymo.com/. The Google self-driving car project – with a sophisticated website.
Group search key: srd.
No comments:
Post a Comment