Читать книгу: «Hardware and software of the brain»

Copyright (c) I. Volkov, January 5, 2017 – November 12, 2023

Modern computers already surpassed complexity of the brain. What is more important, in the process of development, theoretical cybernetics elaborated numerous concepts and solutions which may be applied to living systems. The process is mutually beneficial. You explain how biological organisms operate and simultaneously get some hints about further development of machines.

The human brain is a live automatic control system. Hence it may be described in terms of modern cybernetics. It is very different from common computers, but the main concepts are applicable. A typical workable computer consists of the two main parts: hardware and software that is material and non-material halves. Accordingly, for humans we talk about the body and the soul. The term hardware is not very well suited because a half of the human body is water. Also objective phenomena behind the soul are much wider than just algorithms learned by a person. Nevertheless, the terms will be retained for compatibility.

Programmers of traditional computers know that software is heavily dependent on hardware. With the development of computer industry, large efforts were applied so as to achieve portability, but the talks is about a program which should run on 2 computers of the same type, but made by different manufacturers. It is obvious that if a different hardware has no some feature which is crucial for the program, then they are incompatible in principle. So we should begin from functional architecture of the brain, only then proceed to software which may run on this device.

Several latin words which are often used in medical literature

Lateral – located at the side.

Medial – located in the middle.

Rostral – shifted from the centre to the head.

Caudal – shifted from the centre to the tail.

Dorsal – back (humans) or upper (animals).

Ventral – front (humans) or lower (animals).

Acoronal plane dissects a structure into ventral and dorsal parts.

Asagittal plane separates the right from the left.

Some notes about anatomy

When you disassemble an electronic device, it usually contains several blocks which are functionally different and also well separated. They may be mounted on different printed circuit boards or even in separate boxes. There is no such separation in the brain. It is a smooth 3D mass which may be structured only by morphology, that is microstructure of nerve cells and fibers. Moreover, if you look at a cross-section of the brain, you will see almost nothing. Thin serial slices used for reconstruction are just transparent. Only after special staining that microstructure becomes visible. The next question. Suppose you have singled out some part of the brain as structurally different. Who can guarantee that it is functionally different too? If it is functionally different, is this function confined within this structure only? Such questions resulted in several systems of anatomical terminology. Different brain subdivisions may overlap and all of them have quite remote relation to functionality. Nevertheless this knowledge is crucial because without it you won't be able to understand the location of a certain point from its description in special literature.

Anatomy of the brain

Fig. 1.

The most rough division is: the hindbrain, midbrain, and forebrain. In Latin this will be: rhombencephalon, mesencephalon, and prosencephalon. Going in down-top direction, rhombencephalon is further subdivided into myelencephalon and metencephalon and prosencephalon – into diencephalon (the intermediate brain) and telencephalon. Latin terms sound terribly, but fortunately they may be encountered mainly in very specialized literature. Myelencephalon is also called medulla oblongata (the oblong brain).

Another often term is the brain stem. It begins from the spinal cord and includes medulla oblongata, pons of the hindbrain, the midbrain, and sometimes diencephalon too.

The next tier of anatomy is more relevant to functionality.

Fig. 2.

The reticular formation is an elongate structure or a chain of nuclei spreading from medulla oblongata into diencephalon. The reticular formation is located in the middle of the brain stem and may be considered as its core.

Fig. 3.

Metencephalon consists of cerebellum and pons. The former is also called a small brain because its structure is a simplified variant of the big brain. Pons means a bridge. It is formed by axons going to and from hemispheres of cerebellum. Important ascending and descending pathways obviously travel through pons. Also it includes the reticular formation and several specialized nuclei.

The dorsal part of the midbrain is formed by tectum that is the roof. It consists of the inferior and the superior colliculi (2 pairs of hillocks). The superior colliculus implements low-level visual processing. The inferior colliculus does the same for hearing.

If we need the line which separates peripheral devices from a computer case, it's here. With a few exclusions, what was named previously corresponds to controllers of periphery while the thalamus of diencephalon may be regarded as several expansion cards of PC.

Fig. 4. The thalamus.

Below the thalamus resides the hypothalamus which plays a similar role, only in regard to internal bodily functions.

Remaining telencephalon is a motherboard of a neurocomputer. It consists of 2 almost symmetrical hemispheres. Each hemisphere is covered with the cerebral cortex – its visible surface. Basal ganglia are a complex of subcortical nuclei which are hidden beneath.

Fig. 5. Coronal cut of anterior section of the Brain showing basal ganglia.

Finally, anatomy names a complex which is defined not by proximity or similarity of its parts, but by strong connections between them.

Fig. 6. Schematic briefly summarizing neural systems proposed to process emotion, highlighting structures that are visible on the medial surface of the brain. Papez's (1937) original circuit (A) was expanded upon in the concept of the limbic system (B) to include a variety of subcortical and cortical territories (MacLean, 1952; Heimer and Van Hoesen, 2006).

This is the limbic system. Its components are located in the midbrain and forebrain.

Hardware

Computers are made of electronic components which were also known as radio details because before the advent of microprocessors computers were made of the same details as radio, TV, and other consumer electronics. Elementary components of the brain are neurons – specialized nerve cells which can generate electric pulses and conduct them to long distances of tens of centimetres. There are different types of radio details: resistors, capacitors, transistors, and a few others. Likewise, there are a few (of the order of 10) different types of neurons which may be repeatedly encountered in the different parts of the nervous system. The most substantial difference between them is that some are excitatory, others – inhibitory. That is, firing of the first neuron may force the second to fire too, or suppress its background firing rate instead. Neurons are not the only cells of the brain. There is also glia (which serves as damper for neurons and insulator for long wires – nerves) and blood vessels which are indeed a special type of muscles.

Computer is a device that processes information. How is information represented in the brain? We can't determine immediately how our ideas are represented, but we can draw conclusions watching what happens when they come out to periphery and convert themselves into physical actions. It was definitely established that muscular tension depends upon the average firing rate in the nerve that ends up on this muscle. Hence, we can suppose that a single spike of a single neuron in the central nervous system doesn't matter and our ideas are encoded by the pulse activity averaged over a group of adjacent cells (called a cluster) and a certain period of time. The typical number of elements in such clusters is of the order of 1000. As to time parameters, the duration of one spike is approximately 1 millisecond so the firing rate of 1 neuron can't be more than 1 kilohertz. Multiplying it by the number of elements in the cluster, we get 1 megahertz, but keep in mind that the reaction time still can't be less than 1 millisecond because spikes are not rectangular.

At this time you might realize the shocking truth: our brain is so different from our computers that it is an analog (more exactly digital-analog) device at all. Meanwhile there is something that unites them. It is very symbolical that computer programs and musical records may be stored on the same type of media such as optical disks.

Anatomically, the brain consists of several parts which may be clearly distinguished and reproduce themselves in all humans. Their cell structure is different from the adjacent regions or they simply are visible from the surface. The inner space may be of two types – gray and white matter. The former is composed of cell bodies, the latter – of nerve fibers. Different parts of the brain are heavily interconnected. This supports the hypothesis that regions which look different are functionally different as well. All in all, anatomy distinguishes some couple of dozen different parts, but how to arrange them into a functionally meaningful construct?

Basic principles of a live neurocomputer are different from the Von Neumann architecture. Computer operative memory changes data by an instruction (the differential principle) and keeps data while power is on. Regeneration is a separate unconditional process. In live neural net, dynamic memory is a pattern of neural activity which should be explicitly supported by the system of nonspecific activation or by reverberation. In the second case, circulation of activity is also controlled by the nonspecific system. That is, in a computer, instructions are quick and their results remain forever. In a live neurocomputer actions are lengthy and continue while the activation signal remains. Retaining results also requires continuing activation.

Neurocomputing may be studied by purely mathematical methods. We can take 2D (or 3D) image as a main data unit, take associative instead of linear memory, and design a completely different computational model. The Von Neumann processor retrieves data from memory by the address of a memory cell. Associative memory uses keys instead. Also, a single neural net usually keeps multiple images which are superimposed in distributed storage. There are 2 types of such memory: autoassociative and heteroassociative. In the first case, the goal is just to memorize many images, then to recall one of them using some hints. In the second – associations between images of different types are remembered. This may be used to implement stimulus – reaction or event – handler pairs.

Two types of continuous computation are visible right away. In the second case, the system is placed into real-world conditions where external events come permanently so some reactions will be generated permanently too. In the first, we can use an image retrieved from autoassociative memory as a key for the next operation. This implements "free thinking" or "flight of ideas".

A computer comes with a ready set of instructions. The set of elementary, hardware-supported actions for a neurocomputer is smaller. This resembles the situation with fonts. The first typewriters had fixed sets. Then, they were replaced by graphics and fonts are usually generated by software now.

Science doesn't use the term software in application to living systems. Researchers study behavior instead. There is even a separate branch called behaviorism. It regards the nervous system as a black box and tries to formulate laws which link its inputs and outputs. The goal of brain research is seemingly to establish how different structures participate in generation of complex behavior. The task turned out to be tricky. The primitive approach is to determine correspondence between elementary actions and various parts of the nervous system. This encountered resistance from the opposite group which claimed that it is impossible and any function is equally distributed over the brain as a whole. For a computer engineer, the solution is obvious. Mapping is possible, but it is internal rather than external actions which should be mapped. Such as memory read/write operations.

The brain as a whole may be best approximated as a finite-state automaton, but this approach has one problem. Neural activity is highly dynamic and requires energy consumption. If you change the state and relax, it will always slip into the zero state. This issue is resolved using a specific architecture of neural nets. Karl Pribram in his "Languages of the brain" highlighted that many connections in the nervous system are reciprocal (bidirectional). If the biofeedback is negative, this serves for stabilization. If it is positive, this will create a generator which can maintain an activity once it was launched. As a result, you may look at an object, then close your eyes, but the image will remain and you even will be able to examine its details.

Abstract neurocomputer

Our computers are based on a Turing machine. Its principle is very simple. Complexity of computers comes from software, not from hardware. The same should be true for the human brain. The problem is that there are no tape and read/write head inside the human scull. Then, what is the prototype for the Turing model? Thorough consideration reveals that he formalized work of a human which uses some external storage such as paper. This approach resembles what is known in neuroscience as behaviorism. It doesn't try to penetrate into the head, regards the brain as a black box with inputs and outputs. What we need now – to describe how this box operates inside.

The first hint comes already from the input signal. Turing supposed that this is text while for the brain it is video. Speech and written textual input emerge only later in biological evolution and human civilization. Neurophysiological study shows that topology of an input image is retained well into the deep parts of visual analyzer so we can conclude that the format of 2D images is the main data format of brain hardware. The next principle is determined by the type of memory used. For any processor, memory input/output is one of the most important operations simply because a processor should process some data and data is stored in memory. The Von Neumann processor uses linear memory, that is a sequence of bytes which are accessed by their address. The human brain has associative memory instead. An elementary block uses a 2d (or 3D) array (millions) of neurons where information is represented by the pattern of activity. The image may be retrieved only as a whole, but this storage is still efficient because a single block can retain many different images. The method of retrieval uses a key which is also some image.

These two principles – a 2D image instead of a byte and associative instead of linear memory may be used as an axiomatic foundation of mathematical neurocomputing. The science may theoretically explore all the possible methods of data processing and all the possible constructs of neuro-machines. Meanwhile the brain is a ready working solution optimized by the nature so we can add further details from data gathered by biological experiments. The next striking difference between common computers and a neurocomputer is the absence of the clock generator. A live neurocomputer is asynchronous. Brain rhythms are well described, but alpha-rhythm of the visual cortex emerges when eyes are closed. Instead, activation is manifested by desynchronization. This creates a major problem. At each step of its operation, a computer should decide what to do next. A neurocomputer should decide when in addition. On the other hand, this feature creates additional flexibility and it is used. Human computations are based on insight. It is well known as a source for great discoveries, but indeed is used on routine basis in tens, maybe hundreds per day.The principle of a computer: take the next instruction from memory at the next pulse of a clock generator. In a neurocomputer, associative memory provides an appropriate idea at an appropriate moment. This sounds like the Holy Grail, but unfortunately this formula contains a lot of uncertainty. More specifically, insight is: 1. generated by hardware that is has no psychological explanation, 2. heavily dependent on previous experience. There are two types of insight: sensory and motor, that is related to input (perception) and output (action). These images are generated in different parts of the neocortex, but subjectively we feel them similarly because all the neocortex has almost the same structure like computer memory. Insights of the first type are especially interesting because they have an explanation from the theory of information. Visual input conveys digital-analog data which according to Shannon's formula contains a virtually infinite amount of information. The nervous system is simply unable to process it all so only some portions are captured when appropriate.

How are insights generated? This is related to another problem. The brain has memory for sure, but does it have a processor?

Theory of neurocomputing

Hardware and software are just parts of another item – a whole computational system and it is obvious that a computer based on the Von Neumann architecture and the human brain are very different. Paradox is that computers were thoroughly developed for decades and we know them in details while evolution of the brain spans millions of years, we use it permanently, but have no clues about its operation. Substantial efforts were made during 19 and 20 centuries so as to fill this gap. Let's try to formulate explicitly the most basic principles of neurocomputing. First, we need to determine our goal. What is a neurocomputer? Any construct created of neural nets? Their range would be as wide as diversity of live nervous systems. Let's confine our interest to the human brain, but keep in mind that it isn't the top of perfectness. Maybe the brain of some animals is better in some aspect. Maybe there are better solutions unknown in the nature. The prototype is only a hint for the theory.

Let's formulate the answer at the very beginning, then explain it in details. Thorough consideration shows that the following 3 concepts are workable for both types of computing, but they have different implementation. Moreover, distribution of overall computing between memory, a processor, and software is different. A neurocomputer uses associative memory implemented as blocks of homogeneous neural nets. Even in the simplest form with 2 layers, they are already capable of some processing such as branching or simple arithmetic (addition and subtraction). That's it. We have processing without a processor. Add the difference in data representation. Computer memory stores data immediately. In neural nets, data is an image represented as a pattern of neural activity, but memory is kept in modifiable synapses. Nets don't store images immediately. They store the ability to generate particular images in response to particular keys. This entails another difference. In a computer, data is loaded from memory into a processor, undergo modification, then is stored back. In a neurocomputer, images are processed immediately in the memory. Elementary processor instructions are replaced by hard-wired ability of a particular block of memory to perform particular image transformations. This ability is defined by local interconnections within the same layer or between different layers of the same net. In a computer it corresponds to some procedure or algorithm which implements a particular method of data processing. On the other hand, each homogeneous net working as heteroassociative memory keeps associations between input and output images. This corresponds to rules of traditional programming, only blocks of those rules are kept in different, genetically predetermined hardware locations. You see that in a neurocomputer, the main paradigm is rule-based and types of rules are predetermined at hardware level.

So does a neurocomputer have a processor? Seemingly, yes, but it is striped of many functions known for its counterpart in a computer. Basically, if we regard the neocortex as the memory, the role of a processor will be to activate or suppress different areas and links between them. A good example is attention. That is, a processor of a neurocomputer accepts various signals from outside and inside the brain, but its output is just internal "turn on" or "turn off". Meanwhile there is something else. Software may be further subdivided into applications, system-level, and firmware (hardware emulation). While the first part is associated with the neocortex and constitutes various externally visible human abilities, the last is stored in a processor. Here we encounter an interesting turn. We have already seen that the memory of a neurocomputer is capable of processing. On the other hand, its processor is made of similar blocks of associative memory. Only now processing is regarded as a main function while memory provides storage for system programming.

Subcortical nuclei of the brain are well described both anatomically and functionally. Many of them provide low-level control which is unrelated to the psyche and behavior. Others are a part of motor and sensory systems. For example, the thalamus is a major relay station which conducts signals from sensory organs to the neocortex. It is the remainder that may be regarded as a candidate to the role of a processor. Two main parts may be named: the basal ganglia and the limbic system. Functional study of the basal ganglia hints that this is also memory indeed. Seeking analogy with computers, one can say that the basal ganglia are BIOS – the memory which contains the most basic subroutines. So the only part remains – the limbic system and its blocks are well connected into a functionally complete unit. Only not all of this complex may be regarded as a CPU. The limbic system is usually associated with motivation and emotions. Human motivation is subdivided into 2 absolutely different types which come from different parts of the brain. High-level goals and long-term plans are generated in the frontal neocortex, while biological desires such as hunger – in the hypothalamus. If we put aside such blocks, 2 major structures remain – the hippocampus and the amygdala.

Let's try and figure out how this processor operates in general.

A Turing machine is claimed to be universal. The human brain is even more universal. It can work in the completely analog mode when a person monitors some object and follows its movements. The first thing to do is to determine the class of tasks which the brain processor is used for. The brain as a whole is a regulator so if everything is normal, it may stay in the state of idle run. Despite not sleeping, it may do nothing special, just perform random actions without particular use. An alternative is goal-oriented behavior. That's what is interesting for us. That's when activity becomes highly structured and the processor operates at full power. The typical example is the task to reach some place walking in the city. Other tasks may be described by analogy. Consider manufacturing. The goal is to assemble some product from parts. Performed operations are separate steps or crossroads on the route, only motion happens in virtual space. The method to generate goal-oriented behavior is problem solving. That's the difference. Creativity. A Turing machine is designed to perform ready algorithms, the processor of the brain – to generate algorithms.

The main approach to creativity remains ancient trials-and-errors method. For computers, more orderly variants such as full scan of possible solutions are used. In any case, the processor should prompt some actions, then assess their results. That's exactly what the limbic system does. Looks like the amygdala is a block of system-level memory similar to the basal ganglia. Only they keep elementary programs for sensorimotor coordination while the amygdala keeps genetically predetermined states of the brain itself such as fear or aggression. Human emotions correspond to processor instructions of computers, only the brain doesn't use the Von Neumann architecture. It uses a finite-state machine which has elementary states rather than actions. This approach may be very powerful. It is well known that the brain parts tend to have connections according to the principle of all-to-all. Not all of them are used at once. Instead, only a fraction is employed when necessary. This means that you can dynamically create different working machines for different circumstances.

On the other hand, the hippocampus has all the necessary means to assess the results. It receives input from major sensory channels and can generate sharp pulses of activation at the output. Again, this works differently. When you write a control program, you would create a variable to measure the assessment. Say, in the range [-1,1]. The program will input data, process it, set that variable, then use it for decision making. Looks like the brain has not such a separate variable. Instead, associative memory immediately links an input situation to the appropriate emotional reaction such as attraction or aversion. That's why negative emotions are harmful. You get tensed and if this tension has no exit, you must contain it. That leads to double tension and quick tiredness.

Other parts of functional architecture

If the neocortex is the associative memory of the brain and the limbic system – its processor, the next step would be to describe Input/Output system. In contrast to PC, human memory is anatomically partitioned and different parts are assigned for different purposes. Comparison of human and computer I/O systems may be as lengthy as the preceding comparison of central processing, but now we will just mention that they are very different as well and concentrate on the first. In particular, I/O system of PC was very asymmetric. These computers had quickly got a graphical display with a complicated controller and a video card, but the main input remained a keyboard and a mouse. When decent video input became available, PC itself already lost its market share. Output subsystem was obviously better developed here and the term "programming" reflects this. It refers more to generating output actions and pays less attention to analysis of input data.

In humans, the main input comes from vision and the main output goes to muscles via the motor system. Both are approximately equally developed. They have cortical and subcortical representation of comparable complexity. The human sensory system has several different receptors (vision, hearing, etc.) with separate sensory channels. At the top, all of them converge in the TPO (Temporalis-Parietalis-Occipitalis) region of the neocortex which resides between cortical representations of different sensory modalities. The TPO zone obviously syntheses an abstract picture of the world and is also related to human consciousness.

While the sensory neocortex occupies the back half of the brain surface, the front half is designated for motor output and actions in more general sense. Humans have only one channel of external output – muscles, but the overall output system is slightly more complicated than its input counterpart. It includes the cerebellum sometimes called a small brain inside the big one. The cerebellum may be considered as a close analogy of a video card in PC. It has its own cortex and subcortical nuclei that is seemingly memory and a processor, but is used for motor tasks.

At last, yet another part of architecture. The reticular formation has a few distinct features. It is located among the most ancient parts of the brain, is large, but has virtually no internal structure, and has diffuse output to other parts. This is an activating block of the brain. We could outline the analogy with the power unit of PC, but here the difference is probably the largest. The nervous system has no separate power supply at all. Instead, each neuron generates energy for itself. Nevertheless the On/Off switch is necessary. It comes in the form of the sleep/awakeness cycle. In contrast to computer electronics, neurons are heavily dependent on biochemistry which produces a lot of waste and requires regeneration of stock substances. That's why we sleep every day and the reticular formation controls this cycle. Moreover, it can fine-tune its output so as not just to turn the whole brain on or off, but to do it separately for different structures. The reticular formation not only sends output, but also receives input from other parts. This supports the principle of a dynamic processor discussed previously in connection with the amygdala.

That's all about the core functional architecture of the brain. Now let's formalize it in a concise description.

Formal neurocomputer

This machine is created from blocks of associative memory linked in certain order. Each block has input and output in the format of 2D or 3D image plus a control signal that switches the mode of operation – read out, write in, or inactivate. In the first mode, the block uses the input image as a key to retrieve an output one. In the second – memory remembers the association between 2 images.

The machine is connected to the external world by input and output. Both have hierarchical structure with several tiers and maybe several parallel channels which converge from periphery to the centre. The main difference is the direction of information flow. The sensory system gathers data from various receptors and creates an integral picture of the world in the central representation. The motor system gets a general idea of the action in its central representation, then implements it in concrete actions of various effectors.

Operation of the machine is determined by interaction between central images of sensory and motor systems. A sensory image serves as a key to the motor block. Associative memory generates an idea and while this idea is being realized, the associative pathways between the upper levels are inactivated. As soon as the action is completed, they are activated again, but now the content of sensory operative memory is already changed. Two sources are possible. The action may change something in the environment or data may be written directly into the sensory memory in the course of internal exchange. Hence the key will be different and associative memory will generate the next insight. The cycle repeats. So the work of this machine is based on chain reactions.

Fig. 7.

This scheme is compiled from the most reliably described links between various brain regions. How to understand its operation? The main problem here is multiple feedbacks. The common method in such cases is to break the loop. Then, operation of the open circuit will become obvious. So we need to remove some links from this scheme, but which specifically? The red nucleus is a clearly supplementary structure because the motor cortex already has direct output to motoneurons. So we will throw it out together with the cerebellum. Functions of the latter are well established. It provides muscular coordination and fine-tuning. If it is damaged, patients still cam move, but their movements become more primitive, rough, and non-coordinated. The cerebellum is a typical "improvement".

Fig. 8.

This picture still may be further simplified.

Fig. 9.

The upper link on the picture is a well known reflex. The link through the basal ganglia is a bit more complicated. It combines the previous principle of stimulus-reaction with sensorimotor coordination. The last detail comes from the fact that muscular actions change the environment and these changes will be perceived by external sensors.

Fig. 10.

This device is not so weird as it may seem at the first glance. For illustration, let's demonstrate how to implement common linear programming here. Suppose you have some algorithm, that is a definite sequence of actions. Just associate each one with the corresponding number in the sequence and remember these associations. By the way, memorizing may be performed in any order. Next, you will need a small system program. Namely, you should have a sensory image which keeps the number of the current operation. That is you emulate the register of a common processor. Increment it after each action. Now use this image as a key for sensorimotor associations and you will be able to run common programs on this computer.

Processor of the brain

Fig. 11.

Limbic system.

1 – Hippocampus. 2 – Amygdala. 3 – Mammillary body. 4 – Fornix.

Basal ganglia.

5 – Caudate nucleus. 6 – Lentiform nucleus.

Other.

7 – Pituitary gland.

Like a personal computer, the human brain also has memory and a processor, although they operate and interact quite differently. Neurocomputing does not use linear memory containing bytes or machine words which may be accessed one by one. Instead, it relies upon associative memory that keeps bytes in synapses, but they can't be immediately read out. Humans use a different principle. Memory bytes form a distributed hologram which is only an instrument to retrieve data. The output comes in the form of an image encoded by a current spike rate distributed over some neural net. Again, in contrast to a computer, each neuron outputs not binary data of 1 or 0, but a real value in some range, say [0, 1]. This principle is used both in memory and a processor; they are different only by the architecture. Human memory is located in the neocortex which is a large 2D structure packed into the skull. It has 6 layers of neurons of different types which are basically the same throughout the whole of its area, but this area is genetically partitioned according to the functional principle. The frontal half of the neocortex works mainly for output, the rear – for input. Many different cortical fields are interconnected by direct axonal projections. This system alone already can work like computer software. When some portion of data comes to a sensory input, this is 1:1 as an event in a computer. This event produces activation which spreads to a different field, undergoes various processing, and finally may generate a muscular action. The problem is that this flow is not synchronized, no single clock generator. Also, brain connections tend to be organized by the principle of all-to-all or one-to-many. The uncontrollable spread of activity will lead to informational overflow. Obviously, for a given event, only a fraction of all fields and their links should be used. All the rest need to be inactivated. This is the main function of the brain processor – the limbic system.

In a computer, the processor has a definite set of instructions and handles data loaded from memory. In the brain, data are processed immediately in memory. The processor only starts and stops this process. Nevertheless, it has some instruction set too. The limbic system generates emotions. Motions are activation of muscles, emotions – similar activation of internal brain structures. Like the neocortex, parts of the limbic system also keep various images, but these images encode patterns of activity for other parts of the brain. Some of them need to be boosted, others – suppressed. Such emotional patterns may be learned, but also may be inborn basic distributions of strong reactions.

Another important function of the limbic system is related to operative memory. Associative memory does not keep the image itself – only the ability to evoke it. Nevertheless, neural nets can retain images too. For this purpose, they build a sort of a trigger system using positive biofeedback. In this case, the output of some internal block is supplied to its input starting recirculation in the closed loop. As soon as some pattern gets into this system, it becomes locked and can exist for a prolonged time even if the source was turned off. The limbic system was initially called the Papez circuit. It and the adjacent basal ganglia have many such loops which may be employed in various cases. Usually, some cortical field sends output to the limbic system and also receives input from it. Obviously, when some image is reverberating through the limbic structures, they can control it. That's how this system manages operative memory.

A well known function of emotions is assessment. In fact, most of them are broadly divided into 2 categories – positive and negative. The strength of emotion provides a value for this sign. Suppose you need to write a program for monitoring of some object. How would you do that? Probably you will define an appropriate variable, write the evaluation there, and use it for decision making when necessary. Seemingly the limbic system operates differently. The assessment is not kept separately but appears immediately at the output as the aforementioned control signal. That is, if the results of some actions are negative, the aversive emotion will immediately suppress that activity.

2 key blocks of the limbic system are the amygdala and the hippocampus. They are very different and seemingly complement each other. The amygdala is a typical subcortical structure. It is linked to generation of very basic states of the organism as a whole – such as fear and aggression. The hippocampus is the archicortex. This is a small piece of cortical matter which emerged early in evolution and has just 3 layers. Thus it should perform some important functions. An interesting theory of the hippocampus is as follows. What you usually see in illustrations (CA1, CA3, etc.) is a cross-section. The hippocampus as a whole is a tube or more precisely – a long cone in correspondence to its name Cornu Ammonis. Functionally, the hippocampus generates a 1D image that is a vector. The index of this vector goes along the tube while its cross-section provides a pathway for data rotation. Input comes from the entorhinal cortex, output returns back to it. That is, the entorhinal cortex may be a memory buffer which can receive an image from a wide variety of cortical regions, then keep it for temporary dynamical storage via reverberation through the hippocampus. Thus, the hippocampus may work as famous operative memory. Note that it can't store data because it is 1D, while cortical images are usually 2D. Only plays some important role in this storage. In addition to that loop, another major output of the hippocampus goes via the fornix to several small (0D that is scalar) areas of the brain. Through the mammillary bodies, the hippocampus can control the thalamus. It also sends output to the septum (a pleasure zone). There are several such spots in the brain (the substantia nigra, ventral tegmental area). They use the neuromediator dopamine which can control formation of long-term memory.

At last, need to discuss the relation of the brain processor to such important psychological concepts as motivation and consciousness. For humans, motivation is more than just a Start button which sets the goal and launches processing. Human problem solving reduces the main task to subtasks. Accordingly, motivational structures will be activated in the process again and again. There are 2 different types of human motivation – biological needs and goals of behavior. They are generated in the hypothalamus and the prefrontal cortex respectively. The former is a standard part of the limbic system, while some authors include the prefrontal cortex too. In any case, it is linked to the limbic structures.

As to consciousness, this term may be used in different meanings. The most primitive is just the state of awakeness as opposed to sleeping. Accordingly, in the latter case, activational structures of the brain simply switch off perception and data input stops. In a more subtle meaning, consciousness is the ability of the brain to watch itself. This is usually linked to the TPO (Temporal-Parietal-Occipital) zone of the neocortex – an area where 3 main sensory analyzers come together and create a synthetical image of reality. The associative cortex is interlinked with the entorhinal cortex thus receiving information from other parts of the brain and we can look not only at the external world but also at our own internal activity.

Fig. 12.

This is a processor of the brain aka the limbic system. Emotions serve as processor instructions for an associative computer. How does it work? Let's formulate the answer in advance.

The amygdala and hippocampus complement each other. The first generates various states of the brain that are appropriate for a particular situation. The second outputs sharp bursts of activation to trigger these states, switch, or end them.

Theory of automatic regulation helps to decode this overcomplicated pile. According to this approach, the whole of the brain's computer is a supplement to the hypothalamus which in turn is the highest control center for internal organs. The first simplification is to remove the arrow from the amygdala to the hypothalamus. The link in this direction ensures that emotions produce reactions in the body so it is already beyond the psyche. Instead, the reverse link is a key because it conducts activated biological needs and launches generation of appropriate emotions.

Fig. 13.

All in all, the brain processor has 2 major functions: managing dynamic operative memory and suppressing unnecessary associations. The second function works as follows. When some event comes to sensory inputs, it is evaluated by the limbic system. If it is important – passed further, otherwise blocked. For the valuable events, the limbic system (amygdala) determines blocks of associative memory which should be used for processing. Others are inhibited. That is, for each event the amygdala outputs a pattern of active cortical regions. The principle of dynamic memory via recirculation may be used to store sensory images and these emotional patterns too. In this case, the loop is linked within limbic structures.

Both processes (selection of events and selection of associations) are modulated by the input from the hypothalamus and the prefrontal cortex, that is by primary needs and current goals.

Fig. 14. The functional scheme of the brain processor. Blue links represent static signals which provide information about context. They also may be quasistatic. That is, may change gradually so that at any moment some definite state is processed. Red – distinct pulses known in programming as events. Reciprocal connections at both sides of the entorhinal cortex represent dynamic (operative, working) memory. By this means, an image may be retained even when the initial sensory input disappears. Motivation comes to the amygdala from 2 sources – the hypothalamus and the output half of the neocortex.

Software

Software and behavior is not the same. The latter is objective reality while the former is an invisible nonmaterial component which is responsible for generation of this reality. That's what biology could not understand for centuries. Materialism required "objective" methods and tangible evidence, but everybody knows that software is not just an essential part of a computer, but its main part. Any complicated computer without software is dead, simply useless. Research of brain anatomy quickly revealed existence of input and output channels, but what's next? Reflex? Too primitive for the human psyche. The concept of software resolves this problem. Reflex is only the simplest way to link input and output. Real neurocomputers use much more complicated methods.

What is brain software? In a computer, it consists of binary codes representing processor instructions and some data which accompany these codes. Brain operation is manipulating images. They emerge in some parts, migrate to others, are transformed, and interact with each other. To establish correspondence, we can look what happens in the end. In a computer, data usually arrive to a display and are presented as text, charts, or visual images. In the brain, internal images of movements come out to muscles. So abstract images of the nervous system are data being processed, but where are instructions? They are absent. This is possible because the brain uses associative memory which is capable of some processing too. For example, an instruction set of a typical processor has arithmetical operations which are learned by humans only later. Conditional branching is implemented by memory at hardware level in parallel mode and there is no special command for it. The instruction set of the brain (if any) is much smaller than that of computer. Then, how are programs stored? In associations. The nervous system is able to remember sensory or abstract images and associate them with each other.For computers, this is known as the rule-based programming style and is also used in the event-driven paradigm. Programmers associate important events with appropriate event-handlers.

Neuroprogramming

With that said, the picture of brain software as a whole turns out to be very different as well. For a computer with the single processor, talks is about linear programming that is executing a sequence of instructions. For multiprocessing it will be just several lines running in parallel. For humans, this is different. The brain has a complicated internal structure, but consists of simple blocks. Namely – more or less homogeneous neural nets linked by beams of white matter. These nets can learn some patterns of activity which may be triggered by incoming signals. All in all we get a complicated system of non-synchronized rules which work in parallel both at application and system level.

This is only the first approximation. Don't forget that the brain is an analog system. So far, we used discrete concepts which are applicable with a certain precision. If we try to dig into details, various transitional processes emerge. In a computer, signals are represented by sharp pulses of almost rectangular form. In the nervous system, pulses are much longer and have sloped edges. What is essential, this is not just imperfect representation. Deviation from strict discreteness is functionally significant and used time by time. Borrowing from physics, this may be dubbed as "quantum effects" in cybernetics. For example, suppose you drive a car and need to enter a gate. At one moment you notice that you have missed and one side will be 1 centimetre into the post. It is necessary to stop, move backwards, and try again. Now imagine the same situation, only walking on foot. If your shoulder is going to hit the post, you will slightly deflect the body and pass successfully. Quantum tunnelling. Such effects become vitally important in real-world conditions where strict rules would often encounter various obstacles. Without "corrections" and "exclusions" control systems would stop again and again.

Another important detail. The human brain may be described as a finite-state automaton but how to reconcile this with the highly dynamic nature of human behavior? The solution turns out to be simple. In many cases, dynamics is a cyclic activity. For a computer, this is triggered by some event, then maintained by an algorithm. In the brain, also some control signal is involved, but it isn't a short pulse. Instead, it is a long strobe and activity continues while this strobe remains active. Moreover, recall that nervous signals are digital-analog so the strobe may change its amplitude and regulate the strength of output actions or their frequency. Continuing the example with walking, humans don't think about single steps. They are generated by low-level servo control. The next step is automatically launched when the heel hits ground at the end of the previous one. Dynamism of walking is maintained by spinal gear while central control sends just 2 static commands: walk or stop.

After this introduction, let's try and describe human software in details. For a computer, there is a hierarchy of programming languages. The first one was The Language Of Assembler or just Assembler which consists of instructions that are immediately supported by the processor hardware of the computer in question. The next level contains languages of system programming. The most known is C that was conceived as a universal Assembler which should work on different machines. Further up will be application languages to write software for end users and particular application domains. Typically, the languages of lower levels are used to create a programming tool for the next level upwards. That is, when you use a single operator of an application language, it employs several elementary commands and instructions of lower levels. As a result, very complicated and clever programs are composed of a few very simple instructions. Hence, we need to seek for such most elementary internal actions supported by brain hardware.

The most obvious are memory read/write operations. Humans can remember separate images and later recall them. In addition, they can remember associations between 2 images. Then, when the first is presented, heteroassociative memory returns the second.

The next function is evaluation. The limbic system receives input from different sensory channels and generates a positive or negative drive depending on how the current situation compares to expectations. These drives work as start/stop instructions which are important for asynchronous architecture.

Now add various methods of data processing implemented in structured multilayered neural nets and you will get a computer with a decent set of instructions which may be used so as to develop sophisticated programs.

System software and applications

Newborn humans are virtually helpless. Can't walk, even stand. Unable for coordinated actions. Moreover, they don't know elementary actions themselves. On the other hand, it is well known that if a human hadn't learned any language at early age, he can't do it as an adult. Seemingly, human learning is arranged similarly to installation of computer software. An operating system comes at the beginning. Accordingly, you need to learn some basic skills first. Then, the rest will be added on this foundation.

Meanwhile the difference is substantial too. A computer comes with a ready instruction set and electronic circuits for algorithmic processing. Creation of human system software begins with creation of a discrete processor on the basis of the analog neurosubstrate. This firmware operates in accordance with principles outlined in Formal neurocomputer.

The rest of system software constitutes what is known as "culture". It includes such elements as habits to wash your face or clean your teeth, preferred clothes and dishes, usual time when you go to bed and wake up, etc. The most characteristic feature of a culture is its language. As in computers, system software divides into that which supports individual behavior and that for communication and cooperation (networking). Each nation has its unique traditions of interaction.

Finally, when a person has more or less remembered these components, it's time for professional education. That is, installation of application software begins. Computers have many programming languages. Each was developed for a certain class of applications. In humans, a natural language serves as a universal one. Nevertheless, each profession develops its own dialect with special terminology and even a specific style of thinking.

Complete theory of natural language

When we hear "theory" during a discussion of human language, the first idea is grammar. Syntax, punctuation, word spelling, etc. Meanwhile this is only a superficial wrapping. There is also at least semantics which is more important than purely formal grammar, but you won't find a comprehensive course on this discipline. On the other hand, when professional linguists discuss language usage, there is one problem. How to tell a correct construct from incorrect one? For a student, it is simple. Just open a textbook, but how to decide it for a professor who writes these textbooks? It turns out that they can't suggest anything better than intuitive "well-formedness".

Human language is a part of a live computational system. For a neurocomputer, the problem of "well-formedness" becomes quite practical. It may be resolved on the basis of computational efficiency, reliability, informational capacity, and so on. Let's try and formulate the structure of natural language in this aspect. We will keep in mind English because it is a lingua franca and also its fixed word order makes it easy for parsing. Nevertheless, usually speculation should be applicable to other human languages as well.

What is natural language?

Human language is a communication channel for a live computing system. It is very different from common computers. On one hand, knowledge of this system as a whole is very useful for understanding of how this language works. On the other, even if we don't know its inner details, we still can access its functions via the language interface. Existing computers are famous for being universal. On one hand, the human brain is universal too, on the other it is extremely specialized. In fact, it has just the single task – homeostasis. Existence is the ultimate goal of this automatic control system. Variability is needed to adapt to a wide range of living conditions. The whole complexity of human behavior may be logically derived from this root. The overall brain may be represented as 2 regulators – the internal and external one. The first looks inside, the other – outside, but they do the same – maintain the reality in accord with some ideal image. If your blood contains little nutrients, eat something. If your shoes became old and don't protect from water, throw them out and buy new ones. To evaluate the disbalance, we need 2 images – an ideal and real one. Our sensory organs permanently update the second. We don't know exact details of its internal representation but can communicate it via the language channel with any precision required.

Assignment

In computers, we have many different languages, each for a specific purpose. Instead, humans use one language in all of these cases. You can even tell some words which directly affect internal organs or the state of the brain itself. This is an assignment of Assembler – the language for hardware programming. You can code an algorithm like with Basic, formulate a question, that is a query to a database like with SQL, or use it for communication as HTML.

Classical hierarchy of language structure

This is widely known as

Lexicon – Syntax – Semantics – Pragmatics.

The first level contains various words. All of them are divided into several different parts of speech – nouns, adjectives, adverbs, … Syntax is a set of rules which define how these words group into noun phrases, verb phrases, and other constructs. They further group into clauses and sentences that represent elementary complete ideas. The flow of the text represents the flow of ideas in human mind. Note that syntax rules are formulated in grammatical categories. They are meaningless, used only to link words with each other. Some meaning emerges on the next, semantical level when the syntax structure is considered together with concrete words within it. Nevertheless, this meaning is still incomplete. It is called "direct" one. On the last, pragmatical level, context is added and we receive "indirect", derived meaning of the text. Context also plays the crucial role in disambiguation.

This scheme looks perfect, but is it really workable? Grammar was added later in human history. It is mainly associated with written language. Like any theory, it only partially fits the reality of live human communication. A typical cycle for learning a foreign language is studying a grammar textbook, passing the exam, then forgetting it all and taking a practical course of business English. Why such discrepancy?

The problem is that the human brain groups words not according to formal categories of the part of speech, but according to their meaning and takes context into account simultaneously. In fact, there are no sequential levels in the live neurocomputer. Everything happens in a complicated computational system using many different neural nets which operate simultaneously in the parallel mode. Then, is syntax really useful? Maybe we should reduce the previous scheme to Lexicon – Pragmatics?

In the very distant past, the language, and the life itself, was simpler. In the mind of those people word phrases were translated directly into static or dynamic images. Probably word categories were used too, but they were meaningful. Not the noun but the object. Not the verb but the action. With the development of civilization, new features were added. The gerund is a noun-type word derived from the verb. Why not? Can we take a movie, pick one frame from it, and consider this frame as a static object? Yes, of course. The next example is abstract concepts. Take justice. Is it an object or an action? Looks like parts of speech and abstract grammar are necessary indeed.

Language imperfectness

Human language is a product of evolution. Nobody intentionally developed it. Various features were added by different people in different epochs. Some elements used by mathematics may be easily found, but look into the matters and you will discover that their development was simply uncompleted. Moreover, it works on the live computational system which was created using the same principle. The main goal of this language is not precision and efficiency but workability in the quite various, often harsh environment. It is easy and convenient for simple tasks, everyday use by millions of people. If you face complications and want super reliability, it is better to use more formalized tools.

The current state of human languages is the state of overcomplication. Too many features were piled together. Let's consider an example. The normal attribute to the noun is the adjective. What if we want to use another noun for this purpose? The normal way will be 'leg of chair'. For simplicity English allows 'chair leg', but how to agree it with syntax rules? In Russian, there is a simple way to produce an adjective from a noun. Their spelling will be different. In English 2 variants are used. The syntax allows a noun as an attribute to another noun. Alternatively, we can make a double entry into the dictionary. One as a noun, another as an adjective. Both create problems for a parser in computers. Even more problems will be on the semantical level because all semantical rules should be doubled as well.

Semantics of natural language

Basic semantic categories depend both on the structure of the real world and the operation of our perception. They represent what features we extract out of nature.

Language describes different types of reality. This may be external events in the environment, own actions of the speaker, the same actions of another man.

When we learn a language, be it our native one in school or a foreign language, the focus is usually made on grammar. Accordingly, the success of education is evaluated by the number of grammatical errors. Meanwhile, this is not the main goal of communication. If you miss a comma in a sentence, but the reader understands it correctly, no matter. Much worse if the sentence is grammatically correct but meaningless. I would prefer a language where I can freely choose options to express my ideas better rather than permanently fear to make an error.

Let's look how natural language represents meaning. It generates 2D images in the neocortex. The first sentence creates an image. The next – add details. To group words inside the sentence, grammar is used. A Part Of Speech has some generalized semantical load, but is mainly needed in syntax rules.

POS is defined for each word separately by enumeration. This does not obey any rules. Instead, POS itself defines how this word will be used in syntax.

It turns out that live humans use 2 principally different systems of language processing. 1 – intuitive, which you learned as a native spoken language. 2 – grammatical, learned with writing or as a foreign language.

Formal semantics

Semantics of human language may be formalized like it was already done with lexicon and syntax. What elements of meaning is it possible to single out from a text? Words have their own meaning which directly links language to the real world. We will concentrate on the next level of semantics – the meaning of the syntax, that is the meaning which emerges when words interact with each other according to the principle of compositionality in linguistics. The sentence (or the clause of complex sentences) is the smallest complete structure of language. This is enough to represent an idea. How sentences group in the text is a separate question. Let's discuss the meaning of the single sentence now.

Actions and items

3 different types exist: affirmative, interrogative sentences, and orders. The last 2 types are variations of the first one so let's consider semantics of the affirmative sentence. When a person conceives it, this is transformed into some internal image. It may have no details like 'A large air balloon hanged in the sky.' or may be rich in various parts. In this case parts are designated by various phrases of the sentence. The structure usually forms a hierarchy where large-scale parts have further details. At the upper layer, the sentence is divided into the subject phrase and the predicate phrase. Which is the main part of the sentence? Probably, predicate is more preferable. In this case, the whole sentence denotes some action. Static sentences such as 'An apricot is a fruit.' are not an exclusion. Rather, they are a particular case of inaction when nothing changes. If the text is a list of actions, then the whole of it is the answer to the question, "What happens?" A quite reasonable approach to the world and especially to the life with its dynamism.

Other parts of the sentence play certain roles in this action. The subject is an actor, the direct object is an application of this action while the prepositional object is an instrument or any other supplementary part. The roles may vary. If there is no actor in an action, the subject may designate the focus of attention. Note that the term 'object' is used differently in linguistics and programming. The former is a purely formal element of the syntax while the latter is meaningful and may be a very complicated construct. Objects in programming may represent both actions and items.

Now, we need some lexical semantics. Of course, any word has its own meaning, but words fall into several large groups. Verbs usually designate actions, nouns – items. Other words are used so as to build complex constructs. Adjectives denote properties of items. If you add an adjective to a noun, you will create a noun phrase and can add color, dimension, smell, even a texture of the surface to some object. Similarly, adverbs modify actions. The simplest verb phrase is verb + adverb. In addition, it may include other elements. As mentioned above, the subject denotes the main participant of the action. There is also the indirect object in the form of the prepositional phrase (He came with a new book.). The indirect object without the preposition (I gave him a new article.) is ellipsis where the preposition is dropped. An equivalent prepositional variant exists (I gave a new article to him.) so such reduced constructs may be considered derivative, auxiliary. The preposition itself designates a relation. This is especially obvious for spatial prepositions. 'Upon' and 'under' designate a direction (vertical as opposed to horizontal in this case) and also determine what is on top.

Overall semantics of the sentence may be denoted as

predicate(subject_phrase, direct_object, adverb, prepositional_object)

This looks like a function of C programming language or mathematics, right? Sentences of natural language are a powerful tool to describe variability of the analog world in discrete words. They are a subset of all the structures possible in mathematics. Why? Because they represent the internal gear of human perception. If you write a program in a human-like language, the computer will think like a human being.

This is only basics. It was enough for people millennia ago, but further, language evolved and got more and more complicated. This evolution is controversial. On one hand this made it possible to describe more situations, but on the other hand more and more troubles emerged. As new elements were added or old ones found additional usage, it affected the previously perfect composition. Nobody supervised these "amendments". Those who introduced new elements even didn't think about what they do so now we have literally a pile of features which are often not coordinated with each other. If you try to implement them mechanically from the list, the program simply will not work. The main problem for those who want to work with a more close approximation of human language is not only to implement more features separately, but also ensure that they will work in various combinations. Let's try and list these features one by one.

The adverb may modify the adjective. 'very big', 'brightly green'. How is it possible if we defined the adverb as an attribute of the verb? In principle, receptors of the human eye encode both color and brightness so correct expression would be 'bright and green'. 'brightly green' may be shortening from 'a green leaf brightly shines in the sun'.

The action may turn into the item. The verb has even not one form for this. 'To define the concept is the first stage.' 'Defining the concept is the first stage.' Here the infinitive and the gerund are used. This may be easily explained. The action is represented in the brain as a dynamic image, a movie. Take a single picture from this movie and you will have the static item which may represent this action.

The action can also become a property of the noun instead of the adjective. 'Running man'. 'Running' is a participle here and we already see ambiguity with the gerund on the level of word forms which could be easily fixed. Where did creators of the language look? The explanation is in the next paragraph.

The noun may be a property of another noun. 'Animal paw'. This also has a more distinct variant – 'animal's paw' but may be expressed by the basic construct – 'paw of the animal'. There is yet another variant in English. Many words may be nouns and adjectives simultaneously so this feature may be implemented both on lexical and syntax levels.

Auxiliary verbs. 'Be' and 'have' are used in verb tenses and compound predicates, but they retain usual meaning there. In 'Maple is a tree.' 'is a tree' may be interpreted either as a purely formal construct representing a predicate as a whole or as a normal predicate phrase. In the latter case 'tree' will be a direct object. This has deeper semantical issues, but is very convenient programmatically. Nevertheless this also may have problems on the syntax level. 'The lemon is yellow.' The direct object can't consist of the attribute only. A full-scale noun phrase is required. A solution may be that this is ellipsis from 'The lemon is a yellow fruit.'

Different types of actions. In some cases, an action is directed at some object. In others, this object is absent. Accordingly, language distinguishes transitive and intransitive verbs. Reflexive verbs denote a situation when an action performed by some subject is directed at this subject itself. In English, this is encoded by means of the reflexive pronoun (he washed himself) or is not marked at all (he looks nice).

In complex sentences, the simple sentence becomes a clause. It still represents an action, but may play different roles. 'That I have a vacation is convenient now.' The subordinate clause stands in place of the subject. 'I decided that I must take a vacation.' Here, the similar clause is the direct object already. Indeed, the subordinate clause may be any part of the sentence. No problems if the action becomes the item and the item is the attribute.

Compound sentences like 'People stood on the shore, and the ship moved in front of them.' are short lists. A more extended variant is the paragraph of the text.

Affirmative sentences are basic units of a description. They represent knowledge. Questions and orders are related to using this knowledge. The former are queries for information extraction. The latter are used primarily in communication so as to prompt some action rather than transmit data.

Relations

There are 2 main types of relations: cause-consequence and general-particular (or abstract-concrete). The first type links actions. It may be expressed by complex sentences in both directions – with subordinate clauses of cause or goal. Conditional sentences are semantically close. Here the condition is not the reason completely but may be considered as a part of it. Causes and reasons may also be expressed by adverbial modifiers of the simple sentence. As they are made of noun-type words, some transformations are required. To represent the goal, the infinitive may be used. An example is the previous sentence itself. Reasons are often expressed by nouns. This may be interpreted as a reduced form. 'He turned his vehicle because of a man on the road.' 'He turned his vehicle because a man stood on the road.' The first is a simple sentence while the second – the complex sentence with the full subordinate clause of cause.

The concrete-abstract or is-a relation is used to represent conceptual hierarchy. The human neocortex has at least 3 levels of such hierarchy – the photographic image, abstraction in the limits of a given modality (vision, hearing, etc.), and complex multimodal images. Hence this type of semantics may be processed immediately on the hardware level.

Spatial and temporal relations are the next by popularity. They are encoded by various prepositions: on, upon, by, under, before, after, etc. In this case, the prepositional phrase is called the adverbial modifier.

The text

Now that we have determined semantics of the single sentence, let's try to understand how the meaning of a text is composed. In linguistics it is called coherence. This is closely related to the work of human memory and upper levels of perception. What happens when you try to understand the architecture of some building? You will walk around and look at it from different sides. Human memory is able to do interpolations. If you need a picture from some point which you didn't visit, you can easily imagine it. The result of such exploration is a set of images taken from a number of optimal points. The text reproduces this structure, only each image is replaced by a single sentence. The same principle works for processes. In this case, each reference point corresponds to a dot on the time scale so the set should be ordered. An intermediate state of the process may be easily interpolated from 2 neighboring points.

Nowadays, human texts have more complicated structure. Sentences are grouped into a hierarchy of blocks on the principle paragraph – chapter – book – library. Inside each block, elements of lower levels are listed as described above.

In addition, sentences are not just non-related elements of the list. They are interlinked. The mechanism used resembles variables of programming languages, only the implementation is very rudimental. Close analogue of the variable is the pronoun. Their implementation in human language is so poor that binding pronouns to their values became one of the major problems of natural language understanding.

Another method resembles the class-object pair. In programming languages, you should declare a variable of some class, then assign an instance of that class (object) to that variable. Human language doesn't like formalities. You can immediately use the name of a class as a variable with an already bound value. When you read 'apple', this may be either an abstract concept denoting the whole class or a concrete material fruit. Definite vs. indefinite articles may be used so as to distinguish between them. Unfortunately, the number of possible variants is so large that any programmatic implementation should list them explicitly.

The methods to define a concrete object (to create an instance) may vary. You may mention a class in concrete circumstances. 'A cup stands on the table.' Since then, this class turns into the object and you can use this word with the new meaning. Additional details may be added in separate sentences. All of them will be linked to the same object. After that, it is possible to use this object in some action.

Otherwise you may use the relative clause. 'Take the cup that stands on the table.' Here, all is done inside one sentence. There is also a difference in general use. Programming languages use classes mostly to define their methods and properties and to create object instances from them. The program will use that objects afterwards. Human language, in contrast, manipulates classes and concrete objects in equal degree.

Indirect meaning

All these elements represent the direct meaning of a text. There is also an indirect meaning in addition. It is produced by reasoning. For example: "The price of item1 was $10 last month. The price of item1 is $8 now." The conclusion from this short text will be: "The price dropped." Indirect meaning is always pragmatical that is dependent on the situation. For one person one feature of the sentences is important. For the second – another one. Their conclusions may be different. Context-free languages restrict their semantic scope to direct meaning. This is a relatively simple task routinely solved by standard programming languages. What do humans use in addition?

Semantic transformations

Associative memory of neural nets not only stores data but also provides various processing based on similarity immediately on the hardware level. This is widely used during language understanding.

Anaphora

Pronouns are analogous to variables in programming, but their usage is peculiar. In strict languages, you must declare the type of a variable, then assign a value to it; only then you may use this variable in some expression. In natural language, you just use 'it' which refers to some previously mentioned noun. 'A cup stands on the table. It is large and bright.' The reader often guesses which one you keep in mind. Human language has an even more vague but simultaneously more embracing feature. 'it' or 'this' may refer to the whole paragraph describing some phenomenon of any nature possible. It may be a material object, an action, or even a relation.

Metaphor

This is similar to the theory of analogy which is well developed in physics. The principle is that if we have one phenomenon that is perfectly studied and another one which is barely known but has some analogy with the first, then we can transfer the source knowledge to the target. For example, Hinduism tells that our world is not the first, but before it was created, Shiva destroyed the previous one using a swift dance.

Fig. 15. Shiva as Lord of Dance.

The study of metaphors discovers how we process information. The human brain is very efficient for the comparison by similarity. Neural nets do it immediately in associative memory on the hardware level. Accordingly, we have various types of metaphor.

Allegory

This is an extended type such as fables. They usually employ animals or other non-human creatures to illustrate some moral principle. The story is full of details and stimulates profound thinking.

Hyperbole

Underlines some features by means of exaggeration. A typical example of hyperbole is "million reasons".

Parable

Parables are sample stories for educational purposes. They were actively used by Jesus Christ. In contrast to a fable, it excludes animals or inanimate objects representing speaking beings.

Antithesis

Natural language works with fuzzy objects. In such conditions, it is useful to underline not only some features, but also their negations. In the following sample, antithesis is used to underline a paradox: "The better – the worse; the worse – the better."

Metonymy

In this case, a concept is designated by the name which calls a close association. As an example, a crown is a well decorated headwear, but the word is also used for a royal house or power.

Connotation

As you know, songs convey 2 components – text and music. They correspond to information and emotions. Both are well represented in common speech. In pure texts the second is limited but also present as well. For this purpose we use various synonyms. For such words the meaning is similar but not exactly identical. The difference of the second order may convey our attitude to what we write.

Implicature

When we communicate some useful information, formulating everything explicitly would be boring, especially in dynamic circumstances. By this reason, we usually say essentials taking the rest for granted. For example, when one explains the recipe of a soup, he may enumerate the necessary vegetables, but don't mention salt.

Theory of language

At present, there are many different theories for parts of language such as syntax and language as a whole. Consensus is even not visible, but most of them are, in fact, variants of the same or different solutions of some particular question such as what is the main word of the sentence. Hence, we can formulate a generalized theory taking the best ideas from different approaches or choosing the most practical method for particular problems.

When you create some science from scratch, finding solutions of key problems is the second step. The first – formulating these problems and defining appropriate terms for them.

Natural language as a whole is a communication system for the transmission of 2D cortical images over an 1D sequential channel. Basically, 1 sentence = 1 image. The next sentence either adds details to the existing image or creates a new one. The main problem is how to group words inside the sentence. For this purpose, civilization adds grammar. The Part Of Speech (POS) is an entirely artificial concept. In fact, syntax and punctuation introduce an intermediate level of processing. POS + syntax rules define grouping.

The standard pipeline of language processing is

POS -> Syntax -> Semantics -> Pragmatics

The question is how different steps (levels) interact with each other. A popular principle is that syntax should be self-sufficient, that is independent of adjacent levels from both sides. Word grouping should proceed without words themselves. Only their POS is needed. Also parsing of the sentence should be completed before proceeding to semantic analysis. It is on this next stage that words fill the parse frame. The final meaning is created on the last, pragmatical level taking into account the other sentences of the text.

Unfortunately, this is only a good wish which is possible only for very strict artificial languages. Even for moderately restricted natural language it is impossible. Most words may be several different POS. Accordingly, for each sentence several parse structures are possible. Humans choose one by meaning. For this purpose, the system of analysis should have full backtracking through the whole pipeline.

Another problem is workability of syntax itself. If we take standard English, it has wide variety of phrases on the sub-sentence level. Even filled with real words, such phrases may create an ambiguous construct. The probability will be only higher if we consider POS only. The longer a sentence, the more phrases it contains, the less reliable it is.

Finally, alongside generalized syntax, humans widely use various expressions based on concrete words. Such expressions may form the whole sentence or only a phrase within it.

Human language is like a programming language without automatic error checking. It is up to the users to reduce ambiguity. Human language is very redundant so there are plenty of abilities. Don't attach too many POS to a single word. Say, English syntax allows a noun as an attribute to another noun. Hence no need to declare it as an adjective in the dictionary. Don't use long sentences with several clauses. Break them down to simple sentences. If you see that some construct is ambiguous, replace it. Usually there are several ways to say the same thing.

Computing

A text supplies us with some knowledge that we use later for practical purposes. Which specifically? The term of intelligence may be defined as the ability of problem solving. That's why knowledge accumulation is needed. Let's analyze in details how it happens.

Syntax

Separate words group into phrases, clauses, and sentences. Then into paragraphs, chapters, books, and whole libraries. The structure over the sentence is less standardized. An encyclopedia is like a library, only the latter may contain several books by different authors on the same topic. In the encyclopedia they are concentrated into a single article.

Syntax is a completely artificial formal system. Ideally, it should be detached from both lexicon and semantics. Word grouping should depend only on the part of speech. In real languages, there are lots of exclusions from this principle, but even without them formal grammars are problematic. Let's look into details of these problems.

1. How to represent the structure of the sentence on the very top level? The popular answer is

sentence(subject phrase, predicate phrase)

Both arguments are equal here. Alternatively, one of them is considered the main. If this is the predicate, human sentences become compatible with formal logic

predicate(subject phrase, predicate phrase)

Here the subject will be just one argument of the predicate (in logical sense) alongside the direct object and the other members of the sentence. Also the semantic load of this representation is clear. In this case, each sentence represents some action and the whole text answers the question: "What happens around?"

2. During word grouping, the process passes a hierarchy. Different textbooks present it differently. Some levels may be absent. At the first step, various phrases of the lexical level are recognized. These are: noun phrase, verb phrase, adjective phrase, adverb phrase, prepositional phrase.

At the second, they create phrases of the sentence level: subject phrase, predicate phrase. Secondary members of the sentence, such as the direct object, are usually not single words but whole lexical phrases. Note that the same noun phrase may become either a direct object or a subject.

A clause is like a simple sentence, only as a part of a complex or compound sentence.

Semantics

When syntax analysis is completed, semantics is available by taking into account not only parts of speech, but the words themselves. Meanwhile some part of meaning may be restored already from the syntax structure. Usually a noun corresponds to some object, a verb – to an action. Of course, there are different variants too, but all of them may be explicitly enumerated. Then, we will have a general description of semantics in possible details. If such a description is implemented programmatically, it is enough to supply some dictionary and the program will correctly understand any text composed of these words.

Some addition to the previous Formal semantics

While conjunctions represent relations between clauses, that is actions, prepositions – between objects represented by noun phrases. Generally, there are 2 of them. The second immediately follows the preposition, but what is the first? If the prepositional phrase stands after the subject, it is linked to this subject. If it is a part of the predicate phrase, there are several variants. It may be a prepositional object. 'We spoke about computers.' An adverbial modifier. 'He lives in this town.' Also, it may be an attribute to some noun in the predicate phrase. 'He moved to the table in the corner.' That is the table which stands in the corner.

In the first 2 cases, the first element of the relation expressed by the preposition is the subject. In the third, the whole construct is ellipsis of some relative clause as explained for this particular example.

Long sentences using complicated structures of modern English syntax may usually be dissected into a set of short simple sentences. After that transformation, it becomes clear how humans process such constructs. For the last example, if there are 2 or more tables in the room, we need clarification. Which one? The sentence may be rewritten. 'A table stands in the corner. He moved to this table.'

How do we use this knowledge? There are 2 typical applications: question answering and problem solving. In the first case, the system will retrieve existing data or derive the answer using its inference engine.

In the second, knowledge is used to find an algorithm of solution. Suppose a service robot has got the order: "Bring a bottle of Cola to me." The machine needs to know where it is. The knowledge base contains records which tell that Cola is in the refrigerator which stands in the kitchen. This is enough to calculate the destination for the navigation system.

Words on the sub-sentence level group according to syntax, but also have semantic roles. The same principle works on higher levels. When we read a text sentence by sentence, they produce some ideas in our mind. They may be remembered for later use or exist temporarily just for the process of reading. These ideas may belong to a few different semantic categories.

Facts

If the sentence has a predicate, it denotes some action. 'The corporation A purchased the startup B.' Although, actions may be static. In this case, they denote a state of some object. 'A pear hangs on the tree.' Such sentences immediately add details to the existing picture of the world. Before we remember them, they usually pass the filter of validity. Some facts may be just impossible so should be rejected. Some – negligible. Don't pay attention.

Explanation

If the fact is improbable but still valid, it needs explanation which opens the filter. Oftentimes such facts become especially valuable. Explanation usually reveals the reasons of why it became possible and, as a rule, is intended for temporary use. Although, in thorough reading the person may analyze and remember something from the explanation so as to use it later in similar cases.

Proof

This is a more formalized variant of explanation. Usually it employs logical inference and derives the result from known facts and rules.

Conclusion

Human logics uses rules. They have various facts as conditions in their body. These facts may be either present explicitly in the knowledge base or derived from other rules. Sometimes the needed logical inference may be lengthy so a text may formulate the conclusion for remembering. After that, it becomes a new fact which does not require proof.

Detailing

The first sentence of a paragraph or the heading of a chapter usually creates a new image in perception. The next sentences add details to this image. The next paragraph of the chapter adds the whole of a new image inside the larger image.

Nucleus Language

The nucleus language is a minimally sufficient set of elements for general-purpose communication.

Being explicitly formulated, it has wide variety of applications: human-computer communication, normalization of human texts, language teaching …

A real language consists of this nucleus + general-purpose extension (lexical and syntax synonymy, rare processing functions, etc.) + professional (special) languages.

The elements included are quite different. The most known component is lexicon – some minimal dictionary of the most used words. At the syntax level, these elements are represented by non-terminal nodes of the parse tree which reside between 'Sentence' and 'Word'. Syntax is about word grouping, but this process goes through a few intermediate levels. For English, the hierarchy is: Word -> Phrase -> Clause -> Sentence. The categories may be related to separate words or to sentences. Accordingly, grammar may use parts of speech (noun, verb …) or members of the sentence (subject, predicate …). These categories are not synonyms. Just substantially overlap, but there are differences too. A noun phrase may serve as a subject phrase or direct object. The prepositional phrase requires a separate discussion.

Syntax is not completely detached from semantics. Indeed, parsing using just parts of speech, not words themselves, is preferable, but even parts of speech still have some generalized semantic load. Both prepositions and conjunctions represent some relations. Prepositions – between nouns (or equivalent words). Conjunctions link homogeneous members of the sentence or clauses in compound and complex sentences. Prepositions represent physical while conjunctions – logical relations. Formal logic turned conjunctions of human language into logical functions.

Accordingly, using 'prepositional phrases' is not semantically correct. Each preposition usually requires 2 nouns like in 'a cat with the long tail' -> with(cat, long tail). This draws far-reaching consequences for the whole structure of the sentence. The predicate phrase can contain a prepositional object. 'He took the bread with butter.' -> with(bread, butter) What if there is no direct object? 'He came with his friend.' -> with(he, his friend) In this case, the preposition links its object with the subject. This is not a well formed solution because objects are parts of the predicate phrase and the link goes over the predicate. We can resolve this issue if we abandon the traditional structure sentence(subject phrase, predicate phrase) and replace it with more logicist

predicate(subject, indirect object, direct object, prepositional object, adverbials)

Semantics of such a format is clear. Each sentence represents some action. If nothing changes like in: 'The box stands on the table.', this is just a static action. In any case, the picture is a function of time. The subject is different from objects only by attracting more attention. It is even not required that it should be the actor. 'The apple was taken by his friend.' In passive voice, the subject is the word with the semantic role of the direct object. This is because attention is directed at this object.

Such generalized semantics independent of particular words may be implemented directly in the program while semantical properties of different words require some dictionary. All in all, the following semantically significant elements may be distinguished in the simple sentence. 1. The predicate (verb). 2. Various noun phrases. 3. Adverbs which are not a part of some noun phrase where they modify an adjective. Such adverbs modify the verb. 4. Prepositions and conjunctions. The former represent relations between noun phrases. The latter – between clauses. They also may link homogeneous members of the sentence. Conjunctions work like logical functions in a list.

predicate(subject phrases, prepositional phrases, adverbs) conjunction predicate() …

Besides these basic elements, there are many derivatives. 1. The gerund is a noun-type word created from a verb. 2. The verb also can produce forms which function as an adjective or an adverb. These are participles.

The first noun phrase is the subject. This category should be retained if we want to answer standard human questions. The indirect object is redundant. It may be replaced by the corresponding prepositional object. 'I gave him an apple.' -> 'I gave an apple to him.' The second variant is more reliable for parsing. Only it should be present in the nucleus language. Adverbials may be represented either by a prepositional phrase or by an adverb. The first is often difficult to distinguish from the prepositional object. 'I came with my friend.' 'With' in the prepositional object. 'I lifted it with my hands.' The same in the adverbial of manner. This can be cleared only from low-level semantics. That is, regarding particular words involved. In some cases, it is impossible to distinguish the roles at all. Both are applicable so leaving prepositional phrase in the representation is justified. Then, its semantics should be computed when it is used in inference or question answering. Finally, conjunctions represent relations between simple sentences as clauses.

You see, there are quite a few basic elements. One difference of natural language from programming languages is its universality. It is used both for abstract computations and machine-level programming. Probably these elements represent some very basic operations performed by neural nets. For example, a noun phrase may stand for an image which is a part of a larger construct while prepositions and conjunctions define how these constructs are built of such images. Details of the procedure are defined by the verb in question. Say, if it is intransitive, there won't be any direct object at all. If the verb is transitive and the object is present, the details of object's handling are defined by this verb.

The nucleus language should contain only the most necessary items. On the other hand, human sentences may be very extended. Obviously, they are being broken down to more primitive ones during understanding.

Abstract semantics represents various functions which our live neural nets perform when we process textual data. Being explicitly listed and properly arranged, they may serve as ready terms of reference for software development.

Semantics is hierarchical. There is meaning associated with particular words and there are generalized categories. Let's list them explicitly.

Natural language actively uses attributes. Normally, adjectives modify nouns and adverbs – verbs. However modern languages add many extensions. Adverbs can also modify adjectives. A noun may be an attribute for another noun. Normally, this is expressed by a prepositional phrase after it. 'the leg of a chair', 'a cat with the tail'. In addition, English can put it in front of the noun. 'the chair's leg' or even 'the chair leg'. The exact semantical relation between the attribute and its object is defined by the preposition used. It is necessary to note that the last sample may be modelled also using the dictionary. The noun before another noun may be declared as an adjective. In this case the trouble just moves from syntax to the lexical level. Best of all – to avoid such constructs because they make parsing inefficient.

Gerund is a noun-type word derived from a verb. Interesting that it retains the structure of a verb phrase which begins functioning as a noun phrase. The verb can also produce forms which operate like adjectives or adverbs. These are participles. What is the meaning of this variety? If the verb is represented in neural nets as a movie, then gerund is a snapshot from this movie. The action as an attribute of a noun (a running man) means that this noun participates in this action. The inversion is similar to the passive voice and serves the same goal – to switch attention from one member of the sentence to another. In this case – from the verb to the noun. The action as an adverb, that is an attribute of another action, ('he did it laughing') means that the first action is a part of the second.

Passive voice may be easily explained if we take the predicate as the main member of the sentence and the subject as just one object which gets maximal attention. Passive voice switches this attention. It becomes indispensable when the subject is absent in the active variant. 'The precedent was created.'

In complex sentences, a subordinate clause takes place of one member of the sentence and accordingly plays its role. In compound sentences, coordinated clauses are linked with a conjunction which works as a logical function. This makes it possible to compare natural language with mathematics. You see that formal logics added parentheses which group members of the list and made semantics of functions (and, or, not) absolutely definite (which is not the case in human communication). On the other hand, the list of natural conjunctions is not limited to these 3 ones. Accordingly, the rest of them (such as 'but') may have similar, but slightly different semantics.

The sentence represents some action denoted by its verb with several objects denoted by noun phrases as participants. These objects are linked by relations denoted by prepositions. In principle, this method of constructing a complex object with prepositions may be used without any verb at all. 'A cup on the table'. 'A box under the chair'. 'Instruments in the box'. This renders a quite definite scene. A normal sentence may be regarded as a bag (container represented by the verb) full of such relations.

Homogeneous members of the sentence present a major difference from programming languages. In the latter, some variables may be declared as lists. In natural language, most of members of the sentence are lists by default. This is normal because, say, each noun usually has several attributes. If just one of them is explicitly specified, this is a list with just one element.

In mathematics, variables are placeholders used for 2 different purposes. In equations, they denote unknown values and work for output. In functions, they represent input parameters. In programming, such parameters may be passed by value or by reference. In the latter case, the placeholder works as a link in hypertext. Similar methods may be discovered in natural language.

Pronouns 'it' or 'this' are usually function as a short-distance reference. This case shows clear difference in overall approach of programming and natural language. The last has no strict rules of reference. Oftentimes it is ambiguous and confusing. Meanwhile it may be very useful. For example, a whole paragraph may be devoted to the description of some object. Later, this object may be used via 'this'.

Another method employs generic names. In Object Oriented Programming, the term object denotes a definite data structure while its generalized description is called a class. Before using functionality of a class, you must create some object. Natural language is less restrictive. 'The table' refers to the whole class and you can use it in a sentence. You can use 'this table' so as to select one particular object from this class. You can also use filters to create a subclass 'tables which were purchased for this office'.

Modern languages can generate very long, complicated sentences, but in most cases they may be reduced to a list of simple sentences. The aforementioned variables are used to link them. 'Tables which were purchased for this office arrive to the back door of the building.' -> '20 tables were purchased for this office. The office building has a back door. The tables arrive to the door.' Here the definite articles turn generic names into particular objects. An understanding machine may create a separate internal variable for each more or less significant object of the text. Later use this variable in the semantic representation or update the description of the object.

Copula verbs represent the unique static action, that is a state when nothing changes. Its main verb – 'is'. In principle its syntax is not very different from other verbs. 'He is a sportsman.' Here 'a sportsman' may be parsed as a direct object, but semantically it is very different. This represents conceptual hierarchy, that is what is called class derivation in OOP. Yet another semantics requires a syntax unknown for normal verbs. 'The apple is yellow.' Here the predicate phrase consists only of the adjective. This is equivalent to the sentence without any subject: 'Yellow apple.' with stress on the noun rather than the adjective. It is used so as to set attributes of an object.

Other copula verbs are: become, grow, get, turn, appear, look, seem, etc. The first in this list denote the transitional action of switching the state. The last adds semantics of perception. 'Is' denotes the absolute truth. The other verbs acknowledge that this is only as we perceive it. The last even expresses some doubt. Such samples only underline that our brain is an analog computer which allows infinite variations.