In this paper the principle of minimum relative entropy (PMRE) is proposed as a fundamental principle and idea that can be used in the field of AGI. It is shown to have a very strong mathematical foundation, that it is even more fundamental then Bayes rule or MaxEnt alone and that it can be related to neuroscience. Hierarchical structures, hierarchies in timescales and learning and generating sequences of sequences are some of the aspects that Friston (Fri09) described by using his free-energy principle. These are aspects of cognitive architectures that are in agreement with the foundations of hierarchical memory prediction frameworks (GH09). The PMRE is very similar and often equivalent to Friston's free-energy principle (Fri09), however for actions and the defi nitions of surprise there is a diff erence. It is proposed to use relative entropy as the standard definition of surprise. Experiments have shown that this is currently the best indicator of human surprise (IB09). The learning rate or interestingness can be defi ned as the rate of decrease of relative entropy, so curiosity can then be implemented as looking for situations with the highest learning rate.
The principle of minimum relative entropy is proposed as a general fundamental principle that could be used by the brain to do inference and update beliefs about the world. It originates from information and probability theory, but we relate it it to the brain, to the concept of surprise and to a minimum free-energy principle that has already been proposed for the brain. The measure of surprise that is based on relative entropy (Bayesian surprise) is compared with another definition of surprise (Shannon surprise) that is used by Friston for a minimum free-energy principle. Theoretical and experimental justifi cations are given to propose to use Bayesian surprise as a better and more natural definition of surprise. It can be used as a novel way to quantify surprise or related concepts in developmental robotics. This can then be used in implementations of intrinsic motivations like curiosity to drive exploration, interactive learning and autonomous mental development.
Background: Lexical access problems of inflected verbs are common in aphasia. Previous research addressed these problems either in purely linguistic terms (e.g., verb movement) or in terms of lexical characteristics (e.g., frequency). We propose a new measure of verb complexity, which combines linguistic and lexical characteristics and is formulated in terms of Shannon’s information theory. Aims: We aim to explore the complexity of individual verbs and verb paradigms and its effect on lexical access, both in unimpaired people and people with aphasia (PWA). We apply information theory to investigate the impact of verb complexity on reaction time (RT) for lexical decision. Methods & Procedures: 20 non-fluent aphasic subjects and 11 age-matched and education-matched peers performed an auditory lexical decision task containing 286 real and 286 phonotactically legal non-word past tense forms (regulars and irregulars). RTs and error rates were measured. Two information-theoretic measures were calculated: inflectional entropy (reflecting probabilistic variability of forms within a given verbal family) and information load (I) (reflecting complexity of an individual verb form). The effect for these and other more traditional measures on RT were measured. Outcomes & Results: Linear mixed model analyses to the data for each group with participant and verb as crossed random effects were performed. Results show that for all groups inflectional entropy had a facilitatory effect on RT. There was a group effect for inflectional entropy indicating that for the patients with aphasia the effect of inflectional entropy was less pronounced. At the same time, I did correlate with latencies for healthy adults but not for individuals with aphasia. Conclusions: Our results demonstrate that the decrease in lexical processing capacity characteristic for PWA has a measurable effect that can be calculated using information theoretical means. According to our model, these individuals have particular difficulties with processing lexical items of higher complexity, as measured by individual I, and benefit less from the support normally provided (in comprehension) by other members of the corresponding lexical network. Finally, the proposed information-theoretic complexity measures, which encompass both frequency effects and linguistic parameters, provide a superior measure of lexical access, and have a better explanatory power for the analyses of access problems found in non-fluent aphasia, compared to analyses based on frequency only.
LINK
Despite the benefits of the widespread deployment of diverse Internet-enabled devices such as IP cameras and smart home appliances - the so-called Internet of Things (IoT) has amplified the attack surface that is being leveraged by cyber criminals. While manufacturers and vendors keep deploying new products, infected devices can be counted in the millions and spreading at an alarming rate all over consumer and business networks. The objective of this project is twofold: (i) to explain the causes behind these infections and the inherent insecurity of the IoT paradigm by exploring innovative data analytics as applied to raw cyber security data; and (ii) to promote effective remediation mechanisms that mitigate the threat of the currently vulnerable and infected IoT devices. By performing large-scale passive and active measurements, this project will allow the characterization and attribution of compromise IoT devices. Understanding the type of devices that are getting compromised and the reasons behind the attacker’s intention is essential to design effective countermeasures. This project will build on the state of the art in information theoretic data mining (e.g., using the minimum description length and maximum entropy principles), statistical pattern mining, and interactive data exploration and analytics to create a casual model that allows explaining the attacker’s tactics and techniques. The project will research formal correlation methods rooted in stochastic data assemblies between IoT-relevant measurements and IoT malware binaries as captured by an IoT-specific honeypot to aid in the attribution and thus the remediation objective. Research outcomes of this project will benefit society in addressing important IoT security problems before manufacturers saturate the market with ostensibly useful and innovative gadgets that lack sufficient security features, thus being vulnerable to attacks and malware infestations, which can turn them into rogue agents. However, the insights gained will not be limited to the attacker behavior and attribution, but also to the remediation of the infected devices. Based on a casual model and output of the correlation analyses, this project will follow an innovative approach to understand the remediation impact of malware notifications by conducting a longitudinal quasi-experimental analysis. The quasi-experimental analyses will examine remediation rates of infected/vulnerable IoT devices in order to make better inferences about the impact of the characteristics of the notification and infected user’s reaction. The research will provide new perspectives, information, insights, and approaches to vulnerability and malware notifications that differ from the previous reliance on models calibrated with cross-sectional analysis. This project will enable more robust use of longitudinal estimates based on documented remediation change. Project results and methods will enhance the capacity of Internet intermediaries (e.g., ISPs and hosting providers) to better handle abuse/vulnerability reporting which in turn will serve as a preemptive countermeasure. The data and methods will allow to investigate the behavior of infected individuals and firms at a microscopic scale and reveal the causal relations among infections, human factor and remediation.
