Inference

Inferences are steps in reasoning, moving from premises to logical consequences; etymologically, the word infer means to "carry forward". Inference is theoretically traditionally divided into deduction and induction, a distinction that in Europe dates at least to Aristotle (300s BCE). Deduction is inference deriving logical conclusions from premises known or assumed to be true, with the laws of valid inference being studied in logic. Induction is inference from particular premises to a universal conclusion. A third type of inference is sometimes distinguished, notably by Charles Sanders Peirce, distinguishing abduction from induction, where abduction is inference to the best explanation.

Various fields study how inference is done in practice. Human inference (i.e. how humans draw conclusions) is traditionally studied within the field of cognitive psychology; artificial intelligence researchers develop automated inference systems to emulate human inference. Statistical inference uses mathematics to draw conclusions in the presence of uncertainty. This generalizes deterministic reasoning, with the absence of uncertainty as a special case. Statistical inference uses quantitative or qualitative (categorical) data which may be subject to random variations.

Definition

The process by which a conclusion is inferred from multiple observations is called inductive reasoning. The conclusion may be correct or incorrect, or correct to within a certain degree of accuracy, or correct in certain situations. Conclusions inferred from multiple observations may be tested by additional observations.

This definition is disputable (due to its lack of clarity. Ref: Oxford English dictionary: "induction ... 3. Logic the inference of a general law from particular instances.") The definition given thus applies only when the "conclusion" is general.

Two possible definitions of "inference" are:

  1. A conclusion reached on the basis of evidence and reasoning.
  2. The process of reaching such a conclusion.

Examples

Example for definition #1

Ancient Greek philosophers defined a number of syllogisms, correct three part inferences, that can be used as building blocks for more complex reasoning. We begin with a famous example:

  1. All humans are mortal.
  2. All Greeks are humans.
  3. All Greeks are mortal.

The reader can check that the premises and conclusion are true, but logic is concerned with inference: does the truth of the conclusion follow from that of the premises?

The validity of an inference depends on the form of the inference. That is, the word "valid" does not refer to the truth of the premises or the conclusion, but rather to the form of the inference. An inference can be valid even if the parts are false, and can be invalid even if some parts are true. But a valid form with true premises will always have a true conclusion.

For example, consider the form of the following symbological track:

  1. All meat comes from animals.
  2. All beef is meat.
  3. Therefore, all beef comes from animals.

If the premises are true, then the conclusion is necessarily true, too.

Now we turn to an invalid form.

  1. All A are B.
  2. All C are B.
  3. Therefore, all C are A.

To show that this form is invalid, we demonstrate how it can lead from true premises to a false conclusion.

  1. All apples are fruit. (True)
  2. All bananas are fruit. (True)
  3. Therefore, all bananas are apples. (False)

A valid argument with a false premise may lead to a false conclusion, (this and the following examples do not follow the Greek syllogism):

  1. All tall people are French. (False)
  2. John Lennon was tall. (True)
  3. Therefore, John Lennon was French. (False)

When a valid argument is used to derive a false conclusion from a false premise, the inference is valid because it follows the form of a correct inference.

A valid argument can also be used to derive a true conclusion from a false premise:

  1. All tall people are musicians. (Valid, False)
  2. John Lennon was tall. (Valid, True)
  3. Therefore, John Lennon was a musician. (Valid, True)

In this case we have one false premise and one true premise where a true conclusion has been inferred.

Example for definition #2

Evidence: It is the early 1950s and you are an American stationed in the Soviet Union. You read in the Moscow newspaper that a soccer team from a small city in Siberia starts winning game after game. The team even defeats the Moscow team. Inference: The small city in Siberia is not a small city anymore. The Soviets are working on their own nuclear or high-value secret weapons program.

Knowns: The Soviet Union is a command economy: people and material are told where to go and what to do. The small city was remote and historically had never distinguished itself; its soccer season was typically short because of the weather.

Explanation: In a command economy, people and material are moved where they are needed. Large cities might field good teams due to the greater availability of high quality players; and teams that can practice longer (weather, facilities) can reasonably be expected to be better. In addition, you put your best and brightest in places where they can do the most good—such as on high-value weapons programs. It is an anomaly for a small city to field such a good team. The anomaly (i.e. the soccer scores and great soccer team) indirectly described a condition by which the observer inferred a new meaningful pattern—that the small city was no longer small. Why would you put a large city of your best and brightest in the middle of nowhere? To hide them, of course.

Incorrect inference

An incorrect inference is known as a fallacy. Philosophers who study informal logic have compiled large lists of them, and cognitive psychologists have documented many biases in human reasoning that favor incorrect reasoning.

Applications

Inference engines

AI systems first provided automated logical inference and these were once extremely popular research topics, leading to industrial applications under the form of expert systems and later business rule engines. More recent work on automated theorem proving has had a stronger basis in formal logic.

An inference system's job is to extend a knowledge base automatically. The knowledge base (KB) is a set of propositions that represent what the system knows about the world. Several techniques can be used by that system to extend KB by means of valid inferences. An additional requirement is that the conclusions the system arrives at are relevant to its task.

Prolog engine

Prolog (for "Programming in Logic") is a programming language based on a subset of predicate calculus. Its main job is to check whether a certain proposition can be inferred from a KB (knowledge base) using an algorithm called backward chaining.

Let us return to our Socrates syllogism. We enter into our Knowledge Base the following piece of code:

mortal(X) :- 	man(X).
man(socrates). 

( Here :- can be read as "if". Generally, if P Q (if P then Q) then in Prolog we would code Q:-P (Q if P).)
This states that all men are mortal and that Socrates is a man. Now we can ask the Prolog system about Socrates:

?- mortal(socrates).

(where ?- signifies a query: Can mortal(socrates). be deduced from the KB using the rules) gives the answer "Yes".

On the other hand, asking the Prolog system the following:

?- mortal(plato).

gives the answer "No".

This is because Prolog does not know anything about Plato, and hence defaults to any property about Plato being false (the so-called closed world assumption). Finally ?- mortal(X) (Is anything mortal) would result in "Yes" (and in some implementations: "Yes": X=socrates)
Prolog can be used for vastly more complicated inference tasks. See the corresponding article for further examples.

Semantic web

Recently automatic reasoners found in semantic web a new field of application. Being based upon description logic, knowledge expressed using one variant of OWL can be logically processed, i.e., inferences can be made upon it.

Bayesian statistics and probability logic

Philosophers and scientists who follow the Bayesian framework for inference use the mathematical rules of probability to find this best explanation. The Bayesian view has a number of desirable features—one of them is that it embeds deductive (certain) logic as a subset (this prompts some writers to call Bayesian probability "probability logic", following E. T. Jaynes).

Bayesians identify probabilities with degrees of beliefs, with certainly true propositions having probability 1, and certainly false propositions having probability 0. To say that "it's going to rain tomorrow" has a 0.9 probability is to say that you consider the possibility of rain tomorrow as extremely likely.

Through the rules of probability, the probability of a conclusion and of alternatives can be calculated. The best explanation is most often identified with the most probable (see Bayesian decision theory). A central rule of Bayesian inference is Bayes' theorem.

Non-monotonic logic

[1]

A relation of inference is monotonic if the addition of premises does not undermine previously reached conclusions; otherwise the relation is non-monotonic. Deductive inference is monotonic: if a conclusion is reached on the basis of a certain set of premises, then that conclusion still holds if more premises are added.

By contrast, everyday reasoning is mostly non-monotonic because it involves risk: we jump to conclusions from deductively insufficient premises. We know when it is worth or even necessary (e.g. in medical diagnosis) to take the risk. Yet we are also aware that such inference is defeasible—that new information may undermine old conclusions. Various kinds of defeasible but remarkably successful inference have traditionally captured the attention of philosophers (theories of induction, Peirce's theory of abduction, inference to the best explanation, etc.). More recently logicians have begun to approach the phenomenon from a formal point of view. The result is a large body of theories at the interface of philosophy, logic and artificial intelligence.

See also

References

  1. ^ Fuhrmann, André. Nonmonotonic Logic (PDF). Archived from the original (PDF) on 9 December 2003.

Further reading

Inductive inference:

Abductive inference:

Psychological investigations about human reasoning:

External links

Abductive reasoning

Abductive reasoning (also called abduction, abductive inference, or retroduction) is a form of logical inference which starts with an observation or set of observations then seeks to find the simplest and most likely explanation for the observations. This process, unlike deductive reasoning, yields a plausible conclusion but does not positively verify it. Abductive conclusions are thus qualified as having a remnant of uncertainty or doubt, which is expressed in retreat terms such as "best available" or "most likely". One can understand abductive reasoning as inference to the best explanation, although not all uses of the terms abduction and inference to the best explanation are exactly equivalent.In the 1990s, as computing power grew, the fields of law, computer science, and artificial intelligence research spurred renewed interest in the subject of abduction.

Diagnostic expert systems frequently employ abduction.

Bayesian inference

Bayesian inference is a method of statistical inference in which Bayes' theorem is used to update the probability for a hypothesis as more evidence or information becomes available. Bayesian inference is an important technique in statistics, and especially in mathematical statistics. Bayesian updating is particularly important in the dynamic analysis of a sequence of data. Bayesian inference has found application in a wide range of activities, including science, engineering, philosophy, medicine, sport, and law. In the philosophy of decision theory, Bayesian inference is closely related to subjective probability, often called "Bayesian probability".

Bayesian network

A Bayesian network, Bayes network, belief network, decision network, Bayes(ian) model or probabilistic directed acyclic graphical model is a probabilistic graphical model (a type of statistical model) that represents a set of variables and their conditional dependencies via a directed acyclic graph (DAG). Bayesian networks are ideal for taking an event that occurred and predicting the likelihood that any one of several possible known causes was the contributing factor. For example, a Bayesian network could represent the probabilistic relationships between diseases and symptoms. Given symptoms, the network can be used to compute the probabilities of the presence of various diseases.

Efficient algorithms can perform inference and learning in Bayesian networks. Bayesian networks that model sequences of variables (e.g. speech signals or protein sequences) are called dynamic Bayesian networks. Generalizations of Bayesian networks that can represent and solve decision problems under uncertainty are called influence diagrams.

Charvaka

Charvaka (IAST: Cārvāka), originally known as Lokāyata and Bārhaspatya, is the ancient school of Indian materialism. Charvaka holds direct perception, empiricism, and conditional inference as proper sources of knowledge, embraces philosophical skepticism and rejects Vedas, Vedic ritualism, and supernaturalism.Ajita Kesakambali is credited as the forerunner of the Charvakas, while Brihaspati is usually referred to as the founder of Charvaka or Lokāyata philosophy. Much of the primary literature of Charvaka, the Barhaspatya sutras (ca. 600 BCE), are missing or lost. Its teachings have been compiled from historic secondary literature such as those found in the shastras, sutras, and the Indian epic poetry as well as in the dialogues of Gautama Buddha and from Jain literature.One of the widely studied principles of Charvaka philosophy was its rejection of inference as a means to establish valid, universal knowledge, and metaphysical truths. In other words, the Charvaka epistemology states that whenever one infers a truth from a set of observations or truths, one must acknowledge doubt; inferred knowledge is conditional.Charvaka is categorized as a heterodox school of Indian philosophy. It is considered an example of atheistic schools in the Hindu tradition.

Circumstantial evidence

Circumstantial evidence is evidence that relies on an inference to connect it to a conclusion of fact—such as a fingerprint at the scene of a crime. By contrast, direct evidence supports the truth of an assertion directly—i.e., without need for any additional evidence or inference.

On its own, circumstantial evidence allows for more than one explanation. Different pieces of circumstantial evidence may be required, so that each corroborates the conclusions drawn from the others. Together, they may more strongly support one particular inference over another. An explanation involving circumstantial evidence becomes more likely once alternative explanations have been ruled out.

Circumstantial evidence allows a trier of fact to infer that a fact exists. In criminal law, the inference is made by the trier of fact in order to support the truth of an assertion (of guilt or absence of guilt).

Reasonable doubt is tied into circumstantial evidence as circumstantial evidence is evidence that relies on an inference, and reasonable doubt was put in place so that the circumstantial evidence against someone in a criminal or civil case must be enough to acquit someone fairly. Reasonable doubt is described as the highest standard of proof used in court and means that there must be clear and convincing evidence of what the person has done. Therefore, the circumstantial evidence against someone may not be enough but it can contribute to other decisions made concerning the case.Testimony can be direct evidence or it can be circumstantial. For example, a witness saying that she saw a defendant stab a victim is providing direct evidence. By contrast, a witness who says that she saw the defendant enter a house, that she heard screaming, and that she saw the defendant leave with a bloody knife gives circumstantial evidence. It is the necessity for inference, and not the obviousness of a conclusion, that determines whether evidence is circumstantial.

Forensic evidence supplied by an expert witness is usually treated as circumstantial evidence. For example, a forensic scientist may provide results of ballistic tests proving that the defendant’s firearm fired the bullets that killed the victim, but not necessarily that the defendant fired the shots.

Circumstantial evidence is especially important in civil and criminal cases where direct evidence is lacking.

Deductive reasoning

Deductive reasoning, also deductive logic, logical deduction is the process of reasoning from one or more statements (premises) to reach a logically certain conclusion.Deductive reasoning goes in the same direction as that of the conditionals, and links premises with conclusions. If all premises are true, the terms are clear, and the rules of deductive logic are followed, then the conclusion reached is necessarily true.

Deductive reasoning ("top-down logic") contrasts with inductive reasoning ("bottom-up logic") in the following way; in deductive reasoning, a conclusion is reached reductively by applying general rules which hold over the entirety of a closed domain of discourse, narrowing the range under consideration until only the conclusion(s) is left. In inductive reasoning, the conclusion is reached by generalizing or extrapolating from specific cases to general rules, i.e., there is epistemic uncertainty. However, the inductive reasoning mentioned here is not the same as induction used in mathematical proofs – mathematical induction is actually a form of deductive reasoning.

Deductive reasoning differs from abductive reasoning by the direction of the reasoning relative to the conditionals. Deductive reasoning goes in the same direction as that of the conditionals, whereas abductive reasoning goes in the opposite direction to that of the conditionals.

Expert system

In artificial intelligence, an expert system is a computer system that emulates the decision-making ability of a human expert.

Expert systems are designed to solve complex problems by reasoning through bodies of knowledge, represented mainly as if–then rules rather than through conventional procedural code. The first expert systems were created in the 1970s and then proliferated in the 1980s. Expert systems were among the first truly successful forms of artificial intelligence (AI) software.

An expert system is divided into two subsystems: the inference engine and the knowledge base. The knowledge base represents facts and rules. The inference engine applies the rules to the known facts to deduce new facts. Inference engines can also include explanation and debugging abilities.

Frequentist inference

Frequentist inference is a type of statistical inference that draws conclusions from sample data by emphasizing the frequency or proportion of the data. An alternative name is frequentist statistics. This is the inference framework in which the well-established methodologies of statistical hypothesis testing and confidence intervals are based. Other than frequentistic inference, the main alternative approach to statistical inference is Bayesian inference, while another is fiducial inference.

While "Bayesian inference" is sometimes held to include the approach to inference leading to optimal decisions, a more restricted view is taken here for simplicity.

Ian Hacking

Ian MacDougall Hacking (born February 18, 1936) is a Canadian philosopher specializing in the philosophy of science. Throughout his career, he has won numerous awards, such as the Killam Prize for the Humanities and the Balzan Prize, and been a member of many prestigious groups, including the Order of Canada, the Royal Society of Canada and the British Academy.

Inductive reasoning

Inductive reasoning is a method of reasoning in which the premises are viewed as supplying some evidence for the truth of the conclusion, this is in contrast to deductive reasoning. While the conclusion of a deductive argument is certain, the truth of the conclusion of an inductive argument may be probable, based upon the evidence given.Many dictionaries define inductive reasoning as the derivation of general principles from specific observations, though some sources find this usage "outdated".

Logic

Logic (from the Ancient Greek: λογική, translit. logikḗ) is the systematic study of the form of valid inference, and the most general laws of truth. A valid inference is one where there is a specific relation of logical support between the assumptions of the inference and its conclusion. In ordinary discourse, inferences may be signified by words such as therefore, hence, ergo, and so on.

There is no universal agreement as to the exact scope and subject matter of logic (see § Rival conceptions, below), but it has traditionally included the classification of arguments, the systematic exposition of the 'logical form' common to all valid arguments, the study of proof and inference, including paradoxes and fallacies, and the study of syntax and semantics. Historically, logic has been studied in philosophy (since ancient times) and mathematics (since the mid-19th century), and recently logic has been studied in computer science, linguistics, psychology, and other fields.

Mathematical statistics

Mathematical statistics is the application of probability theory, a branch of mathematics, to statistics, as opposed to techniques for collecting statistical data. Specific mathematical techniques which are used for this include mathematical analysis, linear algebra, stochastic analysis, differential equations, and measure theory.

Nonparametric statistics

Nonparametric statistics is the branch of statistics that is not based solely on parametrized families of probability distributions (common examples of parameters are the mean and variance). Nonparametric statistics is based on either being distribution-free or having a specified distribution but with the distribution's parameters unspecified. Nonparametric statistics includes both descriptive statistics and statistical inference.

Parametric statistics

Parametric statistics is a branch of statistics which assumes that sample data comes from a population that can be adequately modelled by a probability distribution that has a fixed set of parameters. Conversely a non-parametric model differs precisely in that the parameter set (or feature set in machine learning) is not fixed and can increase, or even decrease, if new relevant information is collected.Most well-known statistical methods are parametric. Regarding nonparametric (and semiparametric) models, Sir David Cox has said, "These typically involve fewer assumptions of structure and distributional form but usually contain strong assumptions about independencies".

Point estimation

In statistics, point estimation involves the use of sample data to calculate a single value (known as a point estimate or statistic) which is to serve as a "best guess" or "best estimate" of an unknown population parameter (for example, the population mean). More formally, it is the application of a point estimator to the data to obtain a point estimate.

Point estimation can be contrasted with interval estimation: such interval estimates are typically either confidence intervals, in the case of frequentist inference, or credible intervals, in the case of Bayesian inference.

Rule of inference

In logic, a rule of inference, inference rule or transformation rule is a logical form consisting of a function which takes premises, analyzes their syntax, and returns a conclusion (or conclusions). For example, the rule of inference called modus ponens takes two premises, one in the form "If p then q" and another in the form "p", and returns the conclusion "q". The rule is valid with respect to the semantics of classical logic (as well as the semantics of many other non-classical logics), in the sense that if the premises are true (under an interpretation), then so is the conclusion.

Typically, a rule of inference preserves truth, a semantic property. In many-valued logic, it preserves a general designation. But a rule of inference's action is purely syntactic, and does not need to preserve any semantic property: any function from sets of formulae to formulae counts as a rule of inference. Usually only rules that are recursive are important; i.e. rules such that there is an effective procedure for determining whether any given formula is the conclusion of a given set of formulae according to the rule. An example of a rule that is not effective in this sense is the infinitary ω-rule.Popular rules of inference in propositional logic include modus ponens, modus tollens, and contraposition. First-order predicate logic uses rules of inference to deal with logical quantifiers.

Sampling distribution

In statistics, a sampling distribution or finite-sample distribution is the probability distribution of a given random-sample-based statistic. If an arbitrarily large number of samples, each involving multiple observations (data points), were separately used in order to compute one value of a statistic (such as, for example, the sample mean or sample variance) for each sample, then the sampling distribution is the probability distribution of the values that the statistic takes on. In many contexts, only one sample is observed, but the sampling distribution can be found theoretically.

Sampling distributions are important in statistics because they provide a major simplification en route to statistical inference. More specifically, they allow analytical considerations to be based on the probability distribution of a statistic, rather than on the joint probability distribution of all the individual sample values.

Statistical inference

Statistical inference is the process of using data analysis to deduce properties of an underlying probability distribution. Inferential statistical analysis infers properties of a population, for example by testing hypotheses and deriving estimates. It is assumed that the observed data set is sampled from a larger population.

Inferential statistics can be contrasted with descriptive statistics. Descriptive statistics is solely concerned with properties of the observed data, and it does not rest on the assumption that the data come from a larger population.

Type inference

Type inference refers to the automatic detection of the data type of an expression in a programming language.

It is a feature present in some strongly statically typed languages. It is often characteristic of functional programming languages in general. Some languages that include type inference include C++11, C# (starting with version 3.0), Chapel, Clean, Crystal, D, F#, FreeBASIC, Go, Haskell, Java (starting with version 10), Julia, Kotlin, ML, Nim, OCaml, Opa, RPython, Rust, Scala, Swift, Vala and Visual Basic (starting with version 9.0).

The majority of them use a simple form of type inference, while especially those ones who use the Hindley-Milner type system provide a more complete type inference. The ability to infer types automatically makes many programming tasks easier, leaving the programmer free to omit type annotations while still permitting type checking.

Fields
Foundations
Lists

This page is based on a Wikipedia article written by authors (here).
Text is available under the CC BY-SA 3.0 license; additional terms may apply.
Images, videos and audio are available under their respective licenses.