A logistic function or logistic curve is a common "S" shape (sigmoid curve), with equation:
For values of x in the domain of real numbers from −∞ to +∞, the S-curve shown on the right is obtained, with the graph of f approaching L as x approaches +∞ and approaching zero as x approaches −∞.
The logistic function finds applications in a range of fields, including artificial neural networks, biology (especially ecology), biomathematics, chemistry, demography, economics, geoscience, mathematical psychology, probability, sociology, political science, linguistics, and statistics.
The logistic function was introduced in a series of three papers by Pierre François Verhulst between 1838 and 1847, who devised it as a model of population growth by adjusting the exponential growth model, under the guidance of Adolphe Quetelet. Verhulst first devised the function in the mid 1830s, publishing a brief note in 1838, then presented an expanded analysis and named the function in 1844 (published 1845);[a] the third paper adjusted the correction term in his model of Belgian population growth.
The initial stage of growth is approximately exponential (geometric); then, as saturation begins, the growth slows to linear (arithmetic), and at maturity, growth stops. Verhulst did not explain the choice of the term "logistic" (French: logistique), but it is presumably in contrast to the logarithmic curve,[b] and by analogy with arithmetic and geometric. His growth model is preceded by a discussion of arithmetic growth and geometric growth (whose curve he calls a logarithmic curve, instead of the modern term exponential curve), and thus "logistic growth" is presumably named by analogy, logistic being from Ancient Greek: λογῐστῐκός, romanized: logistikós, a traditional division of Greek mathematics.[c] The term is unrelated to the military and management term logistics, which is instead from French: logis "lodgings", though some believe the Greek term also influenced logistics; see Logistics § Origin for details.
The standard logistic function is the logistic function with parameters (k = 1, x0 = 0, L = 1) which yields
In practice, due to the nature of the exponential function e−x, it is often sufficient to compute the standard logistic function for x over a small range of real numbers such as a range contained in [−6, +6] as it quickly converges very close to its saturation values of 0 and 1.
The logistic function has the symmetry property that:
Thus, is an odd function.
The logistic function is an offset and scaled hyperbolic tangent function
This follows from
The standard logistic function has an easily calculated derivative:
The derivative of the logistic function is an even function, that is,
In artificial neural networks, this is known as the softplus function, and (with scaling) is a smooth approximation of the ramp function, just as the logistic function (with scaling) is a smooth approximation of the Heaviside step function.
The standard logistic function is the solution of the simple first-order non-linear ordinary differential equation
The qualitative behavior is easily understood in terms of the phase line: the derivative is 0 when the function is 1; and the derivative is positive for f between 0 and 1, and negative for f above 1 or less than 0 (though negative populations do not generally accord with a physical model). This yields an unstable equilibrium at 0, and a stable equilibrium at 1, and thus for any function value greater than 0 and less than 1, it grows to 1.
The logistic equation is a special case of the Bernoulli differential equation and has the following solution:
Choosing the constant of integration gives the other well-known form of the definition of the logistic curve
More quantitatively, as can be seen from the analytical solution, the logistic curve shows early exponential growth for negative argument, which slows to linear growth of slope 1/4 for an argument near 0, then approaches 1 with an exponentially decaying gap.
The logistic function is the inverse of the natural logit function and so can be used to convert the logarithm of odds into a probability. In mathematical notation the logistic function is sometimes written as expit in the same form as logit. The conversion from the log-likelihood ratio of two alternatives also takes the form of a logistic curve.
The hyperbolic tangent relationship leads to another form for the logistic function's derivative:
which ties the logistic function into the logistic distribution.
The sum of the logistic function and its reflection about the vertical axis, f (−x) is
The logistic function is thus rotationally symmetrical about the point (0, 1/).
A typical application of the logistic equation is a common model of population growth (see also population dynamics), originally due to Pierre-François Verhulst in 1838, where the rate of reproduction is proportional to both the existing population and the amount of available resources, all else being equal. The Verhulst equation was published after Verhulst had read Thomas Malthus' An Essay on the Principle of Population. Verhulst derived his logistic equation to describe the self-limiting growth of a biological population. The equation was rediscovered in 1911 by A. G. McKendrick for the growth of bacteria in broth and experimentally tested using a technique for nonlinear parameter estimation. The equation is also sometimes called the Verhulst-Pearl equation following its rediscovery in 1920 by Raymond Pearl (1879–1940) and Lowell Reed (1888–1966) of the Johns Hopkins University. Another scientist, Alfred J. Lotka derived the equation again in 1925, calling it the law of population growth.
Letting P represent population size (N is often used in ecology instead) and t represent time, this model is formalized by the differential equation:
In the equation, the early, unimpeded growth rate is modeled by the first term +rP. The value of the rate r represents the proportional increase of the population P in one unit of time. Later, as the population grows, the modulus of the second term (which multiplied out is −rP2/K) becomes almost as large as the first, as some members of the population P interfere with each other by competing for some critical resource, such as food or living space. This antagonistic effect is called the bottleneck, and is modeled by the value of the parameter K. The competition diminishes the combined growth rate, until the value of P ceases to grow (this is called maturity of the population). The solution to the equation (with being the initial population) is
Which is to say that K is the limiting value of P: the highest value that the population can reach given infinite time (or come close to reaching in finite time). It is important to stress that the carrying capacity is asymptotically reached independently of the initial value P(0) > 0, and also in the case that P(0) > K.
In ecology, species are sometimes referred to as r-strategist or K-strategist depending upon the selective processes that have shaped their life history strategies. Choosing the variable dimensions so that n measures the population in units of carrying capacity, and τ measures time in units of 1/r, gives the dimensionless differential equation
Since the environmental conditions influence the carrying capacity, as a consequence it can be time-varying: K(t) > 0, leading to the following mathematical model:
A particularly important case is that of carrying capacity that varies periodically with period T:
It can be shown that in such a case, independently from the initial value P(0) > 0, P(t) will tend to a unique periodic solution P*(t), whose period is T.
A typical value of T is one year: In such case K(t) may reflect periodical variations of weather conditions.
Another interesting generalization is to consider that the carrying capacity K(t) is a function of the population at an earlier time, capturing a delay in the way population modifies its environment. This leads to a logistic delay equation, which has a very rich behavior, with bistability in some parameter range, as well as a monotonic decay to zero, smooth exponential growth, punctuated unlimited growth (i.e., multiple S-shapes), punctuated growth or alternation to a stationary level, oscillatory approach to a stationary level, sustainable oscillations, finite-time singularities as well as finite-time death.
Logistic functions are used in several roles in statistics. For example, they are the cumulative distribution function of the logistic family of distributions, and they are, a bit simplified, used to model the chance a chess player has to beat his opponent in the Elo rating system. More specific examples now follow.
where x is the explanatory variable and a and b are model parameters to be fitted and f is the standard logistic function.
Logistic regression and other log-linear models are also commonly used in machine learning. A generalisation of the logistic function to multiple inputs is the softmax activation function, used in multinomial logistic regression.
Another application of the logistic function is in the Rasch model, used in item response theory. In particular, the Rasch model forms a basis for maximum likelihood estimation of the locations of objects or persons on a continuum, based on collections of categorical data, for example the abilities of persons on a continuum based on responses that have been categorized as correct and incorrect.
Logistic functions are often used in neural networks to introduce nonlinearity in the model or to clamp signals to within a specified range. A popular neural net element computes a linear combination of its input signals, and applies a bounded logistic function to the result; this model can be seen as a "smoothed" variant of the classical threshold neuron.
A common choice for the activation or "squashing" functions, used to clip for large magnitudes to keep the response of the neural network bounded is
which is a logistic function. These relationships result in simplified implementations of artificial neural networks with artificial neurons. Practitioners caution that sigmoidal functions which are antisymmetric about the origin (e.g. the hyperbolic tangent) lead to faster convergence when training networks with backpropagation.
The logistic function is itself the derivative of another proposed activation function, the softplus.
Another application of logistic curve is in medicine, where the logistic differential equation is used to model the growth of tumors. This application can be considered an extension of the above-mentioned use in the framework of ecology (see also the Generalized logistic curve, allowing for more parameters). Denoting with X(t) the size of the tumor at time t, its dynamics are governed by:
which is of the type:
where F(X) is the proliferation rate of the tumor.
If a chemotherapy is started with a log-kill effect, the equation may be revised to be
where c(t) is the therapy-induced death rate. In the idealized case of very long therapy, c(t) can be modeled as a periodic function (of period T) or (in case of continuous infusion therapy) as a constant function, and one has that
i.e. if the average therapy-induced death rate is greater than the baseline proliferation rate then there is the eradication of the disease. Of course, this is an oversimplified model of both the growth and the therapy (e.g. it does not take into account the phenomenon of clonal resistance).
The concentration of reactants and products in autocatalytic reactions follow the logistic function. The degradation of platinum group metal-free (PGM-free) oxygen reduction reaction (ORR) catalyst in fuel cell cathodes follows the logistic decay function, suggesting an autocatalytic degradation mechanism.
The logistic function determines the statistical distribution of fermions over the energy states of a system in thermal equilibrium. In particular, it is the distribution of the probabilities that each possible energy level is occupied by a fermion, according to Fermi–Dirac statistics.
In linguistics, the logistic function can be used to model language change: an innovation that is at first marginal begins to spread more quickly with time, and then more slowly as it becomes more universally adopted.
The logistic S-curve can be used for modeling the crop response to changes in growth factors. There are two types of response functions: positive and negative growth curves. For example, the crop yield may increase with increasing value of the growth factor up to a certain level (positive function), or it may decrease with increasing growth factor values (negative function owing to a negative growth factor), which situation requires an inverted S-curve.
The logistic function can be used to illustrate the progress of the diffusion of an innovation through its life cycle.
In The Laws of Imitation (1890), Gabriel Tarde describes the rise and spread of new ideas through imitative chains. In particular, Tarde identifies three main stages through which innovations spread: the first one corresponds to the difficult beginnings, during which the idea has to struggle within a hostile environment full of opposing habits and beliefs; the second one corresponds to the properly exponential take-off of the idea, with ; finally, the third stage is logarithmic, with , and corresponds to the time when the impulse of the idea gradually slows down while, simultaneously new opponent ideas appear. The ensuing situation halts or stabilizes the progress of the innovation, which approaches an asymptote.
In the history of economy, when new products are introduced there is an intense amount of research and development which leads to dramatic improvements in quality and reductions in cost. This leads to a period of rapid industry growth. Some of the more famous examples are: railroads, incandescent light bulbs, electrification, cars and air travel. Eventually, dramatic improvement and cost reduction opportunities are exhausted, the product or process are in widespread use with few remaining potential new customers, and markets become saturated.
Logistic analysis was used in papers by several researchers at the International Institute of Applied Systems Analysis (IIASA). These papers deal with the diffusion of various innovations, infrastructures and energy source substitutions and the role of work in the economy as well as with the long economic cycle. Long economic cycles were investigated by Robert Ayres (1989). Cesare Marchetti published on long economic cycles and on diffusion of innovations. Arnulf Grübler's book (1990) gives a detailed account of the diffusion of infrastructures including canals, railroads, highways and airlines, showing that their diffusion followed logistic shaped curves.
Carlota Perez used a logistic curve to illustrate the long (Kondratiev) business cycle with the following labels: beginning of a technological era as irruption, the ascent as frenzy, the rapid build out as synergy and the completion as maturity.
Nous donnerons le nom de logistique à la courbe [We will give the name logistic to the curve]
The diagram clinched it for me: there two curves labeled "Logistique" and "Logarithmique" are drawn on the same axes, and one can see that there is a region where they match almost exactly, and then diverge.
I concluded that Verhulst's intention in naming the curve was indeed to suggest this comparison, and that "logistic" was meant to convey the curve's "log-like" quality.
Bounded growth occurs when the growth rate of a mathematical function is constantly increasing at a decreasing rate. Asymptotically, bounded growth approaches a fixed value. This contrasts with exponential growth, which is constantly increasing at an accelerating rate, and therefore approaches infinity in the limit.
An example of bounded growth is the logistic function.Dominance (ecology)
Ecological dominance is the degree to which a taxon is more numerous than its competitors in an ecological community, or makes up more of the biomass.
Most ecological communities are defined by their dominant species.
In many examples of wet woodland in western Europe, the dominant tree is alder (Alnus glutinosa).
In temperate bogs, the dominant vegetation is usually species of Sphagnum moss.
Tidal swamps in the tropics are usually dominated by species of mangrove (Rhizophoraceae)
Some sea floor communities are dominated by brittle stars.
Exposed rocky shorelines are dominated by sessile organisms such as barnacles and limpets.Generalised logistic function
The generalised (generalized) logistic function or curve, also known as Richards' curve, originally developed for growth modelling, is an extension of the logistic or sigmoid functions, allowing for more flexible S-shaped curves:
where = weight, height, size etc., and = time.
It has five parameters:
The equation can also be written:
where can be thought of as a starting time, (at which )
Including both and can be convenient:
this representation simplifies the setting of both a starting time and the value of Y at that time.
The general model is sometimes named a "Richards' curve" after F. J. Richards, who proposed the general form for the family of models in 1959.
The logistic, with maximum growth rate at time , is the case where = 1.
The Gompertz curve or Gompertz function, is a type of mathematical model for a time series and is named after Benjamin Gompertz (1779-1865). It is a sigmoid function which describes growth as being slowest at the start and end of a given time period. The right-hand or future value asymptote of the function is approached much more gradually by the curve than the left-hand or lower valued asymptote. This is in contrast to the simple logistic function in which both asymptotes are approached by the curve symmetrically. It is a special case of the generalised logistic function. The function was originally designed to describe human mortality, but since has been modified to be applied in biology, with regards to detailing populations.Hubbert linearization
The Hubbert Linearization is a way to plot production data to estimate two important parameters of a Hubbert curve:
the logistic growth rate and
the quantity of the resource that will be ultimately recovered.The Hubbert curve is the first derivative of a Logistic function, which has been used in modeling depletion of crude oil, predicting the Hubbert peak, population growth predictions and the depletion of finite mineral resources. The linearization technique was introduced by Marion King Hubbert in his 1982 review paper.Log-linear model
A log-linear model is a mathematical model that takes the form of a function whose logarithm equals a linear combination of the parameters of the model, which makes it possible to apply (possibly multivariate) linear regression. That is, it has the general form
in which the fi(X) are quantities that are functions of the variables X, in general a vector of values, while c and the wi stand for the model parameters.
The term may specifically be used for:
The specific applications of log-linear models are where the output quantity lies in the range 0 to ∞, for values of the independent variables X, or more immediately, the transformed quantities fi(X) in the range −∞ to +∞. This may be contrasted to logistic models, similar to the logistic function, for which the output quantity lies in the range 0 to 1. Thus the contexts where these models are useful or realistic often depends on the range of the values being modelled.Logistic distribution
In probability theory and statistics, the logistic distribution is a continuous probability distribution. Its cumulative distribution function is the logistic function, which appears in logistic regression and feedforward neural networks. It resembles the normal distribution in shape but has heavier tails (higher kurtosis). The logistic distribution is a special case of the Tukey lambda distribution.Logistic equation
Logistic equation can refer to:
Logistic map, a nonlinear recurrence relation that plays a prominent role in chaos theory
Logistic regression, a regression technique that transforms the dependent variable using the logistic function
Logistic differential equation, a differential equation for population dynamics proposed by Pierre François VerhulstLogistic regression
In statistics, the logistic model (or logit model) is used to model the probability of a certain class or event existing such as pass/fail, win/lose, alive/dead or healthy/sick. This can be extended to model several classes of events such as determining whether an image contains a cat, dog, lion, etc... Each object being detected in the image would be assigned a probability between 0 and 1 and the sum adding to one.
Logistic regression is a statistical model that in its basic form uses a logistic function to model a binary dependent variable, although many more complex extensions exist. In regression analysis, logistic regression (or logit regression) is estimating the parameters of a logistic model (a form of binary regression). Mathematically, a binary logistic model has a dependent variable with two possible values, such as pass/fail which is represented by an indicator variable, where the two values are labeled "0" and "1". In the logistic model, the log-odds (the logarithm of the odds) for the value labeled "1" is a linear combination of one or more independent variables ("predictors"); the independent variables can each be a binary variable (two classes, coded by an indicator variable) or a continuous variable (any real value). The corresponding probability of the value labeled "1" can vary between 0 (certainly the value "0") and 1 (certainly the value "1"), hence the labeling; the function that converts log-odds to probability is the logistic function, hence the name. The unit of measurement for the log-odds scale is called a logit, from logistic unit, hence the alternative names. Analogous models with a different sigmoid function instead of the logistic function can also be used, such as the probit model; the defining characteristic of the logistic model is that increasing one of the independent variables multiplicatively scales the odds of the given outcome at a constant rate, with each independent variable having its own parameter; for a binary dependent variable this generalizes the odds ratio.
The binary logistic regression model has extensions to more than two levels of the dependent variable: categorical outputs with more than two values are modeled by multinomial logistic regression, and if the multiple categories are ordered, by ordinal logistic regression, for example the proportional odds ordinal logistic model. The model itself simply models probability of output in terms of input, and does not perform statistical classification (it is not a classifier), though it can be used to make a classifier, for instance by choosing a cutoff value and classifying inputs with probability greater than the cutoff as one class, below the cutoff as the other; this is a common way to make a binary classifier. The coefficients are generally not computed by a closed-form expression, unlike linear least squares; see § Model fitting. The logistic regression as a general statistical model was originally developed and popularized primarily by Joseph Berkson, beginning in Berkson (1944), where he coined "logit"; see § History.Logit
In statistics, the logit (/ˈloʊdʒɪt/ LOH-jit) function or the log-odds is the logarithm of the odds p/(1 − p) where p is probability. It is a type of function that creates a map of probability values from to . It is the inverse of the sigmoidal "logistic" function or logistic transform used in mathematics, especially in statistics.
In deep learning, the term logits layer is popularly used for the last neuron layer of neural networks used for classification tasks, which produce raw prediction values as real numbers ranging from .Logit-normal distribution
In probability theory, a logit-normal distribution is a probability distribution of a random variable whose logit has a normal distribution. If Y is a random variable with a normal distribution, and P is the standard logistic function, then X = P(Y) has a logit-normal distribution; likewise, if X is logit-normally distributed, then Y = logit(X)= log (X/(1-X)) is normally distributed. It is also known as the logistic normal distribution, which often refers to a multinomial logit version (e.g.).
A variable might be modeled as logit-normal if it is a proportion, which is bounded by zero and one, and where values of zero and one never occur.Logit analysis in marketing
Logit analysis is a statistical technique used by marketers to assess the scope of customer acceptance of a product, particularly a new product. It attempts to determine the intensity or magnitude of customers' purchase intentions and translates that into a measure of actual buying behaviour. Logit analysis assumes that an unmet need in the marketplace has already been detected, and that the product has been designed to meet that need. The purpose of logit analysis is to quantify the potential sales of that product. It takes survey data on consumers' purchase intentions and converts it into actual purchase probabilities.
Logit analysis defines the functional relationship between stated purchase intentions and preferences, and the actual probability of purchase. A preference regression is performed on the survey data. This is then modified with actual historical observations of purchase behavior. The resultant functional relationship defines purchase probability.
This is the most useful of the purchase intention/rating translations because explicit measures of confidence level and statistical significance can be calculated. Other purchase intention/rating translations include the preference-rank translation and the intent scale translation.
The logit function is the reciprocal function to the sigmoid logistic function.Malthusian growth model
A Malthusian growth model, sometimes called a simple exponential growth model, is essentially exponential growth based on the idea of the function being proportional to the speed to which the function grows. The model is named after Thomas Robert Malthus, who wrote An Essay on the Principle of Population (1798), one of the earliest and most influential books on population.
Malthusian models have the following form:
The model can also been written in the form of a differential equation:
dP/dt = rP
with initial condition: P(0)= P0
This model is often referred to as the exponential law. It is widely regarded in the field of population ecology as the first principle of population dynamics, with Malthus as the founder. The exponential law is therefore also sometimes referred to as the Malthusian Law. By now, it is a widely accepted view to analogize Malthusian growth in Ecology to Newton's First Law of uniform motion in physics.
Malthus wrote that all life forms, including humans, have a propensity to exponential population growth when resources are abundant but that actual growth is limited by available resources:
A model of population growth bounded by resource limitations was developed by Pierre Francois Verhulst in 1838, after he had read Malthus' essay. Verhulst named the model a logistic function.Pairwise comparison
Pairwise comparison generally is any process of comparing entities in pairs to judge which of each entity is preferred, or has a greater amount of some quantitative property, or whether or not the two entities are identical. The method of pairwise comparison is used in the scientific study of preferences, attitudes, voting systems, social choice, public choice, requirements engineering and multiagent AI systems. In psychology literature, it is often referred to as paired comparison.
Prominent psychometrician L. L. Thurstone first introduced a scientific approach to using pairwise comparisons for measurement in 1927, which he referred to as the law of comparative judgment. Thurstone linked this approach to psychophysical theory developed by Ernst Heinrich Weber and Gustav Fechner. Thurstone demonstrated that the method can be used to order items along a dimension such as preference or importance using an interval-type scale.Reed–Muench method
See article above for overview of 50% endpoints and comparison with other methods of calculating 50% endpoints.
The Reed–Muench method is a simple method for determining 50% endpoints in experimental biology, that is, the concentration of a test substance that produces an effect of interest in half of the test units. Examples include LD50 (the median lethal dose of a toxin or pathogen), EC50 and IC50 (half maximal effective or inhibitory concentration, respectively, of a drug), and TCID50 (50% tissue culture infectious dose of a virus).
The reason for using 50% endpoints is that many dose-response relationships in biology follow a logistic function that flattens out as it approaches the minimal and maximal responses, so it is easier to measure the concentration of the test substance that produces a 50% response.S Curve
S curve or S-curve may refer to:
S Curve (art), an art term for a sinuous body form
S-Curve Records, a record company label
S-curve (math), a sigmoid function with an "S"-shape
Logistic function, a common sigmoid curve
Reverse curve, a section of a route in which a curve to the left or right is followed immediately by a curve in the opposite directionSigmoid function
A sigmoid function is a mathematical function having a characteristic "S"-shaped curve or sigmoid curve. Often, sigmoid function refers to the special case of the logistic function shown in the first figure and defined by the formula
Special cases of the sigmoid function include the Gompertz curve (used in modeling systems that saturate at large values of x) and the ogee curve (used in the spillway of some dams). Sigmoid functions have domain of all real numbers, with return value monotonically increasing most often from 0 to 1 or alternatively from −1 to 1, depending on convention.
A wide variety of sigmoid functions including the logistic and hyperbolic tangent functions have been used as the activation function of artificial neurons. Sigmoid curves are also common in statistics as cumulative distribution functions (which go from 0 to 1), such as the integrals of the logistic distribution, the normal distribution, and Student's t probability density functions.Von Bertalanffy function
The von Bertalanffy growth function (VBGF), or von Bertalanffy curve, is a type of growth curve model for a time series and is named after Ludwig von Bertalanffy. It is a special case of the generalised logistic function. The growth curve is used to model mean length from age in animals. The function is commonly applied in ecology to model fish growth.
The model can be written as the following:
where is age, is the growth coefficient, is a value used to calculate size when age is zero, and is asymptotic size. It is the solution of the following linear differential equation: