MENU
- Course Information (for students enrolled in my class)
- Introduction
- Computers and Statistical Data Analysis
- Questionnaire Design and Surveys Sampling
- Time Series Analysis and Forecasting Techniques
- Popular Distributions and Their Typical
- Topics in Statistical Data Analysis
- Business Statistics: Revealing Facts From Figures
- Computational Probability and Statistics Resources
- Books on Statistical Data Analysis
- Interesting and Useful Sites
Computers and Statistical Data
Analysis
A webtext companion site containing SPSS and SAS
programs listing for:
Introductory Data Analysis Routines, Generating
Normal Random Variate, K-S Lilliefors Test for Normality, Outliers
Determination, Chi-square test: Dependency, T-Test, Two Independent
Populations, T-test, Two Dependent populations, Analysis of variance (ANOVA),
Non-parametric ANOVA version, 2-Way ANOVA, MANOVA: Comparison with a Control
Case, and Regression Analysis Routines.
Questionnaire Design and Surveys
Sampling
A webtext companion site which covers questionnaire
design and surveys sampling topics including: Questionnaire Design and
Statistical Data Analysis, Sample Size in Surveys Sampling, Multilevel
Statistical Models, Surveys Sampling Routines, Cronbach's Alpha (Coefficient
Alpha), Instrumentality Theory, and Value Measurements Survey Instruments
(Rokeach's Value Survey).
Time Series Analysis and Forecasting
Techniques
A webtext companion site which covers time series
analysis and forecasting techniques including: A Summary of Forecasting
Methods, How to Do Forecasting by a Regression Analysis, Moving Average and
Exponential Smoothing Methods (Fortran Codes), Winters' Method (Fortran
Codes), Smoothing the Data (Interactive Fortran Codes), Transfer Functions
Methodology, Box-Jenkins Methodology, Modeling Financial Time Series, Census
II Method of Seasonal Analysis, SPSS Programs Listing for Trends and the
Box-Jenkins, SAS Programs Listing for Exponential Smoothing and Winters
Methods, and Measuring for Accuracy (Interactive Fortran Codes).
Popular Distributions and Their Typical Applications
Binomial
, Multinomial
, Hypergeometric
,
Geometric , Pascal
, Negative
Binomial , Poisson
, Normal
, Gamma
, Exponential
, Beta
,
Uniform ,
Log-normal ,
Rayleigh , Cauchy
, Weibull
,
Extreme value ,
t distributions
Visit also the Web sites Statistical Distribution
Plots and Distribution
Function .
Topics in Statistical Data Analysis
Why Is
Every Thing Priced One Penny Off the Dollar?
What is
Statistical Data Analysis? Data are not Information!
Data
Processing: Coding, Typing, and Editing
Variance
of Nonlinear Random Functions
What Is
a Geometric Mean
What Is
Central Limit Theorem?
What Is a
Sampling Distribution?
Least
Squares Models
Least
Median of Squares Models
You
Must Look at Your Scattergrams!
Power of a
Test
ANOVA:
Analysis of Variance
An
Alternative Approach for Estimating a Regression Line
Multivariate
Data Analysis
The
Meaning and Interpretation of P-values (what the data say?)
P-value
for Standard Normal and t-statistics
What is
the Effect Size?
Bias
Reduction Techniques
Area
Under Standard Normal Curve
Number of
Class Interval in Histogram
Structural
Equation Modeling
Tri-linear
Coordinates Triangle
Internal
and Inter-rater Reliability
When to
Use Nonparametric Technique?
Analysis
of Incomplete Data
Interactions
in ANOVA and Regression Analysis
Distance
Sampling
Bayes and
Empirical Bayes Methods
Likelihood
Methods
Accuracy,
Precision, Robustness, and Quality
What is a
Meta-Analysis?
Prediction
Interval
Fitting
Data to a Broken Line
How to
Determine if Two Regression Lines Are Parallel?
Constrained
Regression Model
Semiparametric
and Non-parametric Modeling
Moderation
and Mediation
Discriminant
and Classification
Generalized
Linear and Logistic Models
Survival
Analysis
Association
Among Nominal Variables
Spearman's
Correlation, and Kendall's tau Application
Repeated
Measures and Longitudinal Data
Spatial
Data Analysis
Data
Mining and Knowledge Discovery
Incidence
and Prevalence Rates
Software
Selection
Box-Cox
Power Transformation
Multiple
Comparison Tests
Antedependent
Modeling for Repeated Measurements
Split-half
Analysis
Sequential
Acceptance Sampling
Local
Influence
Variogram
Analysis
Credit
Scoring
Components
of the Interest Rates
Partial
Least Squares
Growth
Curve Modeling
Saturated
Model & Saturated Log Likelihood
Pattern
recognition and Classification
What
is Biostatistics?
Evidential
Statistics
What
Is a Systematic Review?
What
Is a Regression Tree?
Cluster
Analysis for Correlated Variables
Capture-Recapture
Methods
Tchebysheff
Inequality and Its Improvements
Interesting and Useful Sites (topical category)
Search and General Resources
Demo's and Interactive
Tools and Demos on the Internet (Annotated)
Survey Analysis
Books on Statistical Data Analysis
Statistical Software
Data and Data Analysis
Statistics and Probability
Forecasting and Time Series
Statistics Publishers
Introduction
Know that data are only crude information and not knowledge by themselves. The sequence from data to knowledge is: from Data to Information, from Information to Facts, and finally, from Facts to Knowledge. Data becomes information when it becomes relevant to your decision problem. Information becomes fact when the data can support it. Fact becomes knowledge when it is used in the successful completion of decision process. The following figure illustrates the statistical thinking process based on data in constructing statistical models for decision making under uncertainties.
That's why we need statistical data analysis. Statistics arose from the need to place knowledge on a systematic evidence base. This required a study of the laws of probability, the development of measures of data properties and relationships, and so on.
Knowledge is more than knowing something technical. Knowledge needs wisdom, and wisdom comes with age and experience. Wisdom is about knowing how something technical can be best used to meet the needs of the decision-maker. Wisdom, for example, creates statistical software that is useful, rather than technically brilliant.
We will apply the basic concepts and methods of statistics you've already learned in the previous statistics course to the real world problems. The course is tailored to meet your needs in the statistical business-data analysis using widely available commercial statistical computer packages such as SAS and SPSS. By doing this, you will inevitably find yourself asking questions about the data and the method proposed, and you will have the means at your disposal to settle these questions to your own satisfaction. Accordingly, all the applications problems are borrowed from business and economics. By the end of this course you'll be able to think statistically while performing any data analysis.
There are two general views of teaching/learning statistics: Greater and Lesser Statistics. Greater statistics is everything related to learning from data, from the first planning or collection, to the last presentation or report. Lesser statistics is the body of statistical methodology. This is a Greater Statistics course.
There are basically two kinds of "statistics" courses. The real kind shows you how to make sense out of data. These courses would include all the recent developments and all share a deep respect for data and truth. The imitation kind involves plugging numbers into statistics formulas. The emphasis is on doing the arithmetic correctly. These courses generally have no interest in data or truth, and the problems are generally arithmetic exercises. If a certain assumption is needed to justify a procedure, they will simply tell you to "assume the ... are normally distributed" -- no matter how unlikely that might be. It seems like you all are suffering from an overdose of the latter. This course will bring out the joy of statistics in you.
Statistics is a science assisting you to make decisions under uncertainties (based on some numerical and measurable scales). Decision making process must be based on data neither on personal opinion nor on belief.
It is already an accepted fact that "Statistical thinking will one day be as necessary for efficient citizenship as the ability to read and write." So, let us be ahead of our time.
Popular Distributions and Their Typical Applications
Binomial
Application: Gives probability of exactly successes in n independent trials, when probability of success p on single trial is a constant. Used frequently in quality control, reliability, survey sampling, and other industrial problems.Example: What is the probability of 7 or more "heads" in 10 tosses of a fair coin?
Comments: Can sometimes be approximated by normal or by Poisson distribution.
Multinomial
Application: Gives probability of exactly ni outcomes of event i, for i = 1, 2, ..., k in n independent trials when the probability pi of event i in a single trial is a constant. Used frequently in quality control and other industrial problems.Example: Four companies are bidding for each of three contracts, with specified success probabilities. What is the probability that a single company will receive all the orders?
Comments: Generalization of binomial distribution for ore than 2 outcomes.
Hypergeometric
Application: Gives probability of picking exactly x good units in a sample of n units from a population of N units when there are k bad units in the population. Used in quality control and related applications.Example: Given a lot with 21 good units and four defective. What is the probability that a sample of five will yield not more than one defective?
Comments: May be approximated by binomial distribution when n is small related to N.
Geometric
Application: Gives probability of requiring exactly x binomial trials before the first success is achieved. Used in quality control, reliability, and other industrial situations.Example: Determination of probability of requiring exactly five test firings before first success is achieved.
Pascal
Application: Gives probability of exactly x failures preceding the sth success.Example: What is the probability that the third success takes place on the 10th trial?
Negative Binomial
Application: Gives probability similar to Poisson distribution when events do not occur at a constant rate and occurrence rate is a random variable that follows a gamma distribution.
Example: Distribution of number of cavities for a group of dental patients.
Comments: Generalization of Pascal distribution when s is not an integer. Many authors do not distinguish between Pascal and negative binomial distributions.
Poisson
Application: Gives probability of exactly x independent occurrences during a given period of time if events take place independently and at a constant rate. May also represent number of occurrences over constant areas or volumes. Used frequently in quality control, reliability, queuing theory, and so on.Example: Used to represent distribution of number of defects in a piece of material, customer arrivals, insurance claims, incoming telephone calls, alpha particles emitted, and so on.
Comments: Frequently used as approximation to binomial distribution.
Normal
Application: A basic distribution of statistics. Many applications arise from central limit theorem (average of values of n observations approaches normal distribution, irrespective of form of original distribution under quite general conditions). Consequently, appropriate model for many, but not all, physical phenomena.Example: Distribution of physical measurements on living organisms, intelligence test scores, product dimensions, average temperatures, and so on.
Comments: Many methods of statistical analysis presume normal distribution.
A so-called Generalized Gaussian distribution has the following pdf:
A.exp[-B|x|n], where A, B, n are constants. For n=1 and 2 it is Laplacian and Gaussian distribution respectively. This distribution approximates reasonably good data in some image coding application.
Slash distribution is the distribution of the ratio of a normal random variable to an independent uniform random variable, see Hutchinson T., Continuous Bivariate Distributions, Rumsby Sci. Publications, 1990.
Gamma
Application: A basic distribution of statistics for variables bounded at one side - for example x greater than or equal to zero. Gives distribution of time required for exactly k independent events to occur, assuming events take place at a constant rate. Used frequently in queuing theory, reliability, and other industrial applications.Example: Distribution of time between re calibrations of instrument that needs re calibration after k uses; time between inventory restocking, time to failure for a system with standby components.
Comments: Erlangian, exponential, and chi- square distributions are special cases. The Dirichlet is a multidimensional extension of the Beta distribution.
Distribution of a product of iid uniform (0, 1) random? Like many problems with products, this becomes a familiar problem when turned into a problem about sums. If X is uniform (for simplicity of notation make it U(0,1)), Y=-log(X) is exponentially distributed, so the log of the product of X1, X2, ... Xn is the sum of Y1, Y2, ... Yn which has a gamma (scaled chi-square) distribution. Thus, it is a gamma density with shape parameter n and scale 1.
Exponential
Application: Gives distribution of time between independent events occurring at a constant rate. Equivalently, probability distribution of life, presuming constant conditional failure (or hazard) rate. Consequently, applicable in many, but not all reliability situations.Example: Distribution of time between arrival of particles at a counter. Also life distribution of complex nonredundant systems, and usage life of some components - in particular, when these are exposed to initial burn-in, and preventive maintenance eliminates parts before wear-out.
Comments: Special case of both Weibull and gamma distributions.
Beta
Application: A basic distribution of statistics for variables bounded at both sides - for example x between o and 1. Useful for both theoretical and applied problems in many areas.Example: Distribution of proportion of population located between lowest and highest value in sample; distribution of daily per cent yield in a manufacturing process; description of elapsed times to task completion (PERT).
Comments: Uniform, right triangular, and parabolic distributions are special cases. To generate beta, generate two random values from a gamma, g1, g2. The ratio g1/(g1 +g2) is distributed like a beta distribution. The beta distribution can also be thought of as the distribution of X1 given (X1+X2), when X1 and X2 are independent gamma random variables.
There is also a relationship between the Beta and Normal distributions. The conventional calculation is that given a PERT Beta with highest value as b lowest as a and most likely as m, the equivalent normal distribution has a mean and mode of (a + 4M + b)/6 and a standard deviation of (b - a)/6.
See Section 4.2 of, Introduction to Probability by J. Laurie Snell (New York, Random House, 1987) for a link between beta and F distributions (with the advantage that tables are easy to find).
Uniform
Application: Gives probability that observation will occur within a particular interval when probability of occurrence within that interval is directly proportional to interval length.Example: Used to generate random valued.
Comments: Special case of beta distribution.
The density of geometric mean of n independent uniforms(0,1) is:
P(X=x) = n x(n-1) (Log[1/xn])(n-1) / (n-1)!.
zL = [UL-(1-U)L]/L is said to have Tukey's symmetrical lambda distribution.
Log-normal
Application: Permits representation of random variable whose logarithm follows normal distribution. Model for a process arising from many small multiplicative errors. Appropriate when the value of an observed variable is a random proportion of the previously observed value.In the case where the data are lognormally distributed, the geometric mean acts as a better data descriptor than the mean. The more closely the data follow a lognormal distribution, the closer the geometric mean is to the median, since the log re-expression produces a symmetrical distribution.
Example: Distribution of sizes from a breakage process; distribution of
income size, inheritances and bank deposits; distribution of various
biological phenomena; life distribution of some transistor types.
The
ratio of two log-normally distributed variables is log-normal.
Rayleigh
Application: Gives distribution of radial error when the errors in two mutually perpendicular axes are independent and normally distributed around zero with equal variances.Example: Bomb-sighting problems; amplitude of noise envelope when a linear detector is used.
Comments: Special case of Weibull distribution.
Cauchy
Application: Gives distribution of ratio of two independent standardized normal variates.Example: Distribution of ratio of standardized noise readings; distribution of tan(x) when x is uniformly distributed.
Weibull
Application: General time-to-failure distribution due to wide diversity of hazard-rate curves, and extreme-value distribution for minimum of N values from distribution bounded at left.The Weibull distribution is often used to model "time until failure." In this manner, it is applied in actuarial science and in engineering work.
It is also an appropriate distribution for describing data corresponding to resonance behavior, such as the variation with energy of the cross section of a nuclear reaction or the variation with velocity of the absorption of radiation in the Mossbauer effect.
Example: Life distribution for some capacitors, ball bearings, relays, and so on.
Comments: Rayleigh and exponential distribution are special cases.
Extreme value
Application: Limiting model for the distribution of the maximum or minimum of N values selected from an "exponential-type" distribution, such as the normal, gamma, or exponential.Example: Distribution of breaking strength of some materials, capacitor breakdown voltage, gust velocities encountered by airplanes, bacteria extinction times.
t distributions
The t distributions were discovered in 1908 by William Gosset who was a chemist and a statistician employed by the Guinness brewing company. He considered himself a student still learning statistics, so that is how he signed his papers as pseudonym "Student". Or perhaps he used a pseudonym due to "trade secrets" restrictions by Guinness.Note that there are different t distributions, it is a class of distributions. When we speak of a specific t distribution, we have to specify the degrees of freedom. The t density curves are symmetric and bell-shaped like the normal distribution and have their peak at 0. However, the spread is more than that of the standard normal distribution. The larger the degrees of freedom, the closer the t-density is to the normal density.
Why Is Every Thing Priced One Penny Off the Dollar?
Here's a psychological answer. Due to a very limited data processing ability we humans rely heavily on categorization (e.g., seeing things as "black or white" requires just a binary coding scheme, as opposed to seeing the many shades of gray). Our number system has a major category of 100's (e.g., 100 pennies, 200 pennies, 300 pennies) and there is a affective response associated with these groups--more is better if you are getting them; more is bad if you are giving them. Advertising and pricing takes advantage of this limited data processing by $2.99, $3.95, etc. So that $2.99 carries the affective response associated with the 200 pennies group. Indeed, if you ask people to respond to "how close together" are 271 & 283 versus "how close together" are 291 & 303, the former are seen as closer (there's a lot of methodology set up to dissuade the subjects to just subtract the smaller from the larger). Similarly, prejudice, job promotions, competitive sports, and a host of other activates attempt to associate large qualitative differences with what are often minor quantitative differences, e.g., gold metal in Olympic swimming event may be milliseconds difference from no metal.Yet another motivation: Psychologically $9.99 might look better than $10.00, but there is a more basic reason too. The assistant has to give you change from your ten dollars, and has to ring the sale up thru his/her cash register to get at the one cent. This forces the transaction to go through the books, you get a receipt, and the assistant can't just pocket the $10 him/herself. Mind you, there's nothing to stop a particularly untrustworthy employee going into work with a pocketful of cents...
There's sales tax for that. For either price (at least in the US), you'll have to pay sales tax too. So that solves the problem of opening the cash register. That, plus the security cameras ;).
There has been some research in marketing theory on the consumer's behavior at particular price points. Essentially, these are tied up with buyer expectations based on prior experience. A critical case study in UK on price pointing of pantyhose (tights) shown that there were distinct demand peaks at buyer anticipated price points of 59p, 79p, 99p, £1.29 and so on. Demand at intermediate price points was dramatically below these anticipated points for similar quality goods. In the UK, for example, prices of wine are usually set at key price points. The wine retailers also confirm that sales at different prices (even a penny or so different) does result in dramatically different sales volumes.
Other studies showed the opposite where reduced price showed reduced sales volumes, consumers ascribing quality in line with price. However, it is not fully tested to determine if sales volume continued to increase with price.
Other similar research turns on the behavior of consumers to variations in price. The key issue here is that there is a Just Noticeable Difference (JND) below which consumers will not act on a price increase. This has practical application when increasing charge rates and the like. The JND is typically 5% and this provides the opportunity for consultants etc to increase prices above prior rates by less than the JND without customer complaint. As an empirical experiment, try overcharging clients by 1, 2,.., 5, 6% and watch the reaction. Up to 5% there appears to be no negative impact.
Conversely, there is no point in offering a fee reduction of less than 5%
as clients will not recognize the concession you have made. Equally, in
periods of price inflation, price rises should be staged so that the
individual price rise is kept under 5%, perhaps by raising prices by 4% twice
per year rather than a one off 8% rise.
What is Statistical Data Analysis? Data are not Information!
Data are not information!To determine what statistical data analysis is, one must first define statistics. Statistics is a set of methods that are used to collect, analyze, present, and interpret data. Statistical methods are used in a wide variety of occupations and help people identify, study, and solve many complex problems. In the business and economic world, these methods enable decision makers and managers to make informed and better decisions about uncertain situations.Vast amounts of statistical information are available in today's global and economic environment because of continual improvements in computer technology. To compete successfully globally, managers and decision makers must be able to understand the information and use it effectively. Statistical data analysis provides hands on experience to promote the use of statistical thinking and techniques to apply in order to make educated decisions in the business world.
Computers play a very important role in statistical data analysis. The statistical software package, SPSS, which is used in this course, offers extensive data-handling capabilities and numerous statistical analysis routines that can analyze small to very large data statistics. The computer will assist in the summarization of data, but statistical data analysis focuses on the interpretation of the output to make inferences and predictions.
Studying a problem through the use of statistical data analysis usually involves four basic steps.
1. Defining the problem
2. Collecting the data
3. Analyzing the
data
4. Reporting the results
Defining the Problem
An exact definition of the problem is imperative in order to obtain accurate data about it. It is extremely difficult to gather data without a clear definition of the problem.
Collecting the Data
Designing ways to collect data is an important job in statistical data
analysis. Two important aspects of a statistical study are:
Population - a
set of all the elements of interest in a study
Sample - a subset of the
population
Statistical inference is refer to extending your knowledge
obtain from a random sample from a population to the whole population. This is
known in mathematics as an Inductive Reasoning. That is, knowledge of whole
from a particular. Its main application is in hypotheses testing about a given
population.
The purpose of statistical inference is to obtain information
about a population form information contained in a sample. It is just not
feasible to test the entire population, so a sample is the only realistic way
to obtain data because of the time and cost constraints. Data can be either
quantitative or qualitative. Qualitative data are labels or names used to
identify an attribute of each element. Quantitative data are always numeric
and indicate either how much or how many.
For the purpose of statistical data analysis, distinguishing between cross-sectional and time series data is important. Cross-sectional data re data collected at the same or approximately the same point in time. Time series data are data collected over several time periods.
Data can be collected from existing sources or obtained through observation and experimental studies designed to obtain new data. In an experimental study, the variable of interest is identified. Then one or more factors in the study are controlled so that data can be obtained about how the factors influence the variables. In observational studies, no attempt is made to control or influence the variables of interest. A survey is perhaps the most common type of observational study.
Analyzing the Data
Statistical data analysis divides the methods for analyzing data into two categories: exploratory methods and confirmatory methods. Exploratory methods are used to discover what the data seems to be saying by using simple arithmetic and easy-to-draw pictures to summarize data. Confirmatory methods use ideas from probability theory in the attempt to answer specific questions. Probability is important in decision making because it provides a mechanism for measuring, expressing, and analyzing the uncertainties associated with future events. The majority of the topics addressed in this course fall under this heading.
Reporting the Results
Through inferences, an estimate or test claims about the characteristics of a population can be obtained from a sample. The results may be reported in the form of a table, a graph or a set of percentages. Because only a small collection (sample) has been examined and not an entire population, the reported results must reflect the uncertainty through the use of probability statements and intervals of values.
To conclude, a critical aspect of managing any organization is planning for the future. Good judgment, intuition, and an awareness of the state of the economy may give a manager a rough idea or "feeling" of what is likely to happen in the future. However, converting that feeling into a number that can be used effectively is difficult. Statistical data analysis helps managers forecast and predict future aspects of a business operation. The most successful managers and decision makers are the ones who can understand the information and use it effectively.
visit also Different
Approaches to Statistical Thinking
Data Processing: Coding, Typing, and Editing
Data are often recorded manually on data sheets. Unless the numbers of observations and variables are small the data must be analyzed on a computer. The data will then go through three stages:Coding: the data are transferred, if necessary to coded sheets.
Typing: the data are typed and stored by at least two independent data entry persons. For example, when the Current Population Survey and other monthly surveys were taken using paper questionnaires, the U.S. Census Bureau used double key data entry.
Editing: the data are checked by comparing the two independent typed data. The standard practice for key-entering data from paper questionnaires is to key in all the data twice. Ideally, the second time should be done by a different key entry operator whose job specifically includes verifying mismatches between the original and second entries. It is believed that this "double-key/verification" method produces a 99.8% accuracy rate for total keystrokes.
Types of error: Recording error, typing error, transcription error
(incorrect copying), Inversion (e.g., 123.45 is typed as 123.54), Repetition
(when a number is repeated), Deliberate error.
An Alternative Approach for Estimating a Regression Line
The following approach is the so-called "distribution-free method" for estimating parameters in a simple regression y = mx + b:- Rewrite y = mx + b as b = -xm + y.
- Every data point (xi, yi) corresponds to a line b = -xi m + yi in the Cartesian coordinates plane (m, b), and an estimate of m and b can be obtained from the intersection of pairs of such lines. There are at most n(n+1)/2 such estimates.
- Take the medians to get the final estimates.
References and Further Readings:
Cornish-Bowden A., Analysis
of Enzyme Kinetic Data, Oxford Univ Press, 1995.
Hald A., A History
of Mathematical Statistics: From 1750 to 1930, Wiley, New York, 1998.
Among others, the author points out that in the beginning of 18-th Century
researches had four different methods to solve fitting problems: The
Mayer-Laplace method of averages, The Boscovich-Laplace method of least
absolute deviations, Laplace method of minimizing the largest absolute
residual and the Legendre method of minimizing the sum of squared residuals.
The only single way of choosing between these methods was: to compare results
of estimates and residuals.
Multivariate Data Analysis
Data are easy to collect; what we really need in complex problem solving is information. We may view a data base as a domain that requires probes and tools to extract relevant information. As in the measurement process itself, appropriate instruments of reasoning must be applied to the data interpretation task. Effective tools serve in two capacities: to summarize the data and to assist in interpretation. The objectives of interpretive aids are to reveal the data at several levels of detail.Exploring the fuzzy data picture sometimes requires a wide-angle lens to view its totality. At other times it requires a closeup lens to focus on fine detail. The graphically based tools that we use provide this flexibility. Most chemical systems are complex because they involve many variables and there are many interactions among the variables. Therefore, chemometric techniques rely upon multivariate statistical and mathematical tools to uncover interactions and reduce the dimensionality of the data.
Principal component analysis used for exploring data. Two closely related techniques, principal component analysis and factor analysis, are used to reduce the dimensionality of multivariate data. In these techniques correlations and interactions among the variables are summarized in terms of a small number of underlying factors. The methods rapidly identify key variables or groups of variables that control the system under study. The resulting dimension reduction also permits graphical representation of the data so that significant relationships among observations or samples can be identified.
Other techniques include Multidimensional Scaling, Cluster Analysis, and Correspondence Analysis.
Multivariate analysis is a branch of statistics involving the consideration of objects on each of which are observed the values of a number of variables. A wide range of methods is used for the analysis of multivariate data, and this course will give a view of the variety of methods available, as well as going into some of them in detail. Multivariate techniques are used across the whole range of fields of statistical application: in medicine, physical and biological sciences, economics and social science, and of course in many industrial and commercial applications.
References and Further Readings:
Chatfield C., and A. Collins,
Introduction to Multivariate Analysis, Chapman and Hall, 1980.
Hoyle R., Statistical Strategies for small Sample Research, Thousand Oaks,
CA, Sage, 1999.
Krzanowski W., Principles of Multivariate Analysis: A
User's Perspective, Clarendon Press, 1988.
Mardia K., J. Kent and J.
Bibby, Multivariate Analysis, Academic Press, 1979.
The Meaning and Interpretation of P-values (what the data say?)
The P-value, which directly depends on a given sample, attempts to provide a measure of the strength of the results of a test, in contrast to a simple reject or do not reject. If the null hypothesis is true and the chance of random variation is the only reason for sample differences, then the P-value is a quantitative measure to feed into the decision making process as evidence. The following table provides a reasonable interpretation of P-values:
|
|
|
| P< 0.01 | very strong evidence against H0 |
0.01 P < 0.05 |
moderate evidence against H0 |
| 0.05 P < 0.10 |
suggestive evidence against H0 |
| 0.10 P |
little or no real evidence against H0 |
This interpretation is widely accepted, and many scientific journals routinely publish papers using such an interpretation for the result of test of hypothesis.
For the fixed-sample size, when the number of realizations is decided in
advance, the distribution of p is uniform (assuming the null hypothesis). We
would express this as P(p
x) = x. That means the criterion of p <0.05
achieves a of 0.05.
When a p-value is associated with a set of data, it is a measure of the probability that the data could have arisen as a random sample from some population described by the statistical (testing) model.
A p-value is a measure of how much evidence you have against the null hypothesis. The smaller the p-value, the more evidence you have. One may combine the p-value with the significance level to make decision on a given test of hypothesis. In such a case, if the p-value is less than some threshold (usually .05, sometimes a bit larger like 0.1 or a bit smaller like .01) then you reject the null hypothesis.
Understand that the distribution of p-values under null hypothesis H0 is uniform, and thus does not depend on a particular form of the statistical test. In a statistical hypothesis test, the P value is the probability of observing a test statistic at least as extreme as the value actually observed, assuming that the null hypothesis is true. The value of p is defined with respect to a distribution. Therefore, we could call it "model-distributional hypothesis" rather than "the null hypothesis".
In short, it simply means that if the null had been true, the p value is the probability against the null in that case. The p-value is determined by the observed value, however, this makes it difficult to even state the inverse of p.
P-value for Standard Normal and t-statistics
Conversion of a z-statistic Into a (one-side) P-valueINPUT "Z : ", ZValue a1# = .31938153# a2# = -.356563782# a3# = 1.781477937# a4# = -1.821255978# a5# = 1.330274429# w1# = ABS(ZValue) w# = 1 / (1 + .2316419# * w1#) w1# = .39894228# * EXP(-.5 * w1# * w1#) p0# = w# *(a1# + w# *(a2# + w# *(a3# + w# * (a4# + a5# * w#)))) p0# = (w1# * p0#) IF ZValue > 0 THEN p0# = 1 - p0# END IF PRINT p0#
Area from 0 to z for normal density: EXP(-((83*Z+351)*Z+562)*Z/(703+165*Z))/2
Below is a silimar program:
INPUT z
a1 = .31938153#
a2 = -.356563782#
a3 = 1.781477937#
a4 = -1.821255978#
a5 = 1.330274429#
w1 = ABS(z)
w = 1 / (1 + .2316419 * w1)
w1 = .39894228# * EXP(-.5 * w1 * w1)
p0 = w * (a1 + w * (a2 + w * (a3 + w * (a4 + a5 * w))))
p0 = w1 * p0
PRINT ABS(p0);
Conversion of a z-statistic Into a (one-side)
P-value: in C++ code double __declspec(dllexport) NormalProb(double z)
{
const double a1 = .31938153;
const double a2 = -.356563782;
const double a3 = 1.781477937;
const double a4 = -1.821255978;
const double a5 = 1.330274429;
double w1 = absd(z);
double w = 1 / (1 + .2316419 * w1);
w1 = .39894228 * exp(-0.5 * w1 * w1);
double p0 = w * (a1 + w * (a2 + w * (a3 + w * (a4 + a5 * w))));
p0 = w1 * p0;
return absd(p0);
}
Conversion of a t-statistics Into a (one-side) P-value: C++
double __declspec(dllexport) TProb(double t, int df)
{
double a = 0.36338023;
double w = atan(t / sqrt(df));
double s = sin(w);
double c = cos(w);
double t1, t2;
int j1, j2, k2;
if (df % 2 == 0) // even
{
t1 = s;
if (df == 2) // special case df=2
return (0.5 * (1 + t1));
t2 = s;
j1 = -1;
j2 = 0;
k2 = (df - 2) / 2;
}
else
{
t1 = w;
if (df == 1) // special case df=1
return 1 - (0.5 * (1 + (t1 * (1 - a))));
t2 = s * c;
t1 = t1 + t2;
if (df == 3) // special case df=3
return 1 - (0.5 * (1 + (t1 * (1 - a))));
j1 = 0;
j2 = 1;
k2 = (df - 3)/2;
}
for (int i=1; i> = k2; i++)
{
j1 = j1 + 2;
j2 = j2 + 2;
t2 = t2 * c * c * j1/j2;
t1 = t1 + t2;
}
return 1 - (0.5 * (1 + (t1 * (1 - a * (df % 2)))));
}
For more, visit Statistics.
Accuracy, Precision, Robustness, and Quality
Accuracy refers to the closeness of the measurements to the "actual" or "real" value of the physical quantity, whereas the term precision is used to indicate the closeness with which the measurements agree with one another quite independently of any systematic error involved. Therefore, an "accurate" estimate has small bias. A "precise" estimate has both small bias and variance. Quality is proportion to the inverse of variance.The robustness of a procedure is the extent to which its properties do not depend on those assumptions which you do not wish to make. This is a modification of Box's original version, and this includes Bayesian considerations, loss as well as prior. The central limit theorem (CLT) and the Gauss-Markov Theorem qualify as robustness theorems, but the Huber-Hempel definition does not qualify as a robustness theorem.
We must always distinguish between bias robustness and efficiency robustness. It seems obvious to me that no statistical procedure can be robust in all senses. One needs to be more specific about what the procedure must be protected against. If the sample mean is sometimes seen as a robust estimator, it is because the CLT guarantees a 0 bias for large samples regardless of the underlying distribution. This estimator is bias robust, but it is clearly not efficiency robust as its variance can increase endlessly. That variance can even be infinite if the underlying distribution is Cauchy or Pareto with a large scale parameter. This is the reason for which the sample mean lacks robustness according to Huber-Hampel definition. The problem is that the M-estimator advocated by Huber, Hampel and a couple of other folks is bias robust only if the underlying distribution is symmetric.
In the context of survey sampling, two types of statistical inferences are available: the model-based inference and the design-based inference which exploits only the randomization entailed by the sampling process (no assumption needed about the model). Unbiased design-based estimators are usually referred to as robust estimators because the unbiasedness is true for all possible distributions. It seems clear however, that these estimators can still be of poor quality as the variance that can be unduly large.
However, others people will use the word in other (imprecise) ways. Kendall's Vol. 2, Advanced Theory of Statistics, also cites Box, 1953; and he makes a less useful statement about assumptions. In addition, Kendall states in one place that robustness means (merely) that the test size, a, remains constant under different conditions. This is what people are using, apparently, when they claim that two-tailed t-tests are "robust" even when variances and sample sizes are unequal. I, personally, do not like to call the tests robust when the two versions of the t-test, which are approximately equally robust, may have 90% different results when you compare which samples fall into the rejection interval (or region).
I find it easier to use the phrase, "There is a robust difference", which
means that the same finding comes up no matter how you perform the test, what
(justifiable) transformation you use, where you split the scores to test on
dichotomies, etc., or what outside influences you hold constant as covariates.
What is a Meta-Analysis?
A Meta-analysis deals with a set of RESULTs to give an overall RESULT that is comprehensive and valid.a) Especially when Effect-sizes are rather small, the hope is that one can gain good power by essentially pretending to have the larger N as a valid, combined sample.
b) When effect sizes are rather large, then the extra POWER is not needed for main effects of design: Instead, it theoretically could be possible to look at contrasts between the slight variations in the studies themselves.
For example, to compare two effect sizes (r) obtained by two separate studies, you may use:
Z = (z1 - z2)/[(1/n1-3) + (1/n2-3)]1/2
where z1 and z2 are Fisher transformations of r, and the two ni's in the denominator represent the sample size for each study.
If you really trust that "all things being equal" will hold up. The typical "meta" study does not do the tests for homogeneity that should be required
In other words:
1. there is a body of research/data literature that you would like to summarize
2. one gathers together all the admissible examples of this literature (note: some might be discarded for various reasons)
3. certain details of each investigation are deciphered ... most important would be the effect that has or has not been found. ie, how much larger in sd units is the treatment group's performance compared to one or more controls.
4. call the values in each of the investigations in #3 .. mini effect sizes.
5. across all admissible data sets, you attempt to summarize the overall effect size by forming a set of individual effects ... and using an overall sd as the divisor .. thus yielding essentially an average effect size.
6. in the meta analysis literature ... sometimes these effect sizes are further labeled as small, medium, or large ....
You can look at effect sizes in many different ways .. across different factors and variables. but, in a nutshell, this is what is done.
I recall a case in physics, in which, after a phenomenon had been observed in air, emulsion data were examined. The theory would have about a 9% effect in emulsion, and behold, the published data gave 15%. As it happens, there was no significant difference (practical, not statistical) in the theory, and also no error in the data. It was just that the results of experiments in which nothing statistically significant was found were not reported.
This non-reporting of such experiments, and often of the specific results which were not statistically significant, which introduces major biases. This is also combined with the totally erroneous attitude of researchers that statistically significant results are the important ones, and than if there is no significance, the effect was not important. We really need to differentiate between the term "statistically significant", and the usual word significant.
Meta-analysis is a controversial type of literature review in which the results of individual randomized controlled studies are pooled together to try to get an estimate of the effect of the intervention being studied. It increases statistical power and is used to resolve the problem of reports which disagree with each other. It's not easy to do well and there are many inherent problems.
For details, see, Meta-Analysis in Social Research, by Glass,
McGraw and Smith, 1987.
What Is the Effect Size
Effect size (ES) is a ratio of a mean difference to a standard deviation, i.e. it is a form of z-score. Suppose an experimental treatment group has a mean score of Xe and a control group has a mean score of Xc and a standard deviation of Sc, then the effect size is equal to (Xe - Xc)/ScEffect size permits the comparative effect of different treatments to be compared, even when based on different samples and different measuring instruments.
Therefore, the ES is the mean difference between the control group and the treatment group. Howevere, by Glass's method, ES is (mean1 - mean2)/SD of control group while by Hunter-Schmit's method, ES is (mean1 - mean2)/pooled SD and then adjusted by instrument reliability coefficient. ES is commonly used in meta-analysis and power analysis.
References and Further Readings:
Glass G., McGaw B., and M.
Smith, Meta-analysis in Social Research, Newbury Park, CA: Sage, 1981.
Cooper H., and L. Hedges, The Handbook of Research Synthesis, NY,
Russell Sage, 1994.
Bias Reduction Techniques
The most effective tools for bias reduction is non-biased estimators are the Bootstrap and the Jackknifing.According to legend, Baron Munchausen saved himself from drowning in quicksand by pulling himself up using only his bootstraps. The statistical bootstrap, which uses resampling from a given set of data to mimic the variability that produced the data in the first place, has a rather more dependable theoretical basis and can be a highly effective procedure for estimation of error quantities in statistical problems.
Bootstrap is to create a virtual population by duplicating the same sample over and over, and then re-samples from the virtual population to form a reference set. Then you compare your original sample with the reference set to get the exact p-value. Very often, a certain structure is "assumed" so that a residual is computed for each case. What is then re-sampled is from the set of residuals, which are then added to those assumed structures, before some statistic is evaluated. The purpose is often to estimate a P-level.
Jackknife is to re-compute the data by leaving on observation out each time. Leave-one-out replication gives you the same Case-estimates, I think, as the proper jack-knife estimation. Jackknifing does a bit of logical folding (whence, 'jackknife' -- look it up) to provide estimators of coefficients and error that (you hope) will have reduced bias.
References and Further Readings:
Efron B., The Jackknife, The
Bootstrap and Other Resampling Plans, SIAM, Philadelphia, 1982.
Efron
B., and R. Tibshirani, An Introduction to the Bootstrap, Chapman &
Hall, 1994.
Shao J., and D. Tu, The Jackknife and Bootstrap,
Springer Verlag, 1995.
Area Under Standard Normal Curve
Approximate area under standard normal curve from 0 to Z isZ(4.4-Z)/10 for 0
Number of Class Interval in Histogram
Before we can construct our frequency distribution we must determine how many classes we should use. This is purely arbitrary, but too few classes or too many classes will not provide as clear a picture as can be obtained with some more nearly optimum number. An empirical relationship (known as Sturges' rule) which seems to hold and which may be used as a guide to the number of classes (k) is given byk = the smallest integer greater than or equal to 1 + Log(n) / Log (2) = 1 + 3.332Log(n)
To have an 'optimum' you need some measure of quality - presumably in this case, the 'best' way to display whatever information is available in the data. The sample size contributes to this, so the usual guidelines are to use between 5 and 15 classes, with more classes possible if you have a larger sample. You take into account a preference for tidy class widths, preferably a multiple of 5 or 10, because this makes it easier to appreciate the scale.
Beyond this it becomes a matter of judgement - try out a range of class widths and choose the one that works best. (This assumes you have a computer and can generate alternative histograms fairly readily).
There are often management issues that come into it as well. For example, if your data is to be compared to similar data - such as prior studies, or from other countries - you are restricted to the intervals used therein.
If the histogram is very skewed, then unequal classes should be considered. Use narrow classes where the class frequencies are high, wide classes where they are low.
The following approaches are common:
Let n be the sample size, then number of class interval could be
MIN {
n, 10Log(n) }.
Thus for 200 observations you would use 14 intervals but for 2000 you would use 33.
Alternatively,
1. Find the range (highest value - lowest value).
2. Divide the range by a reasonable interval size: 2, 3, 5, 10 or a = multiple of 10.
3. Aim for no fewer than 5 intervals and no more than 15.
Structural Equation Modeling
The structural equation modeling techniques are used to study relations among variables. The relations are typically assumed to be linear. In social and behavioral research most phenomena are influenced by a large number of determinants which typically have a complex pattern of interrelationships. To understand the relative importance of these determinants their relations must be adequately represented in a model, which may be done with structural equation modeling.A structural equation model may apply to one group of cases or to multiple groups of cases. When multiple groups are analyzed parameters may be constrained to be equal across two or more groups. When two or more groups are analyzed, means on observed and latent variables may also be included in the model.
As an application, how do you test the equality of regression slopes coming from the same sample using 3 different measuring methods? You could use a structural modeling approach.
1 - Standardize all three data sets prior to the analysis because b weights are also a function of the variance of the predictor variable and with standardization, you remove this source.
2 - Model the dependent variable as the effect from all three measures and obtain the path coefficient (b weight) for each one.
3 - Then fit a model in which the three path coefficients are constrained to be equal. If a significant decrement in fit occurs, the paths are not equal.
References and Further Readings:
Schumacker R., and R. Lomax,
A Beginner's Guide to Structural Equation Modeling, Lawrence Erlbaum,
New Jersey, 1996.
Visit also the Web sites
Structural Equation Modeling on the
Internet
Research Methods &
Statistics Resources
Structural
Analysis
Tri-linear Coordinates Triangle
A "ternary diagram" is usually used to show the change of opinion (FOR - AGAINST - UNDECIDED). The triangular diagram used first by the chemist Willard Gibbs in his studies on phase transitions. It is based on the proposition from geometry that in an equilateral triangle, the sum of the distances from any point to the three sides is constant. This implies that the percent composition of a mixture of three substances can be represented as a point in such a diagram, since the sum of the percentages is constant (100). The three vertices are the points of the pure substances.The same holds for the "composition" of the opinions in a population. When percents for, against and undecided sum to 100, the same technique for presentation can be used. See the diagram below, which should be viewed with a non-proportional letter. True equilateral may not be preserved in transmission. E.g. let the initial composition of opinions be given by 1. That is, few undecided, roughly equally as much for as against. Let another composition be given by point 2. This point represents a higher percentage undecided and, among the decided, a majority of "for".
Internal and Inter-rater Reliability
"Internal reliability" of a scale is often measured by Cronbach's coefficient a. It is relevant when you will compute a total score and you want to know its reliability, based on no other rating. The "reliability" is *estimated* from the average correlation, and from the number of items, since a longer scale will (presumably) be more reliable. Whether the items have the same means is not usually important.Tau-equivalent: The true scores on items are assumed to differ from each other by no more than a constant. For a to equal the reliability of measure, the items comprising it have to be at a least tau-equivalent, if this assumption is not met, a is lower bound estimate of reliability.
Congeneric measures: This least restrictive model within the framework of classical test theory requires only that true scores on measures said to be measuring the same phenomenon be perfectly correlated. Consequently, on congeneric measures, error variances, true-score means, and true-score variances may be unequal
For "inter-rater" reliability, one distinction is that the importance lies with the reliability of the single rating. Suppose we have the following data
Participants Time Q1 Q2 Q3 to Q17 001 1 4 5 4 4 002 1 3 4 3 3 001 2 4 4 5 3 etc.By examining the data, I think one cannot do better than looking at the paired t-test and Pearson correlations between each pair of raters - the t-test tells you whether the means are different, while the correlation tells you whether the judgments are otherwise consistent.
Unlike the Pearson, the "intra-class" correlation assumes that the raters do have the same mean. It is not bad as an overall summary, and it is precisely what some editors do want to see presented for reliability across raters. It is both a plus and a minus, that there are a few different formulas for intra-class correlation, depending on whose reliability is being estimated.
For purposes such as planning the Power for a proposed study, it does matter whether the raters to be used will be exactly the same individuals. A good methodology to apply in such cases, is the Bland & Altman analysis.
SPSS Commands:
Reliability (Alpha, KR-20) RELIABILITYSAS Commands:
Reliability (Alpha, KR-20) CORR ALPHA
Visit also the Web site Common Correlation
and Reliability Analysis.
When to Use Nonparametric Technique?
One must use statistical technique called nonparametric if it satisfies at least on of the following five types of criteria:1. The data entering the analysis are enumerative - that is, count data representing the number of observations in each category or cross-category.
2. The data are measured and /or analyzed using a nominal scale of measurement.
3. The data are measured and /or analyzed using an ordinal scale of measurement.
4. The inference does not concern a parameter in the population distribution - as, for example, the hypothesis that a time-ordered set of observations exhibits a random pattern.
5. The probability distribution of the statistic upon which the the analysis is based is not dependent upon specific information or assumptions about the population(s) which the sample(s) are drawn, but only on general assumptions, such as a continuous and/or symmetric population distribution.
By this definition, the distinction of nonparametric is accorded either because of the level of measurement used or required for the analysis, as in types 1 through 3; the type of inference, as in type 4 or the generality of the assumptions made about the population distribution, as in type 5.
For example one may use the Mann-Whitney Rank Test as a nonparametric alternative to Students T-test when one does not have normally distributed data.
Mann-Whitney: To be used with two independent groups (analogous to the
independent groups t-test)
Wilcoxon: To be used with two related (i.e.,
matched or repeated) groups (analogous to the related samples t-test)
Kruskall-Wallis: To be used with two or more independent groups (analogous
to the single-factor between-subjects ANOVA)
Friedman: To be used with two
or more related groups (analogous to the single-factor within-subjects ANOVA)
Analysis of Incomplete Data
Methods dealing with analysis of data with missing values can be classified into:- Analysis of complete cases, including weighting adjustments,
-
Imputation methods, and extensions to multiple imputation, and
- Methods
that analyze the incomplete data directly without requiring a rectangular data
set, such as maximum likelihood and Bayesian methods.
Multiple imputation (MI) is a general paradigm for the analysis of incomplete data. Each missing datum is replaced by m> 1 simulated values, producing m simulated versions of the complete data. Each version is analyzed by standard complete-data methods, and the results are combined using simple rules to produce inferential statements that incorporate missing data uncertainty. The focus is on the practice of MI for real statistical problems in modern computing environments.
References and Further Readings:
Rubin D., Multiple Imputation
for Nonresponse in Surveys, New York, Wiley, 1987.
Schafer J.,
Analysis of Incomplete Multivariate Data, London, Chapman and Hall,
1997.
Little R., and D. Rubin, Statistical Analysis with Missing Data, New
York, Wiley, 1987.
Interactions in ANOVA and Regression Analysis
Interactions are ignored only if you permit it. For historical reasons, ANOVA programs generally produce all possible interactions, while (multiple) regression programs generally do not produce any interactions - at least, not so routinely. So it's up to the user to construct interaction terms when using regression to analyze a problem where interactions are, or may be, of interest. (By "interaction terms" I mean variables that carry the interaction information, included as predictors in the regression model.)Regression is the estimation of the conditional expectation of a random variable given another (possibly vector-valued) random variable.
The easiest construction is to multiply together the predictors whose interaction is to be included. When there are more than about three predictors, and especially if the raw variables take values that are distant from zero (like number of items right), the various products (for the numerous interactions that can be generated) tend to be highly correlated with each other, and with the original predictors. This is sometimes called "the problem of multicollinearity", although it would more accurately be described as spurious multicollinearity. It is possible, and often to be recommended, to adjust the raw products so as to make them orthogonal to the original variables (and to lower-order interaction terms as well).
What does it mean if the standard error term is high? Multicolinearity is not the only factor that can cause large SE's for estimators of "slope" coefficients any regression models. SE's are inversely proportional to the range of variability in the predictor variable. For example, if you were estimating the linear association between weight (x) and some dichotomous outcome and x=(50,50,50,50,51,51,53,55,60,62) the SE would be much larger than if x=(10,20,30,40,50,60,70,80,90,100) all else being equal. There is a lesson here for the planning of experiments. To increase the precision of estimators, increase the range of the input. Another cause of large SE's is a small number of "event" observations or a small number of "non-event" observations (analogous to small variance in the outcome variable). This is not strictly controllable but will increase all estimator SE's (not just an individual SE). There is also another cause of high standard errors, it's called serial correlation. This problem is frequent, if not typical, when using time-series, since in that case the stochastic disturbance term will often reflect variables, not included explicitly in the model, that may change slowly as time passes by.
In a linear model representing the variation in a dependent variable Y as a linear function of several explanatory variables, interaction between two explanatory variables X and W can be represented by their product: that is, by the variable created by multiplying them together. Algebraically such a model is represented by:
Y = a +b1X + b2 W + b3 XW + e .
When X and W are category systems. This equation describes a two-way analysis of variance (ANOV) model; when X and W are (quasi-)continuous variables, this equation describes a multiple linear regression (MLR) model.
In ANOV contexts, the existence of an interaction can be described as a difference between differences: the difference in means between two levels of X at one value of W is not the same as the difference in the corresponding means at another value of W, and this not-the-same-ness constitutes the interaction between X and W; it is quantified by the value of b3.
In MLR contexts, an interaction implies a change in the slope (of the
regression of Y on X) from one value of W to another value of W (or,
equivalently, a change in the slope of the regression of Y on W for different
values of X): in a two-predictor regression with interaction, the response
surface is not a plane but a twisted surface (like "a bent cookie tin", in
Darlington's (1990) phrase). The change of slope is quantified by the value of
b 3. To resolve the problem of multi-collinearity., visit the Web site Multicollinearity".
Variance of Nonlinear Random Functions
The variation in nonlinear function of several random variables can be approximated by the "delta method". An approximate variance for a smooth function f(X, Y) of two random variables (X, Y) is obtained by a approximating f(X, Y) by the linear terms of its Taylor expansion in the neighborhood of about the sample means of X and Y.For example, the variance of XY and X/Y based on a large sample size are approximated by:
and
What Is a Geometric Mean
The geometric mean of n positive numerical values is the nth root of the product of the n values. The denominator of the Pearson correlation coefficient is the geometric mean of the two variances. It is useful for averaging "product moment" values.Suppose you have two positive data points x and y, then the geometric mean of these numbers is a number g such that a/g = g/b, and the arithmetic mean is a number a such that x - a = a - y.
The geometric means are used extensively by the U.S. Bureau of Labor Statistics ["Geomeans" as they call them] in the computation of the U.S. Consumer Price Index. The geomeans are also used in price indexes. The statistical use of geometric mean is for index numbers such as the Fisher's ideal index.
If some values are very large in magnitude and others are small, then the geometric mean is a better average. In a Geometric series, the most meaningful average is the geometric mean. The arithmetic mean is very biased toward the larger numbers in the series.
As an example, suppose sales of a certain item increase to 110% in the first year and to 150% of that in the second year. For simplicity, assume you sold 100 items initially. Then the number sold in the first year is 110 and the number sold in the second is 150% x 110 = 165. The arithmetic average of 110% and 150% is 130% so that we would incorrectly estimate that the number sold in the first year is 130 and the number in the second year is 169. The geometric mean of 110% and 150% is 165% so that we would correctly estimate that we would sell 165 items in the second year.
As another similar example, if a mutual fund goes up by 50% one year and down by 50% the next year, and you hold a unit throughout both years, you have lost money at the end. For every dollar you started with, you have now got 75c. Thus, the performance is different from gaining (50%-50%)/2 (= 0%). It is the same as changing by a multiplicative factor of SQRT(1.5 x 0.5) = 0.866 each year. In a multiplicative process, the one value that can be substituted for each of a set of values to give the same "overall effect" is the geometric mean, not the arithmetic mean. As money tends to multiplicatively ("it takes money to make money"), financial data are often better combined in this way.
As a survey analysis example, give a sample of people a list of, say 10, crimes ranging in seriousness:
Ask each respondent to give any numerical value they feel to any crime in the list (e.g. someone might decide to call arson 100). Then ask them to rate each crime in the list on a ratio scale. If a respondent thought rape was five times as bad as arson, then a value of 500 would be assigned, theft a quarter as bad, 25. Suppose we now wanted the "average" rating across respondents given to each crime. Since every respondent is using their own base value, the arithmetic mean would be useless: people who used large numbers as their base value would "swamp" those who had chosen small numbers. However, the geometric mean -- the nth root of the product of ratings for each crime of the n respondents -- gives equal weighting to all responses. I've used this in a class exercise and it works nicely.
It is often good to log-transform such data before regression, ANOVA, etc. These statistical techniques give inferences about the arithmetic mean (which is intimately connected with the least-squares error measure); however, the arithmetic mean of log-transformed data is the log of the geometric mean of the data. So, for instance, a t test on log-transformed data is really a test for location of the geometric mean.
References and Further Readings:
Langley R., Practical
Statistics Simply Explained, 1970, Dover Press.
What Is Central Limit Theorem?
For practical purposes, the main idea of the central limit theorem (CLT) is that the average of a sample of observations drawn from some population with any shape-distribution is approximately distributed as a normal distribution if certain conditions are met. In theoretical statistics there are several versions of the central limit theorem depending on how these conditions are specified. These are concerned with the types of assumptions made about the distribution of the parent population (population from which the sample is drawn) and the actual sampling procedure.One of the simplest versions of the theorem says that if is a random sample of size n (say, n> 30) from an infinite population finite standard deviation , then the standardized sample mean converges to a standard normal distribution or, equivalently, the sample mean approaches a normal distribution with mean equal to the population mean and standard deviation equal to standard deviation of the population divided by square root of sample size n. In applications of the central limit theorem to practical problems in statistical inference, however, statisticians are more interested in how closely the approximate distribution of the sample mean follows a normal distribution for finite sample sizes, than the limiting distribution itself. Sufficiently close agreement with a normal distribution allows statisticians to use normal theory for making inferences about population parameters (such as the mean ) using the sample mean, irrespective of the actual form of the parent population.
It is well known that whatever the parent population is, the standardized variable will have a distribution with a mean 0 and standard deviation 1 under random sampling. Moreover, if the parent population is normal, then is distributed exactly as a standard normal variable for any positive integer n. The central limit theorem states the remarkable result that, even when the parent population is non-normal, the standardized variable is approximately normal if the sample size is large enough (say, > 30). It is generally not possible to state conditions under which the approximation given by the central limit theorem works and what sample sizes are needed before the approximation becomes good enough. As a general guideline, statisticians have used the prescription that if the parent distribution is symmetric and relatively short-tailed, then the sample mean reaches approximate normality for smaller samples than if the parent population is skewed or long-tailed.
On e must study the behavior of the mean of samples of different sizes drawn from a variety of parent populations. Examining sampling distributions of sample means computed from samples of different sizes drawn from a variety of distributions, allow us to gain some insight into the behavior of the sample mean under those specific conditions as well as examine the validity of the guidelines mentioned above for using the central limit theorem in practice.
Under certain conditions, in large samples, the sampling distribution of the sample mean can be approximated by a normal distribution. The sample size needed for the approximation to be adequate depends strongly on the shape of the parent distribution. Symmetry (or lack thereof) is particularly important. For a symmetric parent distribution, even if very different from the shape of a normal distribution, an adequate approximation can be obtained with small samples (e.g., 10 or 12 for the uniform distribution). For symmetric short-tailed parent distributions, the sample mean reaches approximate normality for smaller samples than if the parent population is skewed and long-tailed. In some extreme cases (e.g. binomial with ) samples sizes far exceeding the typical guidelines (say, 30) are needed for an adequate approximation. For some distributions without first and second moments (e.g., Cauchy), the central limit theorem does not hold.
Review also Central Limit
Theorem Applet, CLT,
and Quincunx
to illustrate the Central Limit Theorem.
What is a Sampling Distribution?
The main idea of statistical inference is to take a random sample from a population and then to use the information from the sample to make inferences about particular population characteristics such as the mean (measure of central tendency), the standard deviation (measure of spread) or the proportion of units in the population that have a certain characteristic. Sampling saves money, time, and effort. Additionally, a sample can, in some cases, provide as much or more accuracy than a corresponding study that would attempt to investigate an entire population-careful collection of data from a sample will often provide better information than a less careful study that tries to look at everything.We will study the behavior of the mean of sample values from a different specified populations. Because a sample examines only part of a population, the sample mean will not exactly equal the corresponding mean of the population. Thus, an important consideration for those planning and interpreting sampling results, is the degree to which sample estimates, such as the sample mean, will agree with the corresponding population characteristic.
In practice, only one sample is usually taken (in some cases a small ``pilot sample'' is used to test the data-gathering mechanisms and to get preliminary information for planning the main sampling scheme). However, for purposes of understanding the degree to which sample means will agree with the corresponding population mean, it is useful to consider what would happen if 10, or 50, or 100 separate sampling studies, of the same type, were conducted. How consistent would the results be across these different studies? If we could see that the results from each of the samples would be nearly the same (and nearly correct!), then we would have confidence in the single sample that will actually be used. On the other hand, seeing that answers from the repeated samples were too variable for the needed accuracy would suggest that a different sampling plan (perhaps with a larger sample size) should be used.
A sampling distribution is used to describe the distribution of outcomes that one would observe from replication of a particular sampling plan.
Know that to estimate means to esteem (to give value to).
Know that estimates computed from one sample will be different from estimates that would be computed from another sample.
Understand that estimates are expected to differ from the population characteristics (parameters) that we are trying to estimate, but that the properties of sampling distributions allow us to quantify, probabilistically, how they will differ.
Understand that different statistics have different sampling distributions with distribution shape depending on (a) the specific statistic, (b) the sample size, and (c) the parent distribution.
Understand the relationship between sample size and the distribution of sample estimates.
Understand that the variability in a sampling distribution can be reduced by increasing the sample size.
See that in large samples, many sampling distributions can be approximated with a normal distribution.
Visit also the following Web sites: Sample, and
Sampling
Distribution Applet
Least Squares Models
Many problems in analyzing data involve describing how variables are related. The simplest of all models describing the relationship between two variables is a linear, or straight-line, model. The simplest method of fitting a linear model is to ``eye-ball'' a line through the data on a plot, but a more elegant, and conventional method is that of least squares, which finds the line minimizing the sum of distances between observed points and the fitted line.Realize that fitting the ``best'' line by eye is difficult, especially when there is a lot of residual variability in the data.
Know that there is a simple connection between the numerical coefficients in the regression equation and the slope and intercept of regression line.
Know that a single summary statistic like a correlation coefficient or does not tell the whole story. A scatter plot is an essential complement to examining the relationship between the two variables.
Know that the model checking is an essential part of the process of statistical modelling. After all, conclusions based on models that do not properly describe an observed set of data will be invalid.
Know the impact of violation of regression model assumptions (i.e.,
conditions) and possible solutions by analyzing the residuals.
Least Median of Squares Models
The standard least squares techniques for estimation in linear models are not robust in the sense that outliers or contaminated data can strongly influence estimates. A robust technique which protects against contamination is least median of squares (LMS) estimation. An extension of LMS estimation to generalized linear models, giving rise to the least median of deviance (LMD) estimator.You Must Look at Your Scattergrams!
Learn that given a set data the regression line is unique. However, the inverse of this statement is not true. The following interesting example is from, D. Moore (1997) book, page 349:Data set A: x 10 8 13 9 11 14 y 8.04 6.95 7.58 8.81 8.33 9.96 x 6 4 12 7 5 y 7.24 4.26 10.84 4.82 5.68 Data set B: x 10 8 13 9 11 14 y 9.14 8.14 8.74 8.77 9.26 8.10 x 6 4 12 7 5 y 6.13 3.10 9.13 7.26 4.74 Data set C: x 8 8 8 8 8 8 y 6.58 5.76 7.71 8.84 8.47 7.04 x 8 8 8 8 19 y 5.25 5.56 7.91 6.89 12.50
All three sets have the same correlation and regression line. The important moral is look at your scattergrams.
How to produce a numerical example where the two scatterplots show clearly different relationships (strengths) but yield the same covariance? Perform the following steps:
1. Produce two sets of (X,Y) values that have different correlations;
2.
Calculate the two covariances, say C1 and C2;
3. Suppose you want to make
C2 equal to C1. Then you want to multiply C2 by
(C1/C2);
4. Since C =
r.Sx.Sy, you
want two numbers (one of them might be 1), a and b such that
a.b =
(C1/C2);
5. Multiply all values of X in set 2 by a, and all values of Y by
b: for the new variables,
C = r.a.b.Sx.Sy = C2.(C1/C2) =
C1.
An interesting numerical example showing two identical scatterplots but
with differing covariance is the following: Consider a data set of (X, Y)
values, with covariance C1. Now let V = 2X, and W = 3Y. The covariance of V
and W will be 2(3) = 6 times C1, but the correlation between V and W is the
same as the correlation between X and Y.
Power of a Test
Significance tests are based on certain assumptions: The data have to be random samples out of a well defined basic population and one has to assume that some variables follow a certain distribution - in most cases the normal distribution is assumed.Power of a test is the probability of correctly rejecting a false null hypothesis. This probability is one minus the probability of making a Type II error (b). Recall also that we choose the probability of making a Type I error when we set a and that if we decrease the probability of making a Type I error we increase the probability of making a Type II error.
Power and Alpha
Thus, the probability of correctly retaining a true null has the same relationship to Type I errors as the probability of correctly rejecting an untrue null does to Type II error. Yet, as I mentioned if we decrease the odds of making one type of error we increase the odds of making the other type of error. What is the relationship between Type I and Type II errors?Power and the True Difference Between Population Means: Anytime we test whether a sample differs from a population or whether two sample come from 2 separate populations, there is the assumption that each of the populations we are comparing has it's own mean and standard deviation (even if we do not know it). The distance between the two population means will affect the power of our test.
Power as a Function of Sample Size and Variance: You should notice that what really made the difference in the size of b is how much overlap there is in the two distributions. When the means are close together the two distributions overlap a great deal compared to when the means are farther apart. Thus, anything that effects the extent the two distributions share common values will increase b (the likelihood of making a Type II error).
Sample size has an indirect effect on power because it affects the measure
of variance we use to calculate the t-test statistic. Since we are calculating
the power of a test that involves the comparison of sample means, we will be
more interested in the standard error (the average difference in sample
values) than standard deviation or variance by itself. Thus, sample size is of
interest because it modifies our estimate of the standard deviation. When n is
large we will have a lower standard error than when n is small. In turn, when
N is large well have a smaller b region than when n
is small.
ANOVA: Analysis of Variance
The tests we have learned up to this point allow us to test hypotheses that examine the difference between only two means. Analysis of Variance or ANOVA will allow us to test the difference between 2 or more means. ANOVA does this by examining the ratio of variability between two conditions and variability within each condition. For example, say we give a drug that we believe will improve memory to a group of people and give a placebo to another group of people. We might measure memory performance by the number of words recalled from a list we ask everyone to memorize. A t-test would compare the likelihood of observing the difference in the mean number of words recalled for each group. An ANOVA test, on the other hand, would compare the variability that we observe between the two conditions to the variability observed within each condition. Recall that we measure variability as the sum of the difference of each score from the mean. When we actually calculate an ANOVA we will use a short-cut formula.Thus, when the variability that we predict (between the two groups) is much greater than the variability we don't predict (within each group) then we will conclude that our treatments produce different results.
Levene's Test: Suppose that the sample data does not support the
homogeneity of variance assumption, however, there is a good reason that the
variations in the population are almost the same, then is such a sitiuation
you may like to use the Levene's modified test: In each group first compute
the absolute deviation of the individual values from the median in that group.
Apply the usual one way ANOVA on the set of deviation values and then
interpret the results.
Distance Sampling
The term 'distance sampling' covers a range of methods for assessing wildlife abundance:line transect sampling, in which the distances sampled are distances of detected objects (usually animals) from the line along which the observer travels
point transect sampling, in which the distances sampled are distances of detected objects (usually birds) from the point at which the observer stands
cue counting, in which the distances sampled are distances from a moving observer to each detected cue given by the objects of interest (usually whales)
trapping webs, in which the distances sampled are from the web center to trapped objects (usually invertebrates or small terrestrial vertebrates)
migration counts, in which the 'distances' sampled are actually times of detection during the migration of objects (usually whales) past a watch point
Many mark-recapture models have been developed over the past 40 years. Monitoring of biological populations is receiving increasing emphasis in many countries. Data from marked populations can be used for the estimation of survival probabilities, how these vary by age, sex and time, and how they correlate with external variables. Estimation of immigration and emigration rates, population size and the proportion of age classes that enter the breeding population are often important and difficult to estimate with precision for free-ranging populations. Estimation of the finite rate of population change and fitness are still more difficult to address in a rigorous manner.
For more details read:
Buckland S., D. Anderson, K. Burnham, and J.
Laake, Distance Sampling: Estimating Abundance of Biological
Populations, Chapman and Hall, London, 1993.
Data Mining and Knowledge Discovery
The continuing rapid growth of on-line data and the widespread use of databases necessitate the development of techniques for extracting useful knowledge and for facilitating database access. The challenge of extracting knowledge from data is of common interest to several fields, including statistics, databases, pattern recognition, machine learning, data visualization, optimization, and high-performance computing.Data mining is the process of extracting knowledge from data. The combination of fast computers, cheap storage, and better communication makes it easier by the day to tease useful information out of everything from supermarket buying patterns to credit histories. For clever marketeers, that knowledge can be worth as much as the stuff real miners dig from the ground.
Data Mining as an analytic process designed to explore large amounts of (typically business or market related) data in search for consistent patterns and/or systematic relationships between variables, and then to validate the findings by applying the detected patterns to new subsets of data. The process thus consists of three basic stages: exploration, model building or pattern definition, and validation/verification.
What distinguishes data mining from conventional statistical data analysis is that data mining is usually done for the purpose of "secondary analysis" aimed at finding unsuspected relationships unrelated to the purposes for which the data were originally collected.
Data warehousing as a process of organizing the storage of large, multivariate data sets in a way that facilitates the retrieval of information for analytic purposes.
Data mining is now a rather vague term, but the element that is common to most definitions is "predictive modeling with large data sets as used by big companies". Therefore, data mining is the extraction of hidden predictive information from large databases. It is a powerful new technology with great potential, for example,to help marketing managers "preemptively define the information market of tomorrow." Data mining tools predict future trends and behaviors, allowing businesses to make proactive, knowledge-driven decisions. The automated, prospective analyses offered by data mining move beyond the analyses of past events provided by retrospective tools. Data mining answers business questions that traditionally were too time-consuming to resolve. Data mining tools scour databases for hidden patterns, finding predictive information that experts may miss because it lies outside their expectations.
Data mining techniques can be implemented rapidly on existing software and hardware platforms across the large companies to enhance the value of existing resources, and can be integrated with new products and systems as they are brought on-line. When implemented on high performance client-server or parallel processing computers, data mining tools can analyze massive databases while a customer or analyst takes a coffee break, then deliver answers to questions such as, "Which clients are most likely to respond to my next promotional mailing, and why?"
Knowledge discovery in databases aims at tearing down the last barrier in enterprises' information flow, the data analysis step. It is a label for an activity performed in a wide variety of application domains within the science and business communities, as well as for pleasure. The activity uses a large and heterogeneous data-set as a basis for synthesizing new and relevant knowledge. The knowledge is new because hidden relationships within the data are explicated, and/or data is combined with prior knowledge to elucidate a given problem. The term relevant is used to emphasize that knowledge discovery is a goal-driven process in which knowledge is constructed to facilitate the solution to a problem.
Knowledge discovery maybe viewed as a process containing many tasks. Some
of these tasks are well understood, while others depend on human judgment in
an implicit matter. Further, the process is characterized by heavy iterations
between the tasks. This is very similar to many creative engineering process,
e.g., the development of dynamic models. In this reference mechanistic, or
first principles based, models are emphasized, and the tasks involved in model
development are defined by:
1. Initial data collection and problem
formulation. The initial data are collected, and some more or less precise
formulation of the modeling problem is developed.
2. Tools selection. The
software tools to support modeling and allow simulation are selected.
3.
Conceptual modeling. The system to be modeled, e.g., a chemical reactor, a
power generator, or a marine vessel, is abstracted at first. The essential
compartments and the dominant phenomena occurring are identified and
documented for later reuse.
4. Model representation. A representation of
the system model is generated. Often, equations are used; however, a graphical
block diagram (or any other formalism) may alternatively be used, depending on
the modeling tools selected above.
5. Implementation. The model
representation is implemented using the means provided by the modeling system
of the software employed. These may range from general programming languages
to equation-based modeling languages or graphical block-oriented interfaces.
6. Verification. The model implementation is verified to really capture
the intent of the modeler. No simulations for the actual problem to be solved
are carried out for this purpose.
7. Initialization. Reasonable initial
values are provided or computed, the numerical solution process is debugged.
8. Validation. The results of the simulation are validated against some
reference, ideally against experimental data.
9. Documentation. The
modeling process, the model, and the simulation results during validation and
application of the model are documented.
10. Model application. The model
is used in some model-based process engineering problem solving task.
For other model types, like neural network models where data-driven knowledge is utilized, the modeling process will be somewhat different. Some of the tasks, like the conceptual modeling phase, will vanish.
Typical application areas for dynamic models are control, prediction, planning, and fault detection and diagnosis. A major deficiency of today's methods is the lack of ability to utilize a wide variety of knowledge. As an example, a black-box model structure has very limited abilities to utilize first principles knowledge on a problem. this has provided a basis for developing different hybrid schemes. Two hybrid schemes will highlight the discussion. First, it will be shown how a mechanistic model can be combined with a black-box model to represent a pH neutralization system efficiently. Second, the combination of continuous and discrete control inputs is considered, utilizing a two-tank example as case. Different approaches to handle this heterogeneous case are considered.
The hybrid approach may be viewed as a means to integrate different types of knowledge, i.e., being able to utilize a heterogeneous knowledge base to derive a model. Standard practice today is that methods and software can treat large homogeneous data-sets. A typical example of a homogeneous data-set is time-series data from some system, e.g., temperature, pressure, and compositions measurements over some time frame provided by the instrumentation and control system of a chemical reactor. If textual information of a qualitative nature is provided by plant personnel, the data becomes heterogeneous.
The above discussion will form the basis for analyzing the interaction between knowledge discovery, and modeling and identification of dynamic models. In particular, we will be interested in identifying how concepts from knowledge discovery can enrich state-of-the- art within control, prediction, planning, and fault detection and diagnosis of dynamic systems.
References and Further Readings:
Brodley C., T. Lane, and T.
Stough, Knowledge Discovery and Data Mining, American Scientist,
Jan.-Feb. 1999.
Chatfield Ch., Model Uncertainty, Data Mining and
Statistical Inference, Journal of Royal Statistical Soc. Ser. A.,
419-466, 1995.
Glymour C., D. Madigan, et. al., Statistical themes
and lessons for data mining, Data Mining and Knowledge Discovery, 1,
11-28, 1997.
Hand D. , Data Mining: Statistics and More?, The American
Statistician, 52( 2), 1998.
Heckerman D., Bayesian networks for data
mining," Data Mining and Knowledge Discovery, 1, 79-119, 1997.
Visit also the following Web sites: Data Mining, and SAS.
Bayes and Empirical Bayes Methods
Bayes and empirical Bayes (EB) methods structure combining information from similar components of information and produce efficient inferences for both individual components and shared model characteristics. Many complex applied investigations are ideal settings for this type of synthesis. For example, county-specific disease incidence rates can be unstable due to small populations or low rates. 'Borrowing information' from adjacent counties by partial pooling produces better estimates for each county, and Bayes/empirical Bayes methods structure the approach. Importantly, recent advances in computing and the consequent ability to evaluate complex models, have increase the popularity and applicability of Bayesian methods.Bayes and EB methods can be implemented using modern Markov chain Monte Carlo(MCMC) computational methods. Properly structured Bayes and EB procedures typically have good frequentist and Bayesian performance, both in theory and in practice. This in turn motivates their use in advanced high-dimensional model settings (e.g., longitudinal data or spatio-temporal mapping models), where a Bayesian model implemented via MCMC often provides the only feasible approach that incorporates all relevant model features.
References and Further Readings:
Bayes and Empirical Bayes
Methods for Data Analysis, by Carlin B., and T. Louis, Chapman and Hall,
1996.
Likelihood Methods
Direct Inverse
__________________________________________
Neyman-Pearson Bayesian (decision analysis
Decision Wald (H. Rubin, e.g.)
---------------------------------------------------
Hybrid "Standard" practice Bayesian (subjective)
-------------------------------------------------------
fiducial (Fisher)
Inference Early Fisher Likelihood (Edwards)
Bayesian (modern)
belief functions
(Shafer)
_________________________________________
In the Direct schools, one uses Pr(data | hypothesis), usually from some
model-based sampling distribution, but one does not attempt to give the
inverse probability, Pr(hypothesis | data), nor any other quantitative
evaluation of hypotheses. The Inverse schools do associate numerical values
with hypotheses, either probabilities (Bayesian schools) or something else
(Fisher, Edwards, Shafer).
The decision-oriented methods treat statistics as a matter of action, rather than inference, and attempt to take utilities as well as probabilities into account in selecting actions; the inference-oriented methods treat inference as a goal apart from any action to be taken.
The "hybrid" row could be more properly labeled as "hypocritical"-- these methods talk some Decision talk but walk the Inference walk.
Fisher's fiducial method is included because it is so famous, but the modern consensus is that it lacks justification.
Now it is true, under certain assumptions, some distinct schools advocate highly similar calculations, and just talk about them or justify them differently. Some seem to think this is tiresome or impractical. One may disagree, for three reasons:
First, how one justifies calculations goes to the heart of what the calculations actually MEAN; second, it is easier to teach things that actually make sense (which is one reason that standard practice is hard to teach); and third, methods that do coincide or nearly so for some problems may diverge sharply for others.
The difficulty with the subjective Bayesian approach is that prior knowledge is represented by a probability distribution, and this is more of a commitment than warranted under conditions of partial ignorance. (Uniform or improper priors are just as bad in some respects as anything other sort of prior.) The methods in the (Inference, Inverse) cell all attempt to escape this difficulty by presenting alternative representations of partial ignorance.
Edwards, in particular, uses logarithm of normalized likelihood as a measure of support for a hypothesis. Prior information can be included in the form of a prior support (log likelihood) function; a flat support represents complete prior ignorance.
One place where likelihood methods would deviate sharply from "standard" practice is in a comparison between a sharp and a diffuse hypothesis. Consider H0: X ~ N(0, 100) [diffuse] and H1: X ~ N(1, 1) [standard deviation 10 times smaller]. In standard methods, observing X = 2 would be undiagnostic, since it is not in a sensible tail rejection interval (or region) for either hypothesis. But while X = 2 is not inconsistent with H0, it is much better explained by H1--the likelihood ratio is about 6.2 in favor of H1. In Edwards' methods, H1 would have higher support than H0, by the amount log(6.2) = 1.8. (If these were the only two hypotheses, the Neyman-Pearson lemma would also lead one to a test based on likelihood ratio, but Edwards' methods are more broadly applicable.)
I do not want to appear to advocate likelihood methods. I could give a long discussion of their limitations and of alternatives that share some of their advantages but avoid their limitations. But it is definitely a mistake to dismiss such methods lightly. They are practical (currently widely used in genetics) and are based on a careful and profound analysis of inference.
What is a Meta-Analysis?
Meta-Analysis deals with the art of combining information from the data from different independent sources which are targeted at a common goal. There are plenty of applications of Meta-Analysis in various disciplines such as Astronomy, Agriculture, Biological and Social Sciences, and Environmental Science. This particular topic of statistics has evolved considerably over the last twenty years with applied as well as theoretical developments.A Meta-analysis deals with a set of RESULTs to give an overall RESULT that is (presumably) comprehensive and valid.
a) Especially when Effect-sizes are rather small, the hope is that one can gain good power by essentially pretending to have the larger N as a valid, combined sample.
b) When effect sizes are rather large, then the extra POWER is not needed for main effects of design: Instead, it theoretically could be possible to look at contrasts between the slight variations in the studies themselves.
If you really trust that "all things being equal" will hold up. The typical "meta" study does not do the tests for homogeneity that should be required
In other words:
1. there is a body of research/data literature that you would like to summarize
2. one gathers together all the admissible examples of this literature (note: some might be discarded for various reasons)
3. certain details of each investigation are deciphered ... most important would be the effect that has or has not been found. ie, how much larger in sd units is the treatment group's performance compared to one or more controls.
4. call the values in each of the investigations in #3 .. mini effect sizes.
5. across all admissible data sets, you attempt to summarize the overall effect size by forming a set of individual effects ... and using an overall sd as the divisor .. thus yielding essentially an average effect size.
6. in the meta analysis literature ... sometimes these effect sizes are further labeled as small, medium, or large ....
You can look at effect sizes in many different ways .. across different factors and variables. but, in a nutshell, this is what is done.
I recall a case in physics, in which, after a phenomenon had been observed in air, emulsion data was examined. The theory would have about a 9% effect in emulsion, and behold, the published data gave 15%. As it happens, there was no significant (practical, not statistical) in the theory, and also no error in the data. It was just that the results of experiments in which nothing statistically significant was found were not reported.
This non-reporting of such experiments, and often of the specific results which were not statistically significant, which introduces major biases. This is also combined with the totally erroneous attitude of researchers that statistically significant results are the important ones, and than if there is no significance, the effect was not important. We really need to between the term "statistically significant", and the usual word significant.
It is very important to distinction between statistically significant and generally significant, see Discover Magazine (July, 1987), The Case of Falling Nightwatchmen, by Sapolsky. In this article, Sapolsky uses the example to point out the very important distinction between statistically significant and generally significant: A diminution of velocity at impact may be statistically significant, but not of importance to the falling nightwatchman.
Be careful about the word "significant". It has a technical meaning, not a commonsense one. It is NOT automatically synonymous with "important". A person or group can be statistically significantly taller than the average for the population, but still not be a candidate for your basketball team. Whether the difference is substantively (not merely statistically) significant is dependent on the problem which is being studied.
Meta-analysis is a controversial type of literature review in which the results of individual randomized controlled studies are pooled together to try to get an estimate of the effect of the intervention being studied. It increases statistical power and is used to resolve the problem of reports which disagree with each other. It's not easy to do well and there are many inherent problems.
There is also graphical technique to assess robustness of meta-analysis results. We should carry out the meta-analysis dropping consecutively one study, that is if we have N studies we should do N meta-analysis using N-1 studies in each one. After that we plot these N estimates on the y axis and compare them with a straight line that represent the overall estimate using all the studies.
Topics in Meta-analysis includes: Odds ratios; Relative risk; Risk difference; Effect size; Incidence rate difference and ratio; Plots and exact confidence intervals.
For details, read,
Meta-Analysis in Social Research, by Glass,
McGraw and Smith, 1987, and
Handbook of Research Synthesis, by
Cooper H., and L. Hedges, (Eds.), New York, Russell Sage Foundation, 1994,
Meta -Analysis: Methods of Accumulating Results Across Research Domains.
Prediction Interval
The idea is that if
is the mean of a random
sample of size n from a normal population, and Y is a single additional
observation, then the test statistic - Y is normal with mean 0 and
variance (1 + 1/n)s2.
Since we don't actually know s2, we need to use t in evaluating the test statistic. The appropriate Prediction Interval for Y is
± ta/2.S.(1+1/n)1/2.
This is similar to construction of interval for individual prediction in regression analysis.
Fitting Data to a Broken Line
Fitting data to a broken, how to determine the parameters, a, b, c, and d such thaty = a + b x, for x less than or equal c
y = a - d c + (d + b) x, for x
greater than or equal to c
A simple solution is a brute force search across the values of c. Once c is known, estimating a, b, and d is trivial through the use of indicator variables. One may use (x-c) as your independent variable, rather than x, for computational convenience.
Now, just fix c at a fine grid of x values in the range of your data, estimate a, b, and d, and then note what the mean squared error is. Select the value of c that minimizes the mean squared error.
Unfortunately, you won't be able to get confidence intervals involving c, and the confidence intervals for the remaining parameters will be conditional on the value of c.
For more details, see Applied Regression Analysis, by Draper and Smith, Wiley 1981, Chapter 5, section 5.4 on use of dummy variables. example 6.
How to Determine if Two Regression Lines Are Parallel?
Would like to determine if two regression lines are parallel? Construct the following multiple linear regression model:E(y) = b0 + b1X1 + b2X2 + b3X3That is, E(y|group=1) is a simple regression with a potentially different slope and intercept compared to group=0.where X1 = interval predictor variable, X2 = 1 if group 1, 0 if group 0, and X3 = X1.X2 Then, E(y|group=0) = b0 + b1X1 and E(y|group=1) = b0 + b1X1 + b2.1 + b3.X1.1 = b0 + b1.X1 + b2 + b3X1 = (b0 + b2) + (b1 + b3)X1
Ho: slope(group 1) = slope(group 0) is equivalent to Ho: b3=0
Use t-test from variables-in-the equation table to test this hypothesis.
Constrained Regression Model
If you fit a regression forcing the intercept to be zero, the standard error of the slope is less. That seems counter-intuitive. The intercept should be included in the model because it is significant, so why is the standard error for the slope in the worse-fitting model actually smaller?I agree that it's initially counter-intuitive (see below), but here are two reasons why it's true. The variance of the slope estimate for the constrained model is s2 / SXi 2), where Xi are actual X values and s2 is estimated from the residuals. The variance of the slope estimate for the unconstrained model (with intercept) is s2 / Sxi 2), where xi are deviations from the mean, and s2 is still estimated from the residuals). So, the constrained model can have a larger s2 (mean square error/"residual" and standard error of estimate) but a smaller standard error of the slope because the denominator is larger.
r2 also behaves very strangely in the constrained model; by the conventional formula, it can be negative; by the formula used by most computer packages, it is generally larger than the unconstrained r2 because it is dealing with deviations from 0, not deviations from the mean. This is because, in effect, constraining the intercept to 0 forces us to act as if the mean of X and the mean of Y both were 0.
Once you recognize that the s.e. of the slope isn't really a measure of overall fit, the result starts to make a lot of sense. Assume that all your X and Y are positive. If you're forced to fit the regression line through the origin (or any other point) there will be less "wiggle" in how you can fit the line to the data than there would be if both "ends" could move.
Consider a bunch of points that are ALL way out, far from zero, then if you Force the regression through zero, that line will be very close to all the points, and pass through origin, with LITTLE ERROR. And little precision, and little validity. Therefore, no-intercept model is hardly ever appropriate.
Semiparametric and Non-parametric modeling
Many parametric regression models in applied science have a form like response = function(X1,..., Xp, unknown influences). The "response" may be a decision (to buy a certain product), which depends on p measurable variables and an unknown reminder term. In statistics, the model is usually written asY = m( X1, ..., Xp) + e
and the unknown e is interpreted as error term.
The most simple model for this problem is the linear regression model, an often used generalization is the Generalized Linear Model (GLM)
Y= G(X1b1 + ... + Xpbp) + e
where G is called the link function. All these models lead to the problem of estimating a multivariate regression. Parametric regression estimation has the disadvantage, that by the parametric "form" certain properties of the resulting estimate are already implied.
Nonparametric techniques allow diagnostics of the data without this restriction. However, this requires large sample sizes and causes problems in graphical visualization. Semiparametric methods are a compromise between both: they support a nonparametric modeling of certain features and profit from the simplicity of parametric methods.
References and Further Readings:
Härdle W., S. Klinke, and B.
Turlach, XploRe: An Interactive Statistical Computing Environment,
Springer, New York, 1995.
Moderation and Mediation
"Moderation" is an interactional concept. That is, a moderator variable "modifies" the relationships between two other variables. While "Mediation" is a "causal modeling" concept. The "effect" of one variable on another is "mediated" through another variable. That is, there is no "direct effect", but rather an "indirect effect."Discriminant and Classification
Classification or discrimination involves learning a rule whereby a new observation can be classified into a pre-defined class. Current approaches can be grouped into three historical strands: statistical, machine learning and neural network. The classical statistical methods make distributional assumptions. There are many others which are distribution free, and which require some regularization so that the rule performs well on unseen data. Recent interest has focused on the ability of classification methods to be generalized.We often need to classify individuals into two or more populations based on a set of observed "discriminating" variables. Methods of classification are used when discriminating variables are:
- quantitative and approximately normally distributed;
- quantitative but possibly nonnormal;
- categorical; or
- a combination of quantitative and categorical.
It is important to know when and how to apply linear and quadratic discriminant analysis, nearest neighbor discriminant analysis, logistic regression, categorical modeling, classification and regression trees, and cluster analysis to solve the classification problem. SAS has all the routines you need to for proper use of these classifications. Relevant topics are: Matrix operations, Fisher's Discriminant Analysis, Nearest Neighbor Discriminant Analysis, Logistic Regression and Categorical Modeling for classification, and Cluster Analysis.
For example, two related methods which are distribution free are the k-nearest neighbor classifier and the kernel density estimation approach. In both methods, there are several problems of importance: the choice of smoothing parameter(s) or k, and choice of appropriate metrics or selection of variables. These problems can be addressed by cross-validation methods, but this is computationally slow. An analysis of the relationship with a neural net approach (LVQ) should yield faster methods.
References and Further Readings:
Cherkassky V, and F. Mulier,
Learning from Data: Concepts, Theory, and Methods, John Wiley & Sons,
1998.
Visit also the Web site Tree-Structured & Rules Induction Programs Homepage
Generalized Linear and Logistic Models
The generalized linear model (GLM) is possibly the most important development in practical statistical methodology in the last twenty years. Generalized linear models provide a versatile modeling framework in which a function of the mean response is "linked" to the covariates through a linear predictor and in which variability is described by a distribution in the exponential dispersion family. These models include logistic regression and log-linear models for binomial and Poisson counts together with normal, gamma and inverse Gaussian models for continuous responses. Standard techniques for analyzing censored survival data, such as the Cox regression, can also be handled within the GLM framework. Relevant topics are: Normal theory linear models, Inference and diagnostics for GLMs, Binomial regression, Poisson regression, Methods for handling overdispersion, Generalized estimating equations (GEEs).Hre is how to obtain degree of freedom number for the 2 log-likelihood, in a logistic regression. Degrees of freedom pertain to the dimension of the vector of parameters for a given model. Suppose we know that a model ln(p/(1-p))=Bo + B1x + B2y + B3w fits a set of data. In this case the vector B=(Bo,B1, B2, B3) is an element of 4 dimensional Euclidean space, or R4.
Suppose we want to test the hypothesis: Ho: B3=0. We are imposing a restriction on our parameter space. The vector of parameters must be of the form: B'=B=(Bo,B1, B2, 0). This vector is an element of a subspace of R4. Namely, B4=0 or the X-axis. The likelihood ration statistic has the form:
2 log-likelihood = 2 log(maximum unrestricted likelihood / maximum
restricted likelihood) =
2 log(maximum unrestricted likelihood)-2 log
(maximum restricted likelihood)
Which is unrestricted B vector 4-dimensions or degrees of freedom - restricted B vector 3 dimensions or degrees of freedom = 1 degree of freedom which is the difference vector: B''=B-B'=(0,0,0,B4) [one dimensional subspace of R4.
The standard textbook is Generalized Linear Models by McCullagh and Nelder (Chapman & Hall, 1989).
LOGISTIC REGRESSION VAR=x
/METHOD=ENTER y x1 x2 f1ros f1ach f1grade bylocus byses
/CONTRAST (y)=Indicator
/contrast (x1)=indicator
/contrast (x2)=indicator
/CLASSPLOT /CASEWISE OUTLIER(2)
/PRINT=GOODFIT
/CRITERIA PIN(.05) POUT(.10) ITERATE(20) CUT(.5) .
Other SPSS Commands: Loglinear LOGLINEAR,HILOGLINEAR Logistic Regression LOGLINEAR,PROBITSAS Commands:
Loglinear CATMOD Logistic Regression LOGISTIC, CATMOD,PROBIT
Survival Analysis
Survival analysis is suited to the examination of data where the outcome of interest is 'time until a specific event occurs', and where not all individuals have been followed up until the event occurs.The methods of survival analysis are applicable not only in studies of patient survival, but also studies examining adverse events in clinical trials, time to discontinuation of treatment, duration in community care before re-hospitalisation, contraceptive and fertility studies etc.
If you've ever used regression analysis on longitudinal event data, you've probably come up against two intractable problems:
Censoring: Nearly every sample contains some cases that do not experience an event. If the dependent variable is the time of the event, what do you do with these "censored" cases?
Time-dependent covariates: Many explanatory variables (like income or blood pressure)change in value over time. How do you put such variables in a regression analysis?
Makeshift solutions to these questions can lead to severe biases. Survival methods are explicitly designed to deal with censoring and time-dependent covariates in a statistically correct way. Originally developed by biostatisticians, these methods have become popular in sociology, demography, psychology, economics, political science, and marketing.
In Short, survival Analysis is a group of statistical methods for analysis and interpretation of survival data. Even though survival analysis can be used in a wide variety of applications (e.g. insurance, engineering, and sociology), the main application is for analyzing clinical trials data. Survival and hazard functions, the methods of estimating parameters and testing hypotheses that are the main part of analyses of survival data. Main topics relevant to survival data analysis are: Survival and hazard functions, Types of censoring, Estimation of survival and hazard functions: the Kaplan-Meier and life table estimators, Simple life tables, Peto's Logrank with trend test and hazard ratios and Wilcoxon test, (can be stratified), Wei-Lachin, Comparison of survival functions: The logrank and Mantel-Haenszel tests, The proportional hazards model: time independent and time dependent covariates, The logistic regression model, and Methods for determining sample sizes.
In the last few years the survival analysis software available in several of the standard statistical packages has experienced a major increment in functionality, and is no longer limited to the triad of Kaplan-Meier curves, logrank tests, and simple Cox models.
References and Further Readings:
Kleinbaum D., et al.,
Survival Analysis: A Self-Learning Text, Springer-Verlag, New York,
1996.
Lee E., Statistical Methods for Survival Data Analysis,
Wiley, 1992.
Association Among Nominal Variables
There are many measures of association between two dichotomous variables, such as the odds ratio (AD/BC), Yule's Q = (AD-BC/AD+BC) which is a simple mapping of the odds ratio onto [-1,1], the proportional difference (requires treating one of the variables as "independent" and the other "dependent"), Cramer's V, the contingency coefficient C, the uncertainly coefficient, and the relative risk. Some of those measures may be more appropriate than others for a given situation, however, those based on the odds ratio are easier to interpret. Odds ratios can be thought of as the effect of one outcome on another. If condition 1 is true, what effect has it on the odds of condition 2 being true? Almost all of these statistics are described in the Numerical Recipes, by Press et al.Spearman's Correlation, and Kendall's tau Application
How would you compare the values of two variables to determine whether they are ordered the same? For example:Var1 Var2 Obs 1 x x Obs 2 y z Obs 3 z yIs Var1 ordered the same as Var2? Two measures are Spearman's rank order correlation, and Kendall's tau. For more details see, e.g., Fundamental Statistics for the Behavioral Sciences, by David C. Howell, Duxbury Pr., 1995.
Repeated Measures and Longitudinal Data
Repeated measures and longitudinal data require special attention because they involve correlated data that commonly arise when the primary sampling units are measured repeatedly over time or under different conditions. Normal theory models for split-plot experiments and repeated measures ANOVA can be used to introduce the concept of correlated data. PROC GLM and PROC MIXED in the SAS system may be used. Mixed linear models provide a general framework for modeling covariance structures, a critical first step that influences parameter estimation and tests of hypotheses. The primary objectives are to investigate trends over time and how they relate to treatment groups or other covariates. Techniques applicable to non-normal data, such as McNemar's test for binary data, weighted least squares for categorical data, and generalized estimating equations (GEE) are the main topics. The GEE method can be used to accommodate correlation when the means at each time point are modelled using a generalized linear model. Relevant topics are: Balanced split-plot and repeated measures designs, Modeling covariance structures of repeated measures, Repeated measures with unequally spaced times and missing data, Weighted least squares approach to repeated categorical data, Generalized estimating equation (Gee) method for marginal models, Subject-specific versus population averaged interpretation of regression coefficients, and Computer implementation using S-plus and the SAS system. The following describes the McNemar's test for binary data.McNemar Change Test: For the yes/no questions under the two conditions, set up a 2x2 contingency table:
f11 f10 f01 f00McNemar's test of correlated proportions is z = (f01 - f10)/sqrt(f01 + f10).
For those items yielding a score on a scale, the conventional t-test for
correlated samples would be appropriate, or the Wilcoxon signed-ranks test.
What Is a Systematic Review?
Health care decision makers need to access research evidence to make informed decisions on diagnosis, treatment and health care management for both individual patients and populations. Systematic reviews are recognized as one of the most useful and reliable tools to assist this practice of evidence-based health care. These courses aim to train health care professionals and researchers in the science and methods of systematic reviews.There are few important questions in health care which can be informed by
consulting the result of a single empirical study. Systematic reviews attempt
to provide answers to such problems by identifying and appraising all
available studies within the relevant focus and synthesizing their results,
all according to explicit methodologies. The review process places special
emphasis on assessing and maximizing the value of data, both in issues of
reducing bias and minimizing random error. The systematic review method is
most suitably applied to questions of patient treatment and management,
although it has also been applied to answer questions regarding the value of
diagnostic test results, likely prognoses and the cost-effectiveness of health
care.
Incidence and Prevalence Rates
Incidence rate (IR) is the rate at which new events occur in a population. It is defined as: Number of new events in a specified period divided by Number of persons exposed to risk during this periodPrevalence rate (PR) measures the number of cases that are present at a specified period of time. It is defined as: Number of cases present at a specified period of time divides by Number of persons at risk at that specified time.
These two measures are related when considering the the average duration (D). That is, PR = IR . D
Note that, for example, county-specific disease incidence rates can be unstable due to small populations or low rates. In epidemiology one can say that IR reflects probability to Become thick at given age, while the PR reflects probability to Be thick at given age.
Other topics in clinical epidemiology include the use of receiver operator curves, and the sensitivity, specificity, predictive value of a test.
Further Readings:
Kleinbaum D., L. Kupper, and K. Muller,
Applied Regression Analysis and Other Multivariable Methods, Wadsworth
Publishing Company, 1988.
Kleinbaum D., et al., Survival
Analysis: A Self-Learning Text, Springer-Verlag, New York, 1996.
Miettinen O., Theoretical Epidemiology, Delmar Publishers, 1986.
Software Selection
You have to be careful when selecting a software. A short list of item for comparison is:1) Ease of learning,
2) Amount of help incorporated for the user,
3) Level of the user,
4) Number of tests and routines involved,
5)
Ease of data entry,
6) Data validation (and if necessary, data locking and
security),
7) Accuracy of the tests and routines,
8) Integrated data
analysis (graphs and progressive reporting on analysis in one screen),
9)
Cost
No one software meets everyone's needs. Determine the needs first and then
ask the questions relevant to the above seven criteria.
Spatial Data Analysis
Data which is
geographically or spatially referenced is encountered in a very wide variety
of practical contexts. In the same way that data collected at different points
in time may require specialised analytical techniques, there are a range of
statistical methods devoted to the modelling and analysis of data collected at
different points in space. Increased public sector and commercial recording
and use of data which is geographically referenced, recent advances in
computer hardware and software capable of manipulating and displaying spatial
relationships in the form of digital maps, and an awareness of the potential
importance of spatial relationships in many areas of research, have all
combined to produced an increased interest in spatial analysis. Spatial Data
Analysis is concerned with the study of such techniques---the kind of problems
they are designed to address, their theoretical justification, when and how to
use them in practice.
Many natural phenomena involve a random distribution of points in space. Biologists who observe the locations of cells of a certain type in an organ, astronomers who plot the positions of the stars, botanists who record the positions of plants of a certain species and geologists detecting the distribution of a rare mineral in rock are all observing spatial point patterns in two or three dimensions. Such phenomena can be modelled by spatial point processes.
The spatial linear model is fundamental to a number of techniques used in image processing, for example, for locating gold/ore deposits, or creating maps. There are many unresolved problems in this area such as the behavior of maximum likelihood estimators and predictors, and diagnostic tools. There are strong connections between kriging predictors for the spatial linear model and spline methods of interpolation and smoothing. The two-dimensional version of splines/kriging can be used to construct deformations of the plane, which are of key importance in shape analysis.
For analysis of spatially auto-correlated data in of logistic regression
for example, one may use of the Moran
Coefficient which is available is some statistical packages such as
Spacestat. This statistic tends to be between -1 and +1, though are not
restricted to this range. Values near +1 indicate similar values tend to
cluster; values near -1 indicate dissimilar values tend to cluster; values
near -1/(n-1) indicate values tend to be randomly scattered.
Box-Cox Power Transformation
In certain cases data distribution is not normal (Gaussian), and we wish to find the best transformation of variable in order to obtain a Gaussian data distribution for further statistical processing.Among others the Box-Cox power transformation is often used for this purpose.
y = (xp - 1)/p, for p not zero y = log x, for p = 0trying different values of p between -3 and +3 is usually sufficient but there are MLE methods for estimating the best p. A good source on this and other transformation methods is
Madansky A., Prescriptions for working Statisticians, Springer-Verlag, 1988.
For percentages or proportions (such as for binomial proportions), Arcsine
transformations would work better. The original idea of Arcsin(p)is to establish variances as
equal for all groups. The arcsin transform is derived analytically to be the
variance-stabilizing and normalizing transformation. The same limit theorem
also leads to the square root transform for Poisson variables (such as counts)
and to the arc hyperbolic tangent (i.e., Fisher's Z) transform for
correlations. The Arcsin Test yields a z and the 2x2 contingency test yields a
chi-sq. But z2 = chi-sq, for large sample
size. A good source is
Rao C., Linear Statistical Inference and Its
Applications, Wiley, 1973.
How to normalize a set of data consisting of negative and positive values, and make them positive between the range 0.0 to 1.0? Define XNew = (X-min)/(max-min).
Multiple Comparison Tests
Multiple Comparison Procedures include topics such as Control of the family-Wise Error rate, The closure Principle, Hierarchical Families of Hypotheses, Single-Step and Stepwise Procedures, and P-value Adjustments. Areas of applications include multiple comparisons among treatment means, multiple endpoints in clinical trials, multiple sub-group comparisons, etc.Nemenyi's multiple comparison test is analogous to Tukey's test, using rank
sums in place of means and using sqrt[n2k(nk+1)/12] as the estimate of standard error (SE),
where n is the size of each sample and k is the number of samples (means).
Similarly to the Tukey test, you compare (rank sum A - rank sum B)/SE to the
studentized range for k. It is also equivalent to the Dunn/Miller test which
uses mean ranks and standard error sqrt[k(nk+1)/12].
Antedependent Modeling for Repeated Measurements
Repeated measures data arise when observations are taken on each experimental unit on a number of occasions, and time is a factor of interest.Many techniques can be used to analyze such data. Antedependence modeling
is a recently developed method which models the correlations between
observations at different times.
Split-half Analysis
What is split-half analysis? Split your sample in half. Factor analyses each half. Do they come out the same (or similar) as each other? Alternatively (or also), take more than two 2 random subsample of your sample and do the same.Notice that this is (like factor analysis itself) an "exploratory", not inferential technique, i.e. hypothesis testing, confidence intervals etc. simply do not apply.
Alternative, randomly split the sample in half and then do an exploratory
factor analysis on Sample 1. Use those results to do a confirmatory factor
analysis with Sample 2.
Sequential Acceptance Sampling
Acceptance sampling is a quality control procedure used when a decision on the acceptability of the batch has to be made from tests done on a sample of items from the batch.Sequential acceptance sampling minimizes the number of items tested when the early results show that the batch clearly meets, or fails to meet, the required standards.
The procedure has the advantage of requiring fewer observations, on
average, than fixed sample size tests for a similar degree of accuracy.
Local Influence
Cook's distance measures the effect of removing a single observation on regression estimates. This can be viewed as giving an observation a weight of either zero or one: local influence allows this weight to be small but non-zero.Cook defined local influence in 1986, and made some suggestions on how to
use or interpret it; various slight variations have been defined since then.
But problems associated with its use have been pointed out by a number of
workers since the very beginning.
Variogram Analysis
Variables are often measured at different locations. The patterns in these spatial variables may be extrapolated by variogram analysis.A variogram summarizes the relationship between the variance of the
difference in pairs of measurements and the distance of the corresponding
points from each other.
Credit Scoring
Credit Scoring is now in widespread use across the retail credit industry. At its simplest, a credit scorecard is a model usually statistical, but in use it is embedded in a computer and or human process.Components of the Interest Rates
The interest rates as quoted in the newspapers and by banks consist of several components. The most important three are:The pure rate: This is the time value of money. A promise of 100 units next year is not worth 100 units this year.
The price-premium factor: If prices go up 5% each year, interest rates go up at least 5%. For example, under the Carter Administration, prices rose about 15% per year for a couple of years, interest was around 25%. Same thing during the Civil War. In a deflationary period, prices may drop so this term can be negative.
The risk factor: A junk bond may pay a larger rate than a treasury note because of the chance of losing the principal. Banks in a poor financial condition must pay higher rates to attract depositors for the same reason. Threat of confiscation by the government leads to high rates in some countries.
Other factors are generally minor. Of course, the customer sees only the
sum of these terms. These components fluctuate at different rates themselves.
This makes it hard to compare interest rates across disparate time periods or
economic condition. The main questions are: how are these components combined
to form the index? A simple sum? A weighted sum? In most cases the index is
form both empirically and assigned on basis of some criterion of importance.
The same applies to other index numbers.
Partial Least Squares
Partial Least Squares (PLS) regression is a multivariate data analysis technique which can be used to relate several response (Y) variables to several explanatory (X) variables.The method aims to identify the underlying factors, or linear combination
of the X variables, which best model the Y dependent variables.
Growth Curve Modeling
Growth is a fundamental property of biological systems, occurring at the level of populations, individual animals and plants, and within organisms. Much research has been devoted to modeling growth processes, and there are many ways of doing this: mechanistic models, time series, stochastic differential equations etc.Sometimes we simply wish to summarize growth observations in terms of a few parameters, perhaps in order to compare individuals or groups. Many growth phenomena in nature show an "S" shaped pattern, with initially slow growth speeding up before slowing down to approach a limit.
These patterns can be modelled using several mathematical functions such as
generalized logistic and Gompertz curves.
Saturated Model & Saturated Log Likelihood
A saturated model is usually one that has no residual df. What is a "saturated" log likelihood? So the "saturated LL" is the LL for a saturated model. It is often used when comparisons made between the log likelihood with an intercept only and the log likelihood for a particular model specification.Pattern recognition and Classification
Pattern recognition and classification are fundamental concepts for understanding living systems and essential for realizing artificial intelligent systems. Applications include 3D modelling, motion analysis, feature extraction, device positioning and calibration, feature recognition, solutions to classification problems to industrial and medical applications.What is Biostatistics?
Biostatistics is a subdiscipline of Statistics which focuses on statistical support for the areas of medicine, environmental science, public health, and related fields. Practitioners span the range from the very applied to the very theoretical. The information which is useful to the biostatistician spans the range from that needed by a general statistician, to more subject-specific scientific details, to ordinary information that will improve communication between the biostatistician and other scientists and researchers.Evidential Statistics
Statistical methods aim to answer a variety of questions about observations. A simple example occurs when a fairly reliable test for a condition C, has given a positive result. Three important types of questions are:1. Should this observation lead me to believe that condition C is present?
2. Does this observation justify my acting as if condition C were present?
3. Is this observation evidence that condition C is present?
We must distinguish among these three questions in terms of the variables and principles that determine their answers. Questions of the third type, concerning the "evidential interpretation" of statistical data, are central to many applications of statistics in many fields.
It is already recognized that for answering the evidential question current statistical methods are seriously flawed which could be corrected by a applying the the Law of Likelihood. This law suggests how the dominant statistical paradigm can be altered so as to generate appropriate methods for objective, quantitative representation of the evidence embodied in a specific set of observations, as well as measurement and control of the probabilities that a study will produce weak or misleading evidence.
References and Further Readings:
Royall R., Statistical
Evidence: A Likelihood Paradigm, Chapman & Hall, 1997.
Spatial Statistics
Many natural phenomena involve a random distribution of points in space. Biologists who observe the locations of cells of a certain type in an organ, astronomers who plot the positions of the stars, botanists who record the positions of plants of a certain species and geologists detecting the distribution of a rare mineral in rock are all observing spatial point patterns in two or three dimensions. Such phenomena can be modelled by spatial point processes.Refrences:
Diggle P., The Statistical Analysis of Spatial Point
Patterns, Academic Press, 1983.
Ripley B., Spatial Statistics,
Wiley, 1981.
What Is a Regression Tree
A regression tree is like a classification tree, only with a continuous target (dependent) variable. Prediction of target value for a particular case is made by assigning that case to a node (based on values for the predictor variables) and then predicting the value of the case as the mean of its node (sometimes adjusted for priors, costs, etc.).Refrence:
Breiman L., J. Friedman, R. Olshen and C. Stone,
Classification and Regression Trees, CRC Press, Inc., Boca Raton,
Florida, 1984.
Cluster Analysis for Correlated Variables
Cluster analysis is used to classify observations with respect to a set of variables. The widely used Ward's method is predisposed to find spherical clusters and may perform badly with very ellipsoidal clusters generated by highly correlated variables (within clusters).To deal with high correlations, some model-based methods are implemented in the S-Plus package. However, a limitation of their approach is the need to assume the clusters have a multivariate normal distribution, as well as the need to decide in advance what the likely covariance structure of the clusters is.
Another option is to combine the principal component analysis with cluster analysis.
References and Further Readings:
Baxter M., Exploratory
Multivariate Analysis in Archaeology, pp. 167-170, Edinburgh University
Press, Edinburgh, 1994.
Manly F., Multivariate Statistical Methods: A
Primer, Chapman and Hall, London, 1986.
Capture-Recapture Methods
Capture-recapture methods were originally developed in the wildlife biology to estimate the population size of some species of wild animals.Visit the following Web sites:
Capture-Recapture Methods,
and Population
Ecology Home Page
Tchebysheff Inequality and Its Improvements
The Tchebysheff's inequality is often used to put bounds on the probability that proportion of random variable X will be within k > 1 standard deviation of the mean mu for any probability distribution. In other words:
The symmetric property of Tchebysheff's inequality is useful, e.g., in constructing control limits is the quality control process. However the limits are very conservative because of lack of knowledge about the underlying distribution. This bounds can be improved (i.e., becomes tighter) if we have some knowledge about the population distribution. For example, if the population is homogeneous, that is its distribution is unimodal, then,
The above inequality is known as the Camp-Meidell inequality.
References and Further Readings:
Grant E., and R. Leavenworth,
Statistical Quality Control, McGraw-Hill, 1996.
Ryan T.,
Statistical Methods for Quality Improvement, John Wiley & Sons,
2000. A very good book for a starter.
Computational Tools and Demos on the Internet
Visit also Annotated Review of Statistical Tools on the InternetIntroduction: Today statistics courses are combined with the full pedagogical power of multimedia. Featuring interactive worked examples, animation, video, narration, and written text. These web sites are designed to provide students with a "self-help" learning resource to complement the traditional textbook.
Following is a small sample of such useful sites.
Analysis of Variance, by B. Lewis.
Statistical Calculators Presided at UCLA. Material here includes: Power Calculator, Statistical Tables, Regression and GLM Calculator, Two Sample Test Calculator, Correlation and Regression Calculator, and CDF/PDF Calculators.
External Links, by SPSS, Free resources for spss, excel, word & more...
Interactive Statistics, by University of Illinois. Examples from over '5' Calculators include: Data, Correlations, Scatter plot, Box Models, and Chisquare Applet,
Interactive Statistical Calculation, by John Pezzullo, Web pages that perform most of statistical calculations.
| Interactive Statistics Page, by John Pezzullo. A complete collection.
Globally accessible Statistical Procedures, by University of South Carolina. Examples of Statistical Calculators include: Kernel Density Estimation, Exact Unconditional Homogeneity/Independence Tests for 2X2 Tables.
Guide to Basic Laboratory Statistics, by B. Lewis. This is an informal guide to elementary inferential statistical methods used in the laboratory. It is not a text on statistics. Instead, the focus is on the proper planning of experiments and the interpretation of results. Some examples include: Spearman's Rank Correlation, Simple Least Squares Data Fitting.
Java Applets Includes demos for: Distributions (Histograms, Normal Approximation to Binomial, Normal Density, The T distribution, Area Under Normal Curves, Z Scores & the Normal Distribution.
Probability & Stochastic Processes (Binomial Probabilities, Brownian Motion, Central Limit Theorem, A Gamma Process, Let's Make a Deal Game.
Statistics (Guide to basic stats labs, ANOVA, Confidence Intervals, Regression, Spearman's rank correlation, T-test, Simple Least-Squares Regression, and Discriminant Analysis.
Demos Contains few interesting demos such as changing the parameters of various distributions, convergence of t-distribution to normal, etc.
Online Statistical Textbooks, by Haiko Lüpsen.
Statistics: The Study of Stability in Variation, by Jan de Leeuw, The Textbook has components which can be used on all levels of statistics teaching. It is disguised as an introductory textbook, perhaps, but many parts are completely unsuitable for introductory teaching. Its contents are Introduction, Analysis of a Single Variable, Analysis of a Pair of Variables, and Analysis of Multi-variables.
Introductory Statistics:Concepts, Models, and Applications, by David Stockburger, It represents over twenty years of experience in teaching the material contained therein by the author. The high price of textbooks and a desire to customize course material for his own needs caused him to write this material. It contains projects, interactive exercises, animated examples of the use of statistical packages, and inclusion of statistical packages.
Selecting Statistics, Cornell University. Answer the questions therein correctly, then Selecting Statistics leads you to an appropriate statistical test for your data.
Statistical training on the web, by Mike Talbot
SURFSTAT Australia, by Keith Dear, Summarizing and Presenting Data, Producing Data, Variation and Probability, Statistical Inference, Control Charts.
Introduction to Quantitative Methods, by Gene Glass, A basic statistics course in the College of Education at Arizona State University.
Sanda Kaufman's Teachnig Resources, contains teaching resources for variety of topics including quantitative methods.
Some experimental pages for teaching statistics, by Juha Puranen, contains some - different methods for visualizing statistical phenomena, such as Power and Box-Cox transformations.
Statistical Home Page by David C. Howell, Containing statistical material covered in the author's textbooks (Statistical Methods for Psychology and Fundamental Statistics for the Behavioral Sciences), but it will be useful to others not using this book. It is always under construction.
Statistics in Biology, A collection of useful links.
Statistics with Confidence, by H. Arsham, Confidence Intervals for mean, variance, and proportions are formulated and discussed.
Statistical Resources on the Web, by David Stockburger.
Elementary Statistics, by J. McDowell, Contents: Frequency distributions, Statistical moments, Standard scores and the standard normal distribution, Correlation and regression, Probability, Sampling Theory, Inference: One Sample, Inference: Two Samples.
Encyclopedia Britannica, Description of some elementary topics in statistics.
VassarStats, by Richard Lowry, On-line elementary statistical computation.
Teaching Activities, by Statistics Canada, Contains interactive exercises focusing on data analysis and survey skills.
Unification of Common Statistical Tables, by H. Arsham, Relationships among statistical tables are constructed (needs Acrobat to view).
Relationship
Among Commonly Occurring Distributions, by H. Arsham, Relationships among
widely used distributions are constructed (needs Acrobat to view).
Some Interesting and Useful
Sites:
Collections of the Reciprocal Sites
Reciprocal sites are collected at the following search engines:
| GoTo| HotBot| InfoSeek| LookSmart| Lycos|
Search and General
Resources
| All In One Search
| AAA Search | ASA: American Statistical Association | The
International Biometric Society | Institute of Mathematical Statistics|
| Lecture Hall | Syllabits | How to Study Statistics |Statistical Terms | Statistics Education|
| All Topics on the Web | All Topics Periodicals | Statistical Resources on the Web| Statistical Resources on the Web|
Teaching Resources | Statistics and OR Resources | Statistical Resources | Statistics Archive|
| World Wide Resources | Virtual Library | Books and Journal | OnLine Text Books | ASA Washington D.C. Chapter|
| Online Statistical Textbooks | Statistical Page Resources | Statistical Resources | Psychometrics | StatServ |Argus Clearinghouse|
| Job Listing | Careers in Statistics | Conferences |
| Statistical Computing Journals | Rainer's Home Page for Statisticians | Statistics Around the world | International Statistical Institute|
Demo's and Interactive
| Coin Flipping |
Equation Solver by Newton's Method | Probability Lessons | Real Analysis
| Java
Applets | More
Applets | XploRe|
|JAVA Demo's and Demo Movies | ANOVA Applet | Outliers and Regression Line | Confidence Interval | Let's Make a Deal|
| Histogram | Central Limit Theorem | Power of T-test | Computer-based Learning Statistics|
| Learning Statistics by Demo | Normal Approx. to binomial | Small Sample Size Effect | Statistics Demonstrations|
|Normal Curve Area |Critical Values for the t-Distribution | Critical Values for the F-Distribution | Critical Values for the Chi- square Distribution|
| Tests for Binomial | Normal Density | Chi-square Goodness-of-fit| Confidence Intervals on a Proportion| Confidence Intervals on a Correlation| Confidence Intervals on a Mean| Confidence Intervals on the Difference of Two Independent Means|
| T-test Paired | Statistical Calculators| Test of Two Proportions| Mean| Two Independent Means| Two Dependent Means| Median Test| Sign Test| McNemar's Test| Wilcoxon Matched-Pairs Test| Wilcoxon Test |Randomization Plans|
Rank Correlation Coefficient| Correlation Coefficient| Comparing Correlation Coefficients|
Survey Analysis
| Social Research
and Statistical Links | Survey
Analysis Software | ASA Survey Research
| American Association for Public
Opinion Research |
Association for Survey Computing|
| Applied Social Surveys| General Social Survey | Bureau of the Census | Social Surveys Question Bank | National Mapping Information |On-line Survey Response|
Statistical Software
| StatiBot | Analyse-it |Autobox |BMDP |Data Desk |Lisp-Stat |Matlab |Minitab |SAS |SPSS |Stata |Multiple Imputation |Statistica |ASSUME
|The Spreadsheet page |Add-ins for Excel. | Data analysis and
statistical solutions for Excel|
| Choosing a Statistical Analysis Package | Statistical Software Review | Statistical Software Providers | Visual Statistics System | WebStat | QDStat | Statistical Calculators on Web | Modstat | The AssiStat | XLStatistics|
Data and Data Analysis
|
Economic Data and Links | Agricultural Statistics
| The Gallup Organization | The Data and Story Library
| Dr. B's Data
| Datasets | Research
Resources|
| Forum on Applied Data Analysis | Multivariate Analysis | Multivariate Statistics | Statistical data analysis FAQ| | Data Analysis | Cast Your Vote!|
| National Center for Health Statistics (NCHS) | FEDSTAT: USA | Economic Data sources | Demography & Population Studies|
|Computer Routines and Data Files | Globally Accessible Statistical Procedures| Data Mining and Knowledge Discovery | USDA Economics and Statistics System|
| Brixton Books:
Epi Info |
Software FAQs| Experimental
Interactive Statistics |
Discrete Genetic Data Analysis | Software
Providers | Software
Metrics
Statistics and
Probability
|Test
for Randomness |Confidence
Intervals |ANOVA
in Detail | The Probability
Web | Business
Statistics | Against All
Odds|
| A New View of Statistics | Statistics Homepage |MathForum | SIIP | Introductory Statistics | Cases | HyperStat | Introduction to Statistics|
| Descriptive Statistics | Statistics Lab | Linear Regression | K-12 Statistics | Statistics Sites|
| Statistics Links | Use and abuse of statistics | Statistics on Web | CHANCE Magazine | Statistical Links | Statistics Handouts|
| Basic Statistics | SurfStat | Design Research | Statistics Mailing Lists | Statistics Links | Introduction to Statistics | Topics in Statistics | Statistics Refresher | World Lecture Hall | Probability and Statistics | Statistics Online | Lecture Notes | Conceptual Statistics|
| SimStat Package | Ordination Methods | Glossary for Ordination Methods | Guide for Statisticians | Power Analysis: How many samples? | Statistics for Everyone | American Risk and Insurance Association | Multilevel Models| | International Association For Statistical Computing | Risk Theory Society | StatLib | Statistical Education through Problem Solving|
| AllStat | Institute of Mathematical Statistics | Statistics and Social Sciences | An Introduction to Geostatistics | Annotated Bibliography of Articles for the Statistics User|
| Bibliography for Computational Probability and Statistics
| Probability Archive | Probability Resources|
Forecasting and Time
Series
|Time Series Analysis
Links |Forecasting Resources on the
Web |Business
Forecasting |Forecasting Sites |
Time Series and Forecasting |Forecasting Principles |Univariate Modeling
| Journal of Business
Forecasting|
|FairModel | Time Series Data Library |M-Competition Data |Financial Econometrics | Economic Time Series|
|Data Source | Institute of Business Forecasting|Econometric Links|
Statistics Publishers
Academic Press | Addison Wesley Longman
(Europe)|
Association of American University Presses| Birkhäuser Verlag| Blackwell's| Cambridge University Press| Chadwyck-Healey| Chapman & Hall | Duxbury Press| | WH Freeman and Company| Harcourt Brace | Maths Index| Institute of Physics Publishing| Kluwer Academic Publishers | McGraw-Hill (UK)| Macmillan (UK)| WW Norton| Oxford University Press| Prentice Hall| SAGE Publications|Springer-Verlag| Wiley | Statistics
Index|
A selection of:
AOL, BUBL Catalogue, Computational Probability
Group, Connected
University, CTI
Statistics, Dogpile, Lecturesonline, Links2Go,
McGraw-Hill,
Math Forum, NetFirst, Netscape, OpenHere, Phone-soft Cyber-world International, Physical
Sciences, Probability and Statistics,
SavvySearch
Guide, Second Moment
Serveurs de
statistique, Smallshop, Suite101.Com, World Lecture Hall,
and Yahoo!
The Copy Right Statements: The fair use, according
the 1996 Fair
Use Guidelines for Educational Multimedia, of materials presented on this
Web site is permitted for noncommercial and classroom purposes.
This site
may be mirrored, intact including these notices, on any server with the public
access, it may be linked to any other Web pages.
Kindly e-mail me your comments, suggestions, and concerns. Thank you.