Regression Analysis | Regression Analysis: Step by Step

Regression Analysis : 

Regression Analysis includes the following steps :

Step 1 : Statement of the problem under consideration :

• The first important step in conducting any regression analysis is to specify problem and the objectives to be addressed by the regression analysis.
• The wrong formulation or the wrong understanding of the problem will give the wrong statistical inferences. The choice of variables depends upon the objectives of study and understanding of the problem.

 

Step 2 : Choice of Relevant Variables :

• Once the problem is carefully formulated and objectives have been decided, the next question is to choose the relevant variables.
• It has to keep in mind that the correct choice of variables will determine the statistical inferences correctly.
• For example, in any agricultural experiment, the yield depends on explanatory variables like quantity of fertilizer, rainfall, irrigation, temperature etc. These variables are denoted by x1, x2,….xk, as a set of k explanatory variables.

 

Step 3 : Collection of Data on Relevant Variables :

• Once the objective of study is clearly stated and the variables are chose, the next question arises is to collect data on such relevant variables. The data is essentially the measurement on these variables.
• For example, suppose we want to collect the data on age, For this, it is important to know how to record it. Then either the data of birth can be recorded which will provide the exact age on any specific date or the age in terms of completed years as on specific date.
• Moreover, it is also important to decide that whether the data has to be collected on variables as quantitative variables or qualitative variables.

 

Step 4 :  Specification of Model :

• The experimenter or the person working in the subject usually helps in determining the form of the model. Only the form of the tentative model can be ascertained and it will depend on some unknown parameters.
• For example, a general form will be like
• Y=f(x1,x2,…,xk; B1,B2,…Bk)+E
• Where E  is the random error reflecting mainly the difference in the observe values of y and the value of y obtained through the model. The form of f(x1,x2,…,xk; B1,B2,…Bk) can be linear as well as nonlinear depending on the form of parameters B1,B2,..Bk. A model is said to be linear if it is linear in parameters.

 

Step 5: Choice of Method for Fitting the Data :

• After the model has been defined and the data have been collected, the next task is to estimate the parameters of the model based on the collected data. This is also referred to as parameter estimation or model fitting.
• The most commonly used method of estimation is the lest squares method Under certain assumptions, the least square method produces estimators with desirable properties. The other estimation methods are the maximum likelihood method , ridge principal components method etc.

 

Step 6 : Fitting of Model :

• The estimation of unknown parameters using appropriate method provides the values of the parameter. Substituting these values in the equation gives us a usable model. This termed as model fitting.
• The fitted equation is used for prediction. In this case, y is termed as predicted value. Note that the fitted value is where the values used for explanatory variables correspond to one of the n observations in the data whereas predicted value is the recommended to predict the y values for the set of those values of explanatory variables which lie outside the range of data. When the values of explanatory variables are the future values of explanatory variables, the predicted values are called forecasted values.

 

Step 7  : model validation and Criticism :

• The Validity of statistical method to be used for regression analysis depends on various assumptions. These assumption are essentially the assumptions for the model and the data.
• The quality of statistical inferences heavily depends on whether these assumptions are satisfied or not. For making these assumptions to be valid and to be satisfied, care in needed from beginning of the experiment.
• One has to be care full in choosing the required assumptions and to examine whether the assumptions are valid for the given experimental conditions or not. It is also important to decide the situations in which the assumptions may not meet.
• The validation of the assumptions must be made before drawing any statistical conclusion. Any departure form validity of assumptions will be reflected in the statistical inferences. In fact, the regression analysis is an iterative process where the outputs are used to diagnose, validate, criticize and modify the inputs.

 

Step 8 : Using the chosen model for the solution of the posed problem and forecasting.

• The determination of explicit for of regression equation is the ultimate objective of regression analysis. It is finally a good and valid relationship between study variable and explanatory variables.
• The regression equation helps in understanding the interrelationships among the variables. Such regression regression equation can be used for several purposes.
• For example, to determine the role of any explanatory variable in the joint relationship in any policy formulation, to forecast the values of response variable for given set of values of explanatory variables.