• Let’s suppose we are statistical consultants hired by a client to investigate the association between advertising and sales of a particular product
• The Advertising data set consists of the sales of that product in 200 different markets, along with advertising budgets for the product in each of those markets for three different media: TV, Radio, and Newspaper

• It is not possible for our client to directly increase sales of the product …
• … but they can control the advertising expenditure in each of the three media

• The data are stored in a .csv file (available on the moodle page)
• The R function to read a .csv file is read.csv()
• In this case, no further arguments are passed to the function (see the help)
• Exercise: Try to read the data using read.csv2() function

• After verifying that the process of reading the data has been performed successfully, we can analyze the data structure (dimension of the data set, name of variables, type of the variables)
• We can do that using the functions dim(), names() and str()

• Having graphical (or numerical) information of the variables under analysis. This can be done

1. by using the summary() function
2. by using graphical tools
   

• In our example all the variables are continuos (we can use hist() for 2.)
• The sales are in thousands of units and the advertising budgets are in thousands of dollars

• Histograms

• We are asked to suggest, on the basis of the data, a marketing plan that will result in high product sales
• What information would be useful in order to provide such a recommendation?

1. Is there a relationship between advertising budget and sales?
2. How strong is the relationship between advertising budget and sales?
3. Which media are associated with sales?
4. How large is the association between each medium and sales?
5. How accurately can we predict future sales?
6. Is the relationship linear?
7. Is there any synergy among the advertising media?

• Our first goal should be to determine whether the data provide evidence of an association between advertising expenditure and sales. If the evidence is weak, one might argue that no money should be spent on advertising
• We could start our exploration by fitting simple linear regressions, each of which uses a different advertising medium as a predictor
  
• Let’s consider a simple linear regression of sales on TV (\(x_2\)): \[Y_i = \beta_1 + \beta_2x_{i2} + \varepsilon_i\] with \(\varepsilon_i\sim \mathcal{N}(0, \sigma^2)\), for \(i=1,\ldots,n\), indepedently

• By regressing sales on TV we get the estimated regression line

\[y = 7.033 + 0.047 x_2\]

1. \(\hat \beta_1= 7.033\): For \(0\$\) spent in TV the average number of sales is 7.033 units
2. \(\hat \beta_2= 0.047\): A \(1000\$\) increasing in spending on TV advertising is associated with an increase of sales of around 47.5 units on average

• Graphical representation of data \((x_2, y)\) - black points, regression lines (\(y = 7.033 + 0.047 x_2\)) in red, and predicted values \((x, \hat y)\) in blue;

• The segments connecting \(y\) and \(\hat y\) are composing the residuals \(e = y - \hat y\)

• Confidence interval: \[IC^{1-\alpha}_{\beta_r}=(\hat \beta_r - t_{n-p; 1- \alpha/2} \sqrt{S^2((X^\top X)^{-1}_{rr})}, \hat \beta_r + t_{n-p; 1- \alpha/2} \sqrt{S^2((X^\top X)^{-1}_{rr})})\]

• For instance for \(\beta_1\) (\(1-\alpha=0.95\)) a realization is \[IC^{0.95}_{\beta_1}=(7.033 - t_{198;0.975} \times 0.458, 7.033 + t_{198;0.975} \times 0.458) \approx (6.13, 7.94)\]

• In absence of any TV advertising the sales, will, on average, fall somewhere between 6.13 and 7.94 units

• For each \(1000\$\) increase in television advertising, there will be an average increase in sales betweem 42 and 53 units

• Hypothesis test to assess if there is a relationship between TV and Sales \[\begin{cases}H_0: \text{There is no relationship between}\,\, x_2\,\, \text{and} \,\,y \\ H_1: \text{There is some relationship between}\,\, x_2\,\, \text{and} \,\,y\end{cases} \implies \begin{cases}H_0: \beta_2 = 0 \\ H_1: \beta_2 \neq 0 \end{cases}\] \[t = \frac{\hat \beta_2}{\sqrt{S^2(X^\top X)^{-1}_{22}}} \stackrel{H_0} \sim t_{n-p}\quad \quad p-value = {\rm Pr}(|t_{n-p}| > |t^{obs}_{r}|) = 2P(t_{n-p}\leq -|t^{obs}_{r}|)\]
• Here \(t^{obs} = 0.04753664/0.002690607 = 17.66763\) and \({\rm p-value} = 2{\rm Pr}(t_{198}\leq -17.66763)\)
• We can infer that there is an association between TV advertising and sales

• Assessing the accuracy of the model via residual standard error \(s = \sqrt{\frac{1}{n-p}\sum_{i=1}^{n}e^2_i}\)
• Actual sales in each market deviate from the true regression line by approximately 3.26 units, on average (whether or not 3.26 units is acceptable or not depends on the problem context)
• Further, it is an absolute measure of the lack of fit (because it is measured in the units of \(y\))
• If the predictions obtained using the model are very close to the true outcome values, that is if \(\hat y_i \approx y_i\) for \(i=1, \ldots, n\), then \(s\) will be small and we can conclude that the model fits the data very well
  

• Assessing the accuracy of the model via \(R^2\) coefficient (proportion of variability explained by the model) \[R^2 = 1 - \frac{\sum_{i=1}^{n}(y_i - \hat y_i)^2}{\sum_{i=1}^{n}(y_i - \bar y)^2}\]

• It does not depend on the scale of \(y\): a value close to 1 means that a large proportion of the variability is explained by the regression, while a value close to 0 means that the regression line does not explain much of the variability of \(y\) (this might occur because the linear model is wrong and/or the error variance is high)

• Here, \(R^2=0.61\) means the less than 2/3 of the variability in sales is explained by regressing sales on TV

• However, also in this case it is still challenging to determine what is a good \(R^2\) value and it depends on the application (in typical applications in biology, psychology, marketing … the linear model is at best an extremely rough approximation to the data, and residual errors due to other unmeasured factors are often very large)

1. Considering the simple linear regression of sales on radio: A \(1000\$\) increasing in spending on radio advertising is associated with an increase of sales of around 203 units
   
2. Considering the simple linear regression of sales on newspaper: A \(1000\$\) increasing in spending on newspaper advertising is associated with an increase of sales of around 55 units

• However, we have not yet answered to the first question
• Indeed, the approach of fitting separate simple linear regression models for each predictor is not entirely satisfactory:

1. It is unclear how to make a single prediction of sales given the three advertising media budgets (each of the budgets is associated with a separate regression equation)
2. Each of three regression equations ignores the other two media in forming the estimates for the regression coefficients (since the media budgets are correlated with each other this can lead to very misleading estimates of the associaton between media budgets and sales)

• In order to answer to the first question we must consider a multiple linear regression model where we regress sales onto TV, Radio, and Newspaper budgets, that is \[Y_i = \beta_1 + \beta_2x_{i2} + \beta_3 x_{i3} + \beta_4x_{i4} + \varepsilon_i\]
• \(\varepsilon_i\sim \mathcal{N}(0, \sigma^2)\), for \(i=1,\ldots,n\), indepedently
• \(\beta_j\) for \(j = 2,3,4\) quantifies the association between the predictor \(x_j\) and the response
• \(\beta_j\) is interpreted as the average effect of \(Y\) on a unit increase in \(x_j\), holding all other predictors fixed

• Do you notice something different with respect to the simple linear regression models fitted above?

• For a given amount of

1. Radio and Newspaper advertising, spending an additional \(1000\$\) on TV advertising is associated with additional 46 units of sales
2. TV and Newspaper advertising, spending an additional \(1000\$\) on Radio advertising is associated with additional 189 units of sales
3. TV and Radio advertising, spending an additional \(1000\$\) on Newspaper advertising is associated with 1 unit less of sales

• While the regression coefficients for TV and Radio are almost the same of the simple linear regression models (0.0475 for TV and 0.2025 for Radio), the one for Newspaper (which was 0.0547 and significant) is now negative and no more significant (close to zero and p-value of 0.86)

• This because in the simple linear regression model, the slope term represents the average increase in sales associated with an additional \(1000\$\) in Newspaper, ignoring TV and Radio, while in the multiple linear regression the slope term represents the average increase in sales associated with an additional \(1000\$\) in Newspaper, holding fixed Radio and TV

• This is due to the correlation between Radio and Newspaper (0.35): markets with high Newspaper advertising tend to have high Radio advertising

• Indeed, in markets where we spend more on radio our sales will tend to be higher, and as the correlation shows, we also tend to spend more on newspaper advertising in those same markets

• In other words, Newspaper is a surrogate for Radio Advertising: newspaper gets credit for the association between radio and sales

• Remember that relationship should not be interpreted as Cause-and-Effect; A statistical relationship — even a strong one — between and does not imply a cause-and-effect relationship
• Consider the example introduced in the first lecture: running a regression of shark attacks versus ice cream sales for data collected at a given beach in a community over a period of time would show a positive relationship
• Of course no one has (yet) suggested that ice creams should be banned at beaches to reduce shark attacks
• In reality, higher temperature causes more people to visit the beach, which in turn results in more ice cream sales and more shark attacks
• A multiple regression of shark attacks onto ice cream sales and temperatures reveals that ice cream sales is no longer significant after adjusting for the temperature

• In addition to the \(R^2\) coefficient and the residual standard error, it is interesting to analyze the model output graphically (sometimes numerical summaries are not enough to detect some aspects)
• Let’s look to the 3d plot including the data and the regression surface from the fitted models: as expected some observations lie below and some above the regression plane; however, the linear model seems to overestimates Sales for instances in which most of the advertising money was spent exclusively in TV or Radio, while it underestimates for instances where the budget was split between the two media: this is probably due to an interaction effect (in red the points where \(y > \hat y\) and in blue the points where \(y < \hat y\))