5.6 Stats Question in R

Remind myself of these conversations of learning statistics through using R these days.

Thanks Tim, your comments are very helpful.

1) The reason I excluded the intercept (i.e., -1) is only for the convenience of extracting the coefficients (i.e. return the intercept, which refers to the Estimate column, respectively for each group). I indeed included the intercept (i.e., no -1) when running the actual analyses. The difference of taking in/out of the intercept is just the table outputs, not affecting any result of the analyses.

2) Dieter - Tim was correct. These two commands in fact did the same thing in R, meaning main effects are given by both commands in addition to the interaction term.
>lm(FA_308_27 ~ gp*smoker + Age - 1, data = dat3.2)
                     Estimate   Std. Error   t value     Pr(>|t|)
gp1mALC           0.610268207 0.0425865499 14.330069 4.398040e-22
gp1wkALC          0.603016628 0.0469235498 12.851045 9.528076e-20
smokers           0.028498249 0.0194763982  1.463220 1.480830e-01
Age              -0.001429693 0.0007651574 -1.868495 6.606585e-02
gp1wkALC:smokers -0.040359689 0.0368267023 -1.095935 2.770308e-01

>lm(FA_308_27 ~ gp + smoker + gp*smoker + Age - 1, data = dat3.2)
                     Estimate   Std. Error   t value     Pr(>|t|)
gp1mALC           0.610268207 0.0425865499 14.330069 4.398040e-22
gp1wkALC          0.603016628 0.0469235498 12.851045 9.528076e-20
smokers           0.028498249 0.0194763982  1.463220 1.480830e-01
Age              -0.001429693 0.0007651574 -1.868495 6.606585e-02
gp1wkALC:smokers -0.040359689 0.0368267023 -1.095935 2.770308e-01

3) I have read in pairwise comparisons in R (http://www.r-bloggers.com/r-tutorial-series-two-way-anova-with-pairwise-comparisons/) and Two-way ANOVA with Interactions and Simple Main Effects (http://rtutorialseries.blogspot.com/2011/02/r-tutorial-series-two-way-anova-with.html). I will change my statistical procedure accordingly.

Sincerely,

Yukai
 
===
Hi Yukai,

1) I'm wondering why you do not include the intercept in the model. In your equation, you have -1. I strongly suggest you include the intercept (i.e., +1). Depending on the statistical procedure you run, it can make a huge difference. I don't remember if matters in the standard lm model, but I always include it for form.

2) It is fine to do gp*smoking. In lm, if you use * e.g., variable1*variable2, the main effects for each variable in the interaction term will automatically be included.

3) The reason you do not see means for the 4 groups (ns1mALC, s1mALC, ns1wkALC, and s1wkALC), is because the model you built only includes the main effects and interactions. To get the adjusted means and standard errors for each individual group, you need to do pairwise comparisons among those groups. Just google pairwise comparisons in R, pairwise t-tests in R or try http://www.r-statistics.com/. There should be some sample code that you can adapt, and it is relatively straight-forward to execute.


Tim

4.27 UW Machine Learning Regression Week 2 Assignment 1

4 Quiz Questions that I did wrong:

1) If you double the value of a given feature (i.e. a specific column of the feature matrix), what happens to the least-squares estimated coefficients for every other feature? (assume you have no other feature that depends on the doubled feature i.e. no interaction terms).

It is impossible to tell from the information provided (wrong)

They stay the same

Considering when interpreting a parameter, we assume that other features are constant (i.e. no matter a particular feature's measurement got doubled or not).

2) Gradient descent/ascent is...

An approximation to simple linear regression (wrong)
A modeling technique in machine learning (wrong)

An algorithm for minimizing/maximizing a function
by definition...

3) Let's analyze how many computations are required to fit a multiple linear regression model using the closed-form solution based on a data set with 50 observations and 10 features. In the videos, we said that computing the inverse of the 10x10 matrix (H^T)H was on the order of D^3 operations. Let's focus on forming this matrix prior to inversion. How many multiplications are required to form the matrix (H^T)H?

1000 (wrong) not reading the question carefully... N x N x D = 50 x 50 x 10 = 25000 (wrong) not did the linear algebra correctly... N x D x D = 50 x 10 x 10 = 5000 (review Linear Algebra) be patient

4) More generally, if you have D features and N observations what is the total complexity of computing ((H^T)H)^(-1)?

O(D^3) (wrong) see first failure in 3)

O(N^2D + D^3) (wrong) see second failure in 3)
O(ND^2 + D^3) that's how I got 3) correct