Loading...

ME205: Numerical Methods for Engineers

Unit 6: Regression and Optimization   Regression and optimization are related ideas. Both have to do with finding the best solution to a quantifiable problem subject to certain constraints. For example, you have probably engaged in the exercise of drawing a “best-fit” line through three or more points by eye. In most cases, the line cannot pass directly through all of the points; rather we must make some judgments about which points are important and what deviations are important. This “best-fit” line then permits us to infer two parameters which describe the line; these could be a slope and intercept. If we have several more than three data points, then we may employ more complex models than just lines.

Unit 6 Time Advisory
This unit will take you approximately 15 hours to complete.

☐    Subunit 6.1: 2 hours

☐    Subunit 6.2: 4 hours

☐    Subunit 6.3: 6 hours
 
        ☐    Web Media: 2.5 hours

        ☐    Reading: 2.5 hours

        ☐    Assessment: 1 hour

☐    Subunit 6.4: 3 hours

Unit6 Learning Outcomes
Upon successful completion of this unit, the student will be able to:

  • Define regression.
  • Perform linear least-squares regression.
  • Perform nonlinear regression.

6.1 Introduction to Regression   - Reading: University of South Florida: Holistic Numerical Methods Institute’s “A Primer on Statistical Terminology for Regression Analysis” and “Introduction to Regression” Link: University of South Florida: Holistic Numerical Methods Institute’s “A Primer on Statistical Terminology for Regression Analysis” (PDF) and “Introduction to Regression” (PDF)
 
Instructions: Please read both chapters in their entirety (15 pages).
 
Terms of Use: The article above is released under a Creative Commons Attribution-Non-Commercial-Share-Alike License 3.0 (HTML).  It is attributed to the University of South Florida and the original version can be found here (HTML).

  • Assessment: University of South Florida: Holistic Numerical Methods Institute’s “Test Your Knowledge on Background of Regression” Link:University of South Florida: Holistic Numerical Methods Institute’s “Test Your Knowledge on Background of Regression” (PDF)
     
    Instructions: Attempt all questions in the multiple choice quiz.  You can find the answers here (PDF).
     
    Terms of Use: The article above is released under a Creative Commons Attribution-Non-Commercial-Share-Alike License 3.0 (HTML).  It is attributed to the University of South Florida and the original version can be found here (HTML).

6.2 Linear Regression   - Web Media: University of South Florida: Holistic Numerical Methods Institute’s Lectures on “Linear Regression” Link: University of South Florida:  Holistic Numerical Methods Institute’s "Lectures on “Linear Regression” (YouTube)
 
Instructions: Under the “Digital Audiovisual Lectures” heading, click on the YouTube link after each title to launch each video.  Please watch all 7 videos in their entirety (approximately 55 minutes).
 
Terms of Use: Please respect the copyright and terms of use displayed on the webpage above.

  • Reading: University of South Florida: Holistic Numerical Methods Institute’s “Textbook Chapter on Linear Regression” Link: University of South Florida: Holistic Numerical Methods Institute’s “Textbook Chapter on Linear Regression” (PDF)
     
    Instructions: Please read the entire chapter (21 pages).  Based on your reading about numerical differentiation, can you think of ways to improve or tailor linear regression?
     
    Terms of Use: The article above is released under a Creative Commons Attribution-Non-Commercial-Share-Alike License 3.0 (HTML).  It is attributed to the University of South Florida and the original version can be found here (HTML).

  • Assessment: University of South Florida: Holistic Numerical Methods Institute’s “Test Your Knowledge of Linear Regression” Link: University of South Florida:  Holistic Numerical Methods Institute’s “Test Your Knowledge of Linear Regression” (PDF)
     
    Instructions: Attempt all questions on the multiple choice quiz.  You can find the answers here (PDF).
     
    Terms of Use: The article above is released under a Creative Commons Attribution-Non-Commercial-Share-Alike License 3.0 (HTML).  It is attributed to the University of South Florida and the original version can be found here (HTML).

  • Reading: University of South Florida: Holistic Numerical Methods Institute’s “Industrial Engineering Example of Linear Regression” Link: University of South Florida: Holistic Numerical Methods Institute’s “Industrial Engineering Example of Linear Regression” (PDF)
     
    Instructions: Follow the example, and try to reproduce the calculations as you read.
     
    Terms of Use: The article above is released under a Creative Commons Attribution-Non-Commercial-Share-Alike License 3.0 (HTML).  It is attributed to the University of South Florida and the original version can be found here (HTML).

6.3 Nonlinear Regression   - Web Media: University of South Florida: Holistic Numerical Methods Institute’s Lectures on “Nonlinear Regression” Link: University of South Florida:  Holistic Numerical Methods Institute’s Lectures on “Nonlinear Regression” (YouTube)
 
Instructions: Under the “Digital Audiovisual Lectures” heading, click on the YouTube link after each title to launch each video.  Please watch all 20 videos in their entirety (about 2.5 hours).
 
Terms of Use: Please respect the copyright and terms of use displayed on the webpage above.

  • Reading: University of South Florida: Holistic Numerical Methods Institute’s “Textbook Chapter on Nonlinear Regression” Link: University of South Florida: Holistic Numerical Methods Institute’s “Textbook Chapter on Nonlinear Regression” (PDF)
     
    Instructions: Please read the entire chapter (32 pages).  Based on your reading about numerical differentiation, can you think of ways to improve Nonlinear Regression?
     
    Terms of Use: The article above is released under a Creative Commons Attribution-Non-Commercial-Share-Alike License 3.0 (HTML).  It is attributed to the University of South Florida and the original version can be found here (HTML).

  • Assessment: University of South Florida: Holistic Numerical Methods Institute’s “Test Your Knowledge of Nonlinear Regression” Link: University of South Florida: Holistic Numerical Methods Institute’s “Test Your Knowledge of Nonlinear Regression” (PDF)
     
    Instructions: Attempt all of the questions on the multiple choice quiz.  You can find the answers here (PDF).
     
    Terms of Use: The article above is released under a Creative Commons Attribution-Non-Commercial-Share-Alike License 3.0 (HTML).  It is attributed to the University of South Florida and the original version can be found here (HTML).

  • Reading: University of South Florida: Holistic Numerical Methods Institute’s “Electrical Engineering Example of Nonlinear Regression” Link: University of South Florida: Holistic Numerical Methods Institute’s “Electrical Engineering Example of Nonlinear Regression” (PDF)
     
    Instructions: Follow the example, and try to reproduce the calculations as you read.
     
    Terms of Use: The article above is released under a Creative Commons Attribution-Non-Commercial-Share-Alike License 3.0 (HTML).  It is attributed to the University of South Florida and the original version can be found here (HTML).

6.4 A Brief Exercise in Regression   - Assessment: The Saylor Foundation’s “ME205: Unit 6.4 Exercise” Link: The Saylor Foundation’s “ME205: Unit 6.4 Exercise” (PDF)
 
Instructions: Please perform this exercise.  When you are done, check your work against The Saylor Foundation’s “ME304: Unit 6.4 Exercise Solution Guide" (PDF).  This exercise should require less than 3 hours to complete.