A Visualization of Gradient Descent

MATH 401: Math Through 3D Printing Fall 2019
Savannah Crawford


Gradient descent is a popular optimization method for finding the  minimum of a function. The algorithm is simple and easy to implement in various languages, but can take a long time to run depending on stopping conditions, initial conditions and local minimums.

The algorithm uses the power of the gradient of a given function. Since the gradient gives the direction of greatest increase at a point, the gradient descent takes a given point a0, and calculates . Then

 where delta is fixed step size. The algorithm then runs until the difference between an and an+1 is sufficiently small.

Python code implementation of gradient descent

In particular this print finds the local minimum of

 with a0 = (2,7) and delta = .01. The local minimum was found to be  at (4.12, 4.55).

Graph of F(x,y) generated in Mathematica

The gradient decent was run in Python and the points were exported as a text file. The data was imported into Mathematica to plot a tube around the list of points generated in Python, with the addition of the z-component as F(xn,yn). The supports for the print were generated in openSCAD. I was unable to export the gradient descent points and the surface together, so they were exported as separate .stl files and combined with supports in MeshLab.

Final 3D model meshed in MeshLab

Overall, I'm very happy with the way this print came out and I learned a few new things about file I/O in Mathematica. It demonstrates very nicely the power of the gradient and gradient descent algorithm.