The Path a Rain Drop Takes on a Surface
(or how to find a local minimum via the Method of Gradient Descent)Klaus Volpert, Fall 2014
(with some meddling by bob jantzen)For certain types of functions it is hopeless to find extreme points by analytic (exact) methods. In this worksheet we illustrate a commonly used approximation method, called the Method of Gradient Descent (resp. Ascent), to numerically find extreme points. Essentially one starts at a good guess of a point, and then follows the negative gradient step-by-step to find a minimum, or the positive gradient to find a maximum.
This path down the hill is exactly the same that an imaginary rain drop would follow. . . LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYlLUkjbWlHRiQ2JVEocmVzdGFydEYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy1JI21vR0YkNi1RIjpGJy9GM1Enbm9ybWFsRicvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRj0vJSlzdHJldGNoeUdGPS8lKnN5bW1ldHJpY0dGPS8lKGxhcmdlb3BHRj0vJS5tb3ZhYmxlbGltaXRzR0Y9LyUnYWNjZW50R0Y9LyUnbHNwYWNlR1EsMC4yNzc3Nzc4ZW1GJy8lJ3JzcGFjZUdGTEY5QyQtSSV3aXRoRzYiNiNJJnBsb3RzRzYkJSpwcm90ZWN0ZWRHSShfc3lzbGliR0YlISIiSuppose we are given a complicated surface given by the functionQyQ+SSJmRzYiZio2JEkieEdGJUkieUdGJUYlNiRJKW9wZXJhdG9yR0YlSSZhcnJvd0dGJUYlLCotSSRzaW5HRiU2IyomOSQiIiI5JUYzRjMtSSRjb3NHRiU2I0YyRjNGMSMhIiIiIiZGNCNGM0Y6RiVGJUYlRjk=QyYtSSdwbG90M2RHNiI2LC1JImZHRiU2JEkieEdGJUkieUdGJS9GKjshIiUiIiUvRitGLS9JKHNoYWRpbmdHRiVJK3pncmF5c2NhbGVHRiUvSSVheGVzR0YlSSZib3hlZEdGJS9JLXRyYW5zcGFyZW5jeUdGJSQiIiIhIiIvSSZzdHlsZUdGJUkvc3VyZmFjZWNvbnRvdXJHRiUvSSljb250b3Vyc0dGJSIjXS9JLG9yaWVudGF0aW9uR0YlNyQiI04iIzovSSVncmlkR0YlNyQiJF0jRkpGOj5JLHN1cmZhY2VwbG90R0YlSSIlR0YlRjs=Suppose we would like to illustrate the path a rain drop would take, if it falls on the point LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYlLUkobWZlbmNlZEdGJDYkLUYjNiYtSSNtbkdGJDYkUSIwRicvJSxtYXRodmFyaWFudEdRJ25vcm1hbEYnLUkjbW9HRiQ2LVEiLEYnRjQvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHUSV0cnVlRicvJSlzdHJldGNoeUdGPS8lKnN5bW1ldHJpY0dGPS8lKGxhcmdlb3BHRj0vJS5tb3ZhYmxlbGltaXRzR0Y9LyUnYWNjZW50R0Y9LyUnbHNwYWNlR1EmMC4wZW1GJy8lJ3JzcGFjZUdRLDAuMzMzMzMzM2VtRidGMEY0RjQtRjg2LVEiOkYnRjRGOy9GP0Y9RkFGQ0ZFRkdGSS9GTFEsMC4yNzc3Nzc4ZW1GJy9GT0ZWRjQ=LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYxLUkjbWlHRiQ2JVEnc3RhcnR4RicvJSdpdGFsaWNHUSV0cnVlRicvJSxtYXRodmFyaWFudEdRJ2l0YWxpY0YnLUkjbW9HRiQ2LVEqJmNvbG9uZXE7RicvRjNRJ25vcm1hbEYnLyUmZmVuY2VHUSZmYWxzZUYnLyUqc2VwYXJhdG9yR0Y9LyUpc3RyZXRjaHlHRj0vJSpzeW1tZXRyaWNHRj0vJShsYXJnZW9wR0Y9LyUubW92YWJsZWxpbWl0c0dGPS8lJ2FjY2VudEdGPS8lJ2xzcGFjZUdRLDAuMjc3Nzc3OGVtRicvJSdyc3BhY2VHRkwtSSNtbkdGJDYkUSIwRidGOS1GNjYtUSI6RidGOUY7Rj5GQEZCRkRGRkZIRkpGTS1JJ21zcGFjZUdGJDYmLyUnaGVpZ2h0R1EmMC4wZXhGJy8lJndpZHRoR1EmMC4wZW1GJy8lJmRlcHRoR0Zlbi8lKmxpbmVicmVha0dRKG5ld2xpbmVGJy1GLDYlUSdzdGFydHlGJ0YvRjJGNUZPRlNGVi1GLDYmUSdzdGFydHpGJ0YvLyUrZXhlY3V0YWJsZUdGPUYyLUY2Ni5GOEZkb0Y5RjtGPkZARkJGREZGRkhGSkZNLUYsNiZRImZGJ0YvRmRvRjItSShtZmVuY2VkR0YkNiUtRiM2JS1GLDYmRi5GL0Zkb0YyLUY2Ni5RIixGJ0Zkb0Y5RjsvRj9GMUZARkJGREZGRkgvRktGaG4vRk5RLDAuMzMzMzMzM2VtRictRiw2JkZgb0YvRmRvRjJGZG9GOS1GNjYuRlVGZG9GOUY7Rj5GQEZCRkRGRkZIRkpGTQ==LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYkLUkjbWlHRiQ2JVEoZGlzcGxheUYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy1JKG1mZW5jZWRHRiQ2JC1GIzYkLUY2NiYtRiM2Jy1GLDYlUSxzdXJmYWNlcGxvdEYnRi9GMi1JI21vR0YkNi1RIixGJy9GM1Enbm9ybWFsRicvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRjEvJSlzdHJldGNoeUdGSS8lKnN5bW1ldHJpY0dGSS8lKGxhcmdlb3BHRkkvJS5tb3ZhYmxlbGltaXRzR0ZJLyUnYWNjZW50R0ZJLyUnbHNwYWNlR1EmMC4wZW1GJy8lJ3JzcGFjZUdRLDAuMzMzMzMzM2VtRictRkI2LVEifkYnRkVGRy9GS0ZJRkxGTkZQRlJGVEZWL0ZaRlgtRiw2JVEscG9pbnRwbG90M2RGJ0YvRjItRjY2JC1GIzY1LUY2NiYtRiM2Iy1GNjYmLUYjNictRiw2JVEnc3RhcnR4RidGL0YyRkEtRiw2JVEnc3RhcnR5RidGL0YyRkEtRiw2JVEnc3RhcnR6RidGL0YyRkUvJSVvcGVuR1EiW0YnLyUmY2xvc2VHUSJdRidGRUZjcEZmcEZBLUYsNiVRJ3N5bWJvbEYnRi9GMi1GQjYtUSI9RidGRUZHRmluRkxGTkZQRlJGVC9GV1EsMC4yNzc3Nzc4ZW1GJy9GWkZgcS1GLDYlUSxzb2xpZHNwaGVyZUYnRi9GMkZBRmZuLUYsNiVRK3N5bWJvbHNpemVGJ0YvRjJGXHEtSSNtbkdGJDYkUSMyMEYnRkVGQS1GLDYlUSp0aGlja25lc3NGJ0YvRjJGXHEtRmlxNiRRIjNGJ0ZFRkFGZm4tRiw2JVEmY29sb3JGJ0YvRjJGXHEtRiw2JVEkcmVkRidGL0YyRkVGRS9GZHBRInxmckYnL0ZncFEifGhyRictRiw2I1EhRidGRQ==the rain drop would follow the negative gradient of LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYlLUkjbWlHRiQ2JVEiZkYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJ0YvRjI= at every point. We simulate that path by following the negative gradient for a stepsize LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYkLUkjbWlHRiQ2JVEiaEYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy9GM1Enbm9ybWFsRic=, then recalculating the gradient, and so on, until the path ends in a local minimum, and the difference between successive LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYkLUkjbWlHRiQ2JVEiZkYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy9GM1Enbm9ybWFsRic=-values falls below a small threshold of LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYkLUklbXN1cEdGJDYlLUkjbW5HRiQ2JFEjMTBGJy8lLG1hdGh2YXJpYW50R1Enbm9ybWFsRictRiM2Ji1JI21vR0YkNi1RKiZ1bWludXMwO0YnRjIvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRj0vJSlzdHJldGNoeUdGPS8lKnN5bW1ldHJpY0dGPS8lKGxhcmdlb3BHRj0vJS5tb3ZhYmxlbGltaXRzR0Y9LyUnYWNjZW50R0Y9LyUnbHNwYWNlR1EsMC4yMjIyMjIyZW1GJy8lJ3JzcGFjZUdGTC1GLzYkUSI4RidGMi8lJ2l0YWxpY0dRJXRydWVGJy9GM1EnaXRhbGljRicvJTFzdXBlcnNjcmlwdHNoaWZ0R1EiMEYnRjI=.
First we need to calculate the components of the gradient LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYmLUkjbW9HRiQ2LVEmJkRlbDtGJy8lLG1hdGh2YXJpYW50R1Enbm9ybWFsRicvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRjQvJSlzdHJldGNoeUdGNC8lKnN5bW1ldHJpY0dGNC8lKGxhcmdlb3BHRjQvJS5tb3ZhYmxlbGltaXRzR0Y0LyUnYWNjZW50R0Y0LyUnbHNwYWNlR1EmMC4wZW1GJy8lJ3JzcGFjZUdRLDAuMTExMTExMWVtRictSSNtaUdGJDYlUSJmRicvJSdpdGFsaWNHUSV0cnVlRicvRjBRJ2l0YWxpY0YnLUYsNi1RIj1GJ0YvRjJGNUY3RjlGO0Y9Rj8vRkJRLDAuMjc3Nzc3OGVtRicvRkVGVEYvLUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYkLUkobWZlbmNlZEdGJDYkLUYjNictSSNtaUdGJDYlUSNmeEYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy1JI21vR0YkNi1RIixGJy9GOFEnbm9ybWFsRicvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRjYvJSlzdHJldGNoeUdGQi8lKnN5bW1ldHJpY0dGQi8lKGxhcmdlb3BHRkIvJS5tb3ZhYmxlbGltaXRzR0ZCLyUnYWNjZW50R0ZCLyUnbHNwYWNlR1EmMC4wZW1GJy8lJ3JzcGFjZUdRLDAuMzMzMzMzM2VtRictRjs2LVEifkYnRj5GQC9GREZCRkVGR0ZJRktGTUZPL0ZTRlEtRjE2JVEjZnlGJ0Y0RjdGPkY+Rj4=:LUklZGlmZkclKnByb3RlY3RlZEc2JC1JImZHNiI2JEkieEdGKEkieUdGKEYqLUklZGlmZkclKnByb3RlY3RlZEc2JC1JImZHNiI2JEkieEdGKEkieUdGKEYrQyY+SSNmeEc2ImYqNiRJInhHRiVJInlHRiVGJTYkSSlvcGVyYXRvckdGJUkmYXJyb3dHRiVGJSwoKiYtSSRjb3NHNiQlKnByb3RlY3RlZEdJKF9zeXNsaWJHRiU2IyomOSQiIiI5JUY3RjdGOEY3RjctSSRzaW5HRjE2I0Y2ISIiRjgjRjwiIiZGJUYlRiVGNz5JI2Z5R0YlZipGJ0YlRipGJSwoKiZGL0Y3RjZGN0Y3RjZGPSNGN0Y+RjdGJUYlRiVGNw==Now we start the loop with the starting point of the `rain drop', then adding LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYkLUkjbWlHRiQ2JVEiaEYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy9GM1Enbm9ybWFsRic= times the negative gradient.
[Notice that if we were to look for a local maximum, we would change the two minus signs to plus signs, because we would add LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYkLUkjbWlHRiQ2JVEiaEYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy9GM1Enbm9ybWFsRic= times the positive gradient.]
LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzZDLUkjbWlHRiQ2JVEncG9pbnR4RicvJSdpdGFsaWNHUSV0cnVlRicvJSxtYXRodmFyaWFudEdRJ2l0YWxpY0YnLUkobWZlbmNlZEdGJDYmLUYjNiMtSSNtbkdGJDYkUSIwRicvRjNRJ25vcm1hbEYnRj4vJSVvcGVuR1EiW0YnLyUmY2xvc2VHUSJdRictSSNtb0dGJDYtUSomY29sb25lcTtGJ0Y+LyUmZmVuY2VHUSZmYWxzZUYnLyUqc2VwYXJhdG9yR0ZMLyUpc3RyZXRjaHlHRkwvJSpzeW1tZXRyaWNHRkwvJShsYXJnZW9wR0ZMLyUubW92YWJsZWxpbWl0c0dGTC8lJ2FjY2VudEdGTC8lJ2xzcGFjZUdRLDAuMjc3Nzc3OGVtRicvJSdyc3BhY2VHRmVuLUYsNiVRJ3N0YXJ0eEYnRi9GMi1GRzYtUSI6RidGPkZKRk1GT0ZRRlNGVUZXRllGZm4tSSdtc3BhY2VHRiQ2Ji8lJ2hlaWdodEdRJjAuMGV4RicvJSZ3aWR0aEdRJjAuMGVtRicvJSZkZXB0aEdGY28vJSpsaW5lYnJlYWtHUShuZXdsaW5lRictRiw2JlEncG9pbnR5RidGLy8lK2V4ZWN1dGFibGVHRkxGMi1GNjYnLUYjNiMtRjs2JUY9Rl9wRj5GX3BGPkZARkMtRkc2LkZJRl9wRj5GSkZNRk9GUUZTRlVGV0ZZRmZuLUYsNiZRJ3N0YXJ0eUYnRi9GX3BGMi1GRzYuRl1vRl9wRj5GSkZNRk9GUUZTRlVGV0ZZRmZuRl5vLUYsNiZRJ3BvaW50ekYnRi9GX3BGMkZhcEZGLUYsNiVRJ3N0YXJ0ekYnRi9GMkZbb0Zeby1GX282JkZhb0Zkb0Znby9Gam9RJWF1dG9GJy1GLDYlRmBxRi9GMi1GNjYmLUYjNiQtRkc2LVEqJnVtaW51czA7RidGPkZKRk1GT0ZRRlNGVUZXL0ZaUSwwLjIyMjIyMjJlbUYnL0ZnbkZici1GOzYkUSIxRidGPkY+RkBGQ0ZGRmFxLUZHNi1RIitGJ0Y+RkpGTUZPRlFGU0ZVRldGYXJGY3JGZHJGW28tRkc2LVEifkYnRj5GSkZNRk9GUUZTRlVGVy9GWkZmby9GZ25GZm9GanItRkc2LVEiI0YnRj5GSkZNRk9GUUZTRlVGV0Zdc0Zecy1GLDYlUWdudGhpc35pc35uZWVkZWR+dG9+bWFrZX50aGV+Zmlyc3R+c3RlcH5vZn50aGV+d2hpbGV+bG9vcH53b3JrRidGL0YyRl5vRmRxLUYsNiNRIUYnLUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYvLUkjbWlHRiQ2JVEiaEYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy1JI21vR0YkNi1RKiZjb2xvbmVxO0YnL0YzUSdub3JtYWxGJy8lJmZlbmNlR1EmZmFsc2VGJy8lKnNlcGFyYXRvckdGPS8lKXN0cmV0Y2h5R0Y9LyUqc3ltbWV0cmljR0Y9LyUobGFyZ2VvcEdGPS8lLm1vdmFibGVsaW1pdHNHRj0vJSdhY2NlbnRHRj0vJSdsc3BhY2VHUSwwLjI3Nzc3NzhlbUYnLyUncnNwYWNlR0ZMLUkjbW5HRiQ2JFEjLjFGJ0Y5LUY2Ni1RIjpGJ0Y5RjtGPkZARkJGREZGRkhGSkZNLUknbXNwYWNlR0YkNiYvJSdoZWlnaHRHUSYwLjBleEYnLyUmd2lkdGhHUSYwLjBlbUYnLyUmZGVwdGhHRmVuLyUqbGluZWJyZWFrR1EobmV3bGluZUYnLUYsNiVRIk5GJ0YvRjJGNS1GUDYkUSIwRidGOUZTRlYtRjY2LVEifkYnRjlGO0Y+RkBGQkZERkZGSC9GS0Zobi9GTkZobi1GLDYjUSFGJ0Y5LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzZccC1JI21vR0YkNi1RMSZJbnZpc2libGVUaW1lcztGJy8lLG1hdGh2YXJpYW50R1Enbm9ybWFsRicvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRjQvJSlzdHJldGNoeUdGNC8lKnN5bW1ldHJpY0dGNC8lKGxhcmdlb3BHRjQvJS5tb3ZhYmxlbGltaXRzR0Y0LyUnYWNjZW50R0Y0LyUnbHNwYWNlR1EmMC4wZW1GJy8lJ3JzcGFjZUdGQy1GLDYvUSRmb3JGJy8lJWJvbGRHUSV0cnVlRicvRjBRJWJvbGRGJy8lK2ZvbnR3ZWlnaHRHRk1GMkY1RjdGOUY7Rj1GP0ZBRkQtRiw2LVEifkYnRi9GMkY1RjdGOUY7Rj1GP0ZBRkQtSSNtaUdGJDYlUSJpRicvJSdpdGFsaWNHRksvRjBRJ2l0YWxpY0YnRlAtRiw2L1ElZnJvbUYnRklGTEZORjJGNUY3RjlGO0Y9Rj9GQUZERlAtSSNtbkdGJDYkUSIxRidGL0ZQLUYsNi9RI3RvRidGSUZMRk5GMkY1RjdGOUY7Rj1GP0ZBRkRGUC1GaW42JFElMTAwMEYnRi9GUC1GLDYvUSZ3aGlsZUYnRklGTEZORjJGNUY3RjlGO0Y9Rj9GQUZERlBGUC1GVDYlUSRhYnNGJy9GWEY0Ri8tSShtZmVuY2VkR0YkNiQtRiM2KC1GVDYlUSdwb2ludHpGJ0ZXRlktRmpvNiYtRiM2JkZTLUYsNi1RKCZtaW51cztGJ0YvRjJGNUY3RjlGO0Y9Rj8vRkJRLDAuMjIyMjIyMmVtRicvRkVGaXBGaG5GL0YvLyUlb3BlbkdRIltGJy8lJmNsb3NlR1EiXUYnRmVwRl5wLUZqbzYmLUYjNiZGU0ZlcC1GaW42JFEiMkYnRi9GL0YvRltxRl5xRi9GL0ZQLUYsNi1RIj5GJ0YvRjJGNUY3RjlGO0Y9Rj8vRkJRLDAuMjc3Nzc3OGVtRicvRkVGXHItRmluNiRRKi4wMDAwMDAwMUYnRi9GUC1GLDYvUSNkb0YnRklGTEZORjJGNUY3RjlGO0Y9Rj9GQUZELUknbXNwYWNlR0YkNiYvJSdoZWlnaHRHUSYwLjBleEYnLyUmd2lkdGhHRkMvJSZkZXB0aEdGaXIvJSpsaW5lYnJlYWtHUShuZXdsaW5lRictRlQ2JlEncG9pbnR4RidGVy8lK2V4ZWN1dGFibGVHRjRGWS1Gam82Jy1GIzYkLUZUNiZGVkZXRmRzRllGL0Zkc0YvRltxRl5xLUYsNi1RKiZjb2xvbmVxO0YnRi9GMkY1RjdGOUY7Rj1GP0ZbckZdckZQLUZUNiVGY3NGV0ZZLUZqbzYmLUYjNiZGUy1GLDYtUSomdW1pbnVzMDtGJ0YvRjJGNUY3RjlGO0Y9Rj9GaHBGanBGaG5GL0YvRltxRl5xRlBGZXRGUC1GVDYlUSJoRidGV0ZZLUYsNi1RJyZzZG90O0YnRi9GMkY1RjdGOUY7Rj1GP0ZBRkQtRlQ2JVEjZnhGJ0ZXRlktRmpvNiQtRiM2KEZfdEZhdC1GLDYtUSIsRidGL0YyL0Y2RktGN0Y5RjtGPUY/RkEvRkVRLDAuMzMzMzMzM2VtRictRlQ2JVEncG9pbnR5RidGV0ZZRmF0Ri9GLy1GLDYtUSI6RidGL0YyRjVGN0Y5RjtGPUY/RltyRl1yRlBGZHJGW3YtRmpvNiYtRiM2JEZTRi9GL0ZbcUZecUZcdEZQRlt2RmF0RlBGZXRGUEZodC1GLDYuRl11RldGWUYyRjVGN0Y5RjtGPUY/RkFGRC1GVDYlUSNmeUYnRldGWUZhdUZedkZQRmRyRl5wRmF2Rlx0RlAtRlQ2JVEiZkYnRldGWS1Gam82JC1GIzYoRl90RmF2RmV1Rlt2RmF2Ri9GL0ZedkZkckZQLUZUNiVRIk5GJ0ZXRllGXHRGYXctRiw2LVEiK0YnRi9GMkY1RjdGOUY7Rj1GP0ZocEZqcEZobi1GLDYtUSI7RidGL0YyRmh1RjdGOUY7Rj1GP0ZBRl1yRmRyLUYsNi9RJGVuZEYnRklGTEZORjJGNUY3RjlGO0Y9Rj9GQUZERlBGYXJGXnZGZHItRmVyNiZGZ3JGanJGXHMvRl9zUSVhdXRvRictRlQ2I1EhRidGLw==LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYnLUkjbWlHRiQ2JVEmY3VydmVGJy8lJ2l0YWxpY0dRJXRydWVGJy8lLG1hdGh2YXJpYW50R1EnaXRhbGljRictSSNtb0dGJDYtUSomY29sb25lcTtGJy9GM1Enbm9ybWFsRicvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRj0vJSlzdHJldGNoeUdGPS8lKnN5bW1ldHJpY0dGPS8lKGxhcmdlb3BHRj0vJS5tb3ZhYmxlbGltaXRzR0Y9LyUnYWNjZW50R0Y9LyUnbHNwYWNlR1EsMC4yNzc3Nzc4ZW1GJy8lJ3JzcGFjZUdGTC1JKG1mZW5jZWRHRiQ2Ji1GIzYlLUYsNiVRJHNlcUYnRi9GMi1GUDYkLUYjNiotRlA2Ji1GIzYrLUYsNiVRJ3BvaW50eEYnRi9GMi1GUDYmLUYjNiQtRiw2JVEiaUYnRi9GMkY5RjkvJSVvcGVuR1EiW0YnLyUmY2xvc2VHUSJdRictRjY2LVEiLEYnRjlGOy9GP0YxRkBGQkZERkZGSC9GS1EmMC4wZW1GJy9GTlEsMC4zMzMzMzMzZW1GJy1GLDYlUSdwb2ludHlGJ0YvRjJGXG9GaW8tRiw2JVEncG9pbnR6RidGL0YyRlxvRjlGOUZjb0Zmb0Zpb0Zgby1GNjYtUSI9RidGOUY7Rj5GQEZCRkRGRkZIRkpGTS1JI21uR0YkNiRRIjBGJ0Y5LUY2Ni1RIy4uRidGOUY7Rj5GQEZCRkRGRkZIL0ZLUSwwLjIyMjIyMjJlbUYnL0ZORl5wLUYsNiVRIk5GJ0YvRjJGOUY5RjlGOUZjb0Zmby1GNjYtUSI6RidGOUY7Rj5GQEZCRkRGRkZIRkpGTUY5LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYoLUkjbWlHRiQ2JVEqY3VydmVwbG90RicvJSdpdGFsaWNHUSV0cnVlRicvJSxtYXRodmFyaWFudEdRJ2l0YWxpY0YnLUkjbW9HRiQ2LVEqJmNvbG9uZXE7RicvRjNRJ25vcm1hbEYnLyUmZmVuY2VHUSZmYWxzZUYnLyUqc2VwYXJhdG9yR0Y9LyUpc3RyZXRjaHlHRj0vJSpzeW1tZXRyaWNHRj0vJShsYXJnZW9wR0Y9LyUubW92YWJsZWxpbWl0c0dGPS8lJ2FjY2VudEdGPS8lJ2xzcGFjZUdRLDAuMjc3Nzc3OGVtRicvJSdyc3BhY2VHRkwtRiw2JVEscG9pbnRwbG90M2RGJ0YvRjItSShtZmVuY2VkR0YkNiQtRiM2Oy1GLDYlUSZjdXJ2ZUYnRi9GMi1GNjYtUSIsRidGOUY7L0Y/RjFGQEZCRkRGRkZIL0ZLUSYwLjBlbUYnL0ZOUSwwLjMzMzMzMzNlbUYnLUYsNiVRKGNvbm5lY3RGJ0YvRjItRjY2LVEiPUYnRjlGO0Y+RkBGQkZERkZGSEZKRk0tRiw2JUYxRi9GMkZaLUY2Ni1RMSZJbnZpc2libGVUaW1lcztGJ0Y5RjtGPkZARkJGREZGRkhGaG4vRk5GaW4tRiw2JVEnc3ltYm9sRidGL0YyRl9vLUYsNiVRJGJveEYnRi9GMkZaRmRvLUYsNiVRK3N5bWJvbHNpemVGJ0YvRjJGX28tSSNtbkdGJDYkUSI1RidGOUZaLUY2Ni1RIn5GJ0Y5RjtGPkZARkJGREZGRkhGaG5GZ28tRiw2JVEqdGhpY2tuZXNzRidGL0YyRl9vLUZicDYkUSIyRidGOUZaLUYsNiVRJmNvbG9yRidGL0YyRl9vLUYsNiVRJHJlZEYnRi9GMkY5RjktRjY2LVEiOkYnRjlGO0Y+RkBGQkZERkZGSEZKRk1GOQ==LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYtLUkjbWlHRiQ2JVEoZGlzcGxheUYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy1JKG1mZW5jZWRHRiQ2JC1GIzYqLUY2NiYtRiM2Li1GLDYlUSxzdXJmYWNlcGxvdEYnRi9GMi1JI21vR0YkNi1RIixGJy9GM1Enbm9ybWFsRicvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRjEvJSlzdHJldGNoeUdGSS8lKnN5bW1ldHJpY0dGSS8lKGxhcmdlb3BHRkkvJS5tb3ZhYmxlbGltaXRzR0ZJLyUnYWNjZW50R0ZJLyUnbHNwYWNlR1EmMC4wZW1GJy8lJ3JzcGFjZUdRLDAuMzMzMzMzM2VtRictRkI2LVExJkludmlzaWJsZVRpbWVzO0YnRkVGRy9GS0ZJRkxGTkZQRlJGVEZWL0ZaRlgtRiw2JVEqY3VydmVwbG90RidGL0YyRkEtRkI2LVEifkYnRkVGR0ZpbkZMRk5GUEZSRlRGVkZqbi1GLDYlUSxwb2ludHBsb3QzZEYnRi9GMi1GNjYkLUYjNjYtRjY2Ji1GIzYkLUY2NiYtRiM2KC1GLDYlUSdzdGFydHhGJ0YvRjJGQS1GLDYlUSdzdGFydHlGJ0YvRjJGQS1GLDYlUSdzdGFydHpGJ0YvRjJGRUZFLyUlb3BlbkdRIltGJy8lJmNsb3NlR1EiXUYnRkVGRUZpcEZccUZBLUYsNiVRJ3N5bWJvbEYnRi9GMi1GQjYtUSI9RidGRUZHRmluRkxGTkZQRlJGVC9GV1EsMC4yNzc3Nzc4ZW1GJy9GWkZmcS1GLDYlUSxzb2xpZHNwaGVyZUYnRi9GMkZBRl5vLUYsNiVRK3N5bWJvbHNpemVGJ0YvRjJGYnEtSSNtbkdGJDYkUSMxMEYnRkVGQS1GLDYlUSp0aGlja25lc3NGJ0YvRjJGYnEtRl9yNiRRIjNGJ0ZFRkFGXm8tRiw2JVEmY29sb3JGJ0YvRjJGYnEtRiw2JVEkcmVkRidGL0YyRkVGRUZBRmFvLUY2NiQtRiM2Ni1GNjYmLUYjNiUtRjY2Ji1GIzYtLUYsNiVRJ3BvaW50eEYnRi9GMi1GNjYmLUYjNiQtRiw2JVEiTkYnRi9GMkZFRkVGaXBGXHFGQUZeby1GLDYlUSdwb2ludHlGJ0YvRjJGXXRGQUZeby1GLDYlUSdwb2ludHpGJ0YvRjJGXXRGRUZFRmlwRlxxLUYsNiNRIUYnRkVGRUZpcEZccUZBRl9xRmJxRmhxRkFGXm9GW3JGYnFGXnJGQUZickZicUZlckZBRl5vRmhyRmJxRltzRkVGRUZFRkUvRmpwUSJ8ZnJGJy9GXXFRInxockYnRkFGXm8tRiw2JVEsb3JpZW50YXRpb25GJ0YvRjJGYnEtRjY2Ji1GIzYmLUZfcjYkUSMzN0YnRkVGQS1GX3I2JFEjMTVGJ0ZFRkVGRUZpcEZccUZqdEZFRkVGanQtSSdtc3BhY2VHRiQ2Ji8lJ2hlaWdodEdRJjAuMGV4RicvJSZ3aWR0aEdGWC8lJmRlcHRoR0Zjdi8lKmxpbmVicmVha0dRKG5ld2xpbmVGJy1GX3Y2JkZhdkZkdkZmdi9GaXZRJWF1dG9GJ0ZedkZbd0ZedkZbd0ZqdEZFLUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYyLUkjbWlHRiQ2JVEiTkYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy1JI21vR0YkNi1RIjtGJy9GM1Enbm9ybWFsRicvJSZmZW5jZUdRJmZhbHNlRicvJSpzZXBhcmF0b3JHRjEvJSlzdHJldGNoeUdGPS8lKnN5bW1ldHJpY0dGPS8lKGxhcmdlb3BHRj0vJS5tb3ZhYmxlbGltaXRzR0Y9LyUnYWNjZW50R0Y9LyUnbHNwYWNlR1EmMC4wZW1GJy8lJ3JzcGFjZUdRLDAuMjc3Nzc3OGVtRictSSdtc3BhY2VHRiQ2Ji8lJ2hlaWdodEdRJjAuMGV4RicvJSZ3aWR0aEdGTC8lJmRlcHRoR0ZVLyUqbGluZWJyZWFrR1EobmV3bGluZUYnLUZRNiZGU0ZWRlgvRmVuUSVhdXRvRictRiw2JVEncG9pbnR4RidGL0YyLUkobWZlbmNlZEdGJDYmLUYjNiNGK0Y5LyUlb3BlbkdRIltGJy8lJmNsb3NlR1EiXUYnRjVGUC1GNjYtUSJ+RidGOUY7L0Y/Rj1GQEZCRkRGRkZIRkovRk5GTC1GLDYlUSdwb2ludHlGJ0YvRjJGXm9GNUZQRmlvLUYsNiVRImZGJ0YvRjItRl9vNiQtRiM2J0Zbb0Zeby1GNjYtUSIsRidGOUY7Rj5GQEZCRkRGRkZIRkovRk5RLDAuMzMzMzMzM2VtRidGXnBGXm9GOQ==Our result: we see a path of steepest descent from a starting point at (.1,0) to a local minimum at (3.1117, -.4611) with minimum function value of -1.7955797.
The minimum value is accurate to about 7 decimals by the way we designed the loop.
The accuracy of the LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYlLUkobWZlbmNlZEdGJDYkLUYjNiYtSSNtaUdGJDYlUSJ4RicvJSdpdGFsaWNHUSV0cnVlRicvJSxtYXRodmFyaWFudEdRJ2l0YWxpY0YnLUkjbW9HRiQ2LVEiLEYnL0Y4USdub3JtYWxGJy8lJmZlbmNlR1EmZmFsc2VGJy8lKnNlcGFyYXRvckdGNi8lKXN0cmV0Y2h5R0ZCLyUqc3ltbWV0cmljR0ZCLyUobGFyZ2VvcEdGQi8lLm1vdmFibGVsaW1pdHNHRkIvJSdhY2NlbnRHRkIvJSdsc3BhY2VHUSYwLjBlbUYnLyUncnNwYWNlR1EsMC4zMzMzMzMzZW1GJy1GMTYlUSJ5RidGNEY3Rj5GPi1GMTYjUSFGJ0Y+-location is harder to tell. In our example I find that the coordinates are accurate to about 4 decimals.Final Comment: Naturally, if we were only interested in the minimum, and not the curious path the rain drop takes, we would start much closer to the minimum in question, and be done a bit quicker. . . . :)JSFHphysics interpretationIf one interprets the function as a potential for a conservative 2d force field obtained from its sign-reversed gradient, the orthogonal trajectories are intepreted as field lines of the gradient force field, like an electric field. This converts the problem into finding the integral curves of the force vector field, which gives a system of ordinary differential equations that can be solved numerically.QyYtSSpwb2ludHBsb3RHNiI2JjcjNyQiIiFGKS9JJ3N5bWJvbEclKnByb3RlY3RlZEdJLHNvbGlkY2lyY2xlR0YlL0krc3ltYm9sc2l6ZUdGJSIjOi9JJmNvbG9yR0YlSSRyZWRHRiUhIiI+SSdvcmlnaW5HRiVJIiVHRiVGNA==QyYtSSxjb250b3VycGxvdEc2IjYoLUkiZkdGJTYkSSJ4R0YlSSJ5R0YlL0YqOyEiJSIiJS9GK0YtL0klYXhlc0dGJUkmYm94ZWRHRiUvSSljb250b3Vyc0dGJSIjXS9JJWdyaWRHRiU3JCIkXSNGOiEiIj5JJmNwbG90R0YlSSIlR0YlRjs=LUkoZGlzcGxheUc2IjYkSSdvcmlnaW5HRiRJJmNwbG90R0YkThe negative gradient is the force field:QyU+SSJYRzYiZio2JEkieEdGJUkieUdGJUYlNiRJKW9wZXJhdG9yR0YlSSZhcnJvd0dGJUYlLCQtSSQ8LD5HRiU2JC0tJkkiREdGJTYjIiIiNiNJImZHRiU2JDkkOSUtLSZGNDYjIiIjRjdGOSEiIkYlRiVGJUZBLUYkRic=At the origin it starts out pointing downward:LUkiWEc2IjYkIiIhRiY=The fieldplot is not so useful since the values near the origin are so small compared to the largest values in the viewing window.LUkqZmllbGRwbG90RzYiNiUtSSJYR0YkNiRJInhHRiRJInlHRiQvRik7ISIlIiIlL0YqRiw=But we can instead plot the unit direction vector field.LUkqZmllbGRwbG90RzYiNiYqJi1JIlhHRiQ2JEkieEdGJEkieUdGJCIiIi1JJXNxcnRHNiQlKnByb3RlY3RlZEdJKF9zeXNsaWJHRiQ2Iy1JMGRlbGF5RG90UHJvZHVjdEc2JEYwL0krbW9kdWxlbmFtZUdGJEksVHlwZXNldHRpbmdHRi82JEYnRichIiIvRio7ISIlIiIlL0YrRjwvSSVncmlkR0YkNyQiIzxGQw==If we zoom in on the origin, it is a little better behaved and one can see that the integral curve starting out tangent to the LUklbXJvd0c2Iy9JK21vZHVsZW5hbWVHNiJJLFR5cGVzZXR0aW5nR0koX3N5c2xpYkdGJzYkLUkjbWlHRiQ2JVEieUYnLyUnaXRhbGljR1EldHJ1ZUYnLyUsbWF0aHZhcmlhbnRHUSdpdGFsaWNGJy9GM1Enbm9ybWFsRic=-axis and pointing downward will begin veering to the right.QyYtSSpmaWVsZHBsb3RHNiI2JiomLUkiWEdGJTYkSSJ4R0YlSSJ5R0YlIiIiLUklc3FydEc2JCUqcHJvdGVjdGVkR0koX3N5c2xpYkdGJTYjLUkwZGVsYXlEb3RQcm9kdWN0RzYkRjEvSSttb2R1bGVuYW1lR0YlSSxUeXBlc2V0dGluZ0dGMDYkRihGKCEiIi9GKzskISImRjskIiImRjsvRixGPS9JJWdyaWRHRiU3JCIjPEZGRi0+STFoYWxmc3F1YXJlYXJyb3dzR0YlSSIlR0YlRjs=QyYtSSxjb250b3VycGxvdEc2IjYoLUkiZkdGJTYkSSJ4R0YlSSJ5R0YlL0YqOyQhIiYhIiIkIiImRjAvRitGLS9JJWF4ZXNHRiVJJmJveGVkR0YlL0kpY29udG91cnNHRiUiI10vSSVncmlkR0YlNyQiJF0jRj1GMD5JM2hhbGZzcXVhcmVjb250b3Vyc0dGJUkiJUdGJUYwHere is the plot with the contours.LUkoZGlzcGxheUc2IjYlSTFoYWxmc3F1YXJlYXJyb3dzR0YkSTNoYWxmc3F1YXJlY29udG91cnNHRiRJJ29yaWdpbkdGJA==Now we set up the numeric dsolve procedure.PkklZGVxc0c2IjYkLy0tSSJERzYkJSpwcm90ZWN0ZWRHSShfc3lzbGliR0YkNiNJInhHRiQ2I0kidEdGJCYtSSJYR0YkNiQtRi5GLy1JInlHRiRGLzYjIiIiLy0tRik2I0Y3Ri8mRjI2IyIiIw==PkkmaW5pdHNHNiI2JC8tSSJ4R0YkNiMiIiFGKi8tSSJ5R0YkRilGKg==Pkkkc29sRzYiLUknZHNvbHZlRzYkJSpwcm90ZWN0ZWRHSShfc3lzbGliR0YkNiY8JEklZGVxc0dGJEkmaW5pdHNHRiQ8JC1JInhHRiQ2I0kidEdGJC1JInlHRiRGMS9JJXR5cGVHRihJKG51bWVyaWNHRigvSSdvdXRwdXRHRiRJLmxpc3Rwcm9jZWR1cmVHRiQ=Then we extract the two solution functions, which can be integrated, differentiated, plotted, etc.QyU+SSJYRzYiLUklZXZhbEclKnByb3RlY3RlZEc2JC1JInhHRiU2I0kidEdGJUkkc29sR0YlIiIiPkkiWUdGJS1GJzYkLUkieUdGJUYsRi4=QyYtSSVwbG90RzYkJSpwcm90ZWN0ZWRHSShfc3lzbGliRzYiNiU3JS1JIlhHRig2I0kidEdGKC1JIllHRihGLS9GLjsiIiEiIzUvSSZjb2xvckdGKEkkcmVkR0YoL0kqdGhpY2tuZXNzR0YoIiIjIiIiPkkoc29scGxvdEdGKEkiJUdGKCEiIg==This is the solution, showing the termination of the curve at the local minimum.LUkoZGlzcGxheUc2IjYlSSdvcmlnaW5HRiRJJmNwbG90R0YkSShzb2xwbG90R0YkThe physical interpretation of the system point rolling down the potential well walls to its minimum, picking up speed with the increas of kinetic energy, but that is another story.QyYtSStzcGFjZWN1cnZlRzYiNiY3JS1JIlhHRiU2I0kidEdGJS1JIllHRiVGKi1JImZHRiU2JEYoRiwvRis7IiIhIiM1L0kmY29sb3JHRiVJJHJlZEdGJS9JKnRoaWNrbmVzc0dGJSIiIyIiIj5JKnNvbHBsb3QzZEdGJUkiJUdGJSEiIg==QyYtSSxwb2ludHBsb3QzZEc2IjYnNyM3JSIiIUYpLUkiZkdGJTYkRilGKS9JJ3N5bWJvbEclKnByb3RlY3RlZEdJLHNvbGlkc3BoZXJlR0YlL0krc3ltYm9sc2l6ZUdGJSIjPy9JKnRoaWNrbmVzc0dGJSIiJC9JJmNvbG9yR0YlSSRyZWRHRiUhIiI+SSlvcmlnaW4zZEdGJUkiJUdGJUY6LUkoZGlzcGxheUc2IjYlSSlvcmlnaW4zZEdGJEksc3VyZmFjZXBsb3RHRiRJKnNvbHBsb3QzZEdGJA==Nice.