Solves a nonlinear least-squares problem using a modified Levenberg-Marquardt algorithm.

f—A scalar string specifying a user-supplied function to evaluate the function that defines the least-squares problem. The f  function accepts the following two parameters and returns an array of length m containing the function values at x:

m—The number of functions.

n—The number of variables where n ≤ m.

result—The solution x of the nonlinear least-squares problem.

Double—If present and nonzero, double precision is used.

Fjac—The name of the variable into which an array of size n-by-m containing the Jacobian at the approximate solution is stored.

FScale—An array with m components containing the diagonal scaling matrix for the functions. The i-th component of FScale is a positive scalar specifying the reciprocal magnitude of the i-th component function of the problem. (Default: FScale(*) = 1)

Fvec—The name of the variable into which a real array of length m containing the residuals at the approximate solution is stored.

Good_Digits—The number of good digits in the function. (Default: machine dependent)

Intern_Scale—An internal variable scaling option. With this keyword, the values for XScale are set internally.

Itmax—The maximum number of iterations. (Default: Itmax = 100)

Jacobian—A scalar string specifying a user-supplied function to compute the Jacobian. This function accepts two parameters and returns an n-by-m array containing the Jacobian at the input point s. Note that each derivative ∂fi/∂xi1 should be returned in the (i, j) element of the returned matrix. The parameters of the function are as follows:

JTJ_inverse—The name of the variable into which an array of size n-by-m containing the inverse matrix of J TJ , where J is the final Jacobian, is stored. If J TJ is singular, the inverse is a symmetric gz inverse of J TJ. (See the PV‑WAVE IMSL Mathematics Reference CHNNDSOL function for a discussion of generalized inverses and the definition of the gz inverse.)

Max_Evals—The maximum number of function evaluations. (Default: Max_Evals = 400)

Max_Jacobian—The maximum number of Jacobian evaluations. (Default: Max_Jacobian = 400)

Max_Step—The maximum allowable step size.
(Default: Max_Step = 1000max(ε1, ε2), where:

Rank—Name of the variable into which the rank of the Jacobian is stored.

Tol_Afcn—The absolute function tolerance. (Default: Tol_Afcn = max(10–20, ε2), [max(10–40, ε2) in double], where ε is the machine precision)

Tolerance—The tolerance used in determining linear dependence for the computation of the inverse of J TJ . (If Jacobian is specified, Tolerance = 100ε where ε is the machine precision, is the default; otherwise, ε1/2, where ε is the machine precision, is the default.)

Tol_Grad—The scaled gradient tolerance. The i-th component of the scaled gradient at x is calculated as:

where , s = XScale, and:

(Default: Tol_Step = ε1/2 (ε1/3 in double), where ε is the machine precision)

Tol_Rfcn—The relative function tolerance. (Default: Tol_Rfcn = max(10–10, ε2/3), [max(10–40, ε2/3) in double], where ε is the machine precision)

Tol_Step—The scaled step tolerance. The i-th component of the scaled step between two points x and y is computed as:

where s = XScale. (Default: Tol_Step = ε2/3, where ε is machine precision)

Trust_Region—The size of initial trust-region radius. (Default: based on the initial scaled Cauchy step)

XGuess—An array with N  components containing an initial guess. (Default: XGuess(*) = 0)

XScale—An array with n components containing the scaling array for the variables. Keyword XScale is used mainly in scaling the gradient and the distance between two points. See keywords Tol_Grad and Tol_Step for more detail. (Default: XScale(*) = 1)

See CHNNDSOL for a discussion of generalized inverses and the definition of the gz inverse.

NLINLSQ is based on the MINPACK routine LMDER by Moré et al. (1980), and uses a modified Levenberg-Marquardt method to solve nonlinear least-squares problems.

(For more details about the algorithms used by NLINLSQ, refer to the description of NLINLSQ in the PV‑WAVE IMSL Mathematics Reference.)

The problem is stated as follows:

where m ≥ n, F : Rn → Rm and fi(x) is the i-th component function of F(x). From a current point, the algorithm uses the trust region approach:

to get a new point x0, compute xs as:

where μc = 0 if , and μc ≥ 0 otherwise. The value μc is defined by the function. The array and matrix F(xc) and J(xc) are the function values and the Jacobian evaluated at the current point xc. This function is repeated until the stopping criteria are satisfied.

The first stopping criterion for NLINLSQ occurs when the norm of the function is less than the absolute function tolerance, Tol_Afcn. The second stopping criterion occurs when the norm of the scaled gradient is less than the given gradient tolerance Tol_Grad. The third stopping criterion for NLINLSQ occurs when the scaled distance between the last two steps is less than the step tolerance Tol_Step. For more details, see Levenberg (1944), Marquardt (1963), or Dennis and Schnabel (1983, Chapter 10).

This example uses the nonlinear data-fitting problem found in Dennis and Schnabel (1983, p. 225).

The problem is stated as follows:

where:

is solved with the data t = [1, 2, 3] and y = [2, 4, 3].

.RUN

; Define the function that defines the least-squares problem.

FUNCTION f, m, x

y = [2, 4, 3]

t = [1, 2, 3]

RETURN, EXP(x(0) * t) - y

END

% Compiled module: F.

solution = NLINLSQ("f", 3, 1)

PM, solution, Title = 'The solution is:'

PM, f(m, solution), Title = 'The function values are:'

In the PV‑WAVE IMSL Mathematics Reference:

CHNNDSOL

Dennis and Schnabel, 1983, Chapter 10. Levenberg, 1944. Marquardt, 1963.

Moré, Garbow and Hillstrom,1980.