Get 20M+ Full-Text Papers For Less Than $1.50/day. Start a 14-Day Trial for You or Your Team.

Learn More →

Logarithmic Barrier Method Via Minorant Function for Linear Semidefinite Programming

Logarithmic Barrier Method Via Minorant Function for Linear Semidefinite Programming Annales Mathematicae Silesianae 37 (2023), no. 1, 95–116 DOI: 10.2478/amsil-2022-0021 LOGARITHMIC BARRIER METHOD VIA MINORANT FUNCTION FOR LINEAR SEMIDEFINITE PROGRAMMING Assma Leulmi Abstract. We propose in this study, a new logarithmic barrier approach to solve linear semidefinite programming problem. We are interested in computa- tion of the direction by Newton’s method and of the displacement step using minorant functions instead of line search methods in order to reduce the com- putation cost. Our new approach is even more beneficial than classical line search meth- ods. This purpose is confirmed by some numerical simulations showing the effectiveness of the algorithm developed in this work, which are presented in the last section of this paper. 1. Introduction Semidefinite programming (SDP) problem is an optimization model of a linear function of a symmetric matrix subject to linear equality constraints and the additional condition that the matrix be positive semidefinite. SDPs include as special cases linear programmings (LP), when all the symmetric matrices involved are diagonal. General SDP is perhaps one of the most pow- erful forms of convex optimization. Semidefinite programming permits to solve numerous problems, as non-linear programming (NLP) problems, quadratic programming (QP) problems,.... Received: 21.02.2021. Accepted: 27.12.2022. Published online: 07.02.2023. (2020) Mathematics Subject Classification: 90C22, 90C51. Key words and phrases: linear semidefinite programming, barrier methods, line search. c 2022 The Author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution License CC BY (http://creativecommons.org/licenses/by/4.0/). 96 Assma Leulmi Many algorithms have been suggested to resolve linear SDP problem such that, Interior-point methods (IPMs) for SDP have been pioneered by Nesterov and Nemirovskii ([9]) as well as Alizadeh et al. ([1]). Several methods have been proposed to solve SDP such that, projective IPMs and their alternatives ([12, 7]), central trajectory methods ([13]), loga- rithmic barrier methods ([4]). Our work is based on the latter type of IPMs, the obstacle for establishing an iteration is the determination and computation of the displacement step. Unfortunately, the computation of the displacement step, especially when us- ing line search methods, is costly and even more difficult in the case of semi- definite problems ([4]). In this paper, we are interested in solving SDP by IPMs. The idea of this method consists to solve SDP with a new logarithmic barrier approach. Then, we use Newton’s method to treat the associated perturbed equations to obtain a descent direction. We propose an alternative ways to determine the displacement step along the direction which are more efficient than classical line searches. We consider the following SDP problem min b x; x A C 2 S ; (1) i i i=1 x 2 R : Here S designs the cone of the symmetrical semidefinite positive nn matrix, matrices C, A ; with i = 1; : : : ; m; are the given symmetrical matrices and b 2 R : The problem (1) is the dual of the following semidefinite problem maxhC; Yi; hA ; Yi = b ; 8i = 1; : : : ; m; (2) i i Y 2 S : We denote by hC; Yi the trace of the matrix (C Y ), and recall that h;i corresponds to an inner product on the space of n n matrices. Their feasible sets involving a non polyhedral convex cone of positive semi- definite matrices are called linear semidefinite programs. A priory, one of the advantages of the problem (1) with respect to its dual problem (2) is that variable of the objective function is a vector instead to be a matrix in the type problem (2). Furthermore, under certain convenient hypothesis, the resolution of the problem (1) is equivalent to the problem (2) in the sense that the optimal solution of one of the two problems can be Logarithmic barrier method via minorant function... 97 reduced directly from the other through the application of the theorem of the slackness complementary, see for instance [1, 6, 8]. In all which follows, we denote by m + 1. X = fx 2 R : x A C 2 S g; the set of feasible solutions of (1). i i i=1 m + 2. X = fx 2 R : x A C 2 int(S )g; the set of strictly feasible i i i=1 n solutions of (1). 3. F = fY 2 S : hA ; Yi = b ; 8i = 1; : : : ; mg; the set of feasible solutions i i of (2). 4. F = fY 2 F : Y 2 int(S )g; the set of strictly feasible solutions of (2). int(S ) is the set of the symmetrical definite positive n n matrices. The problem (1) is approximated by the following problem (SDP ) min f (x); (SDP ) x 2 R ; with the penalty parameter  > 0 and f : R ! ]1; +1] is the barrier function defined by b x + n ln   ln[det( x A C )] if x 2 X; i i f (x) = i=1 +1 if not. The problem (SDP ) can be solved via a classical Newton descent method. The difficulty with the line search is the presence of the determinant in the definition of the logarithmic barrier function which leads to a very high cost in the classical procedures of exact or approximate line search. In our approach, instead of minimizing f ; along the descent direction at a current point x; we propose minorants functions G for which the optimal solution of the displacement step is obtained explicitly. Let us minimize the function G such that [f (x + d) f (x)] = G( )  G( ); 8 > 0; 0 0 with G(0) = G(0) = 0; G (0) = G (0) < 0: The best quality of the approximations G of G is ensured by the condition 00 00 G (0) = G (0): The idea of this new approach is to introduce one original process to calculate the displacement step based on minorants functions. Then, we obtain an explicit approximation which leads to reducing the objective, adding to this, it is economical and robust, contrary to the traditional methods of line search. 98 Assma Leulmi Backdrop and brief information in linear semidefinite programming. Let us state the following necessary assumptions (A1) The system of equations hA ; Yi = b ; i = 1; : : : ; m is of rank m: i i b b (A2) The sets X and F are not empty. We know that (see [1, 2]) 1. The sets of optimal solutions of problems (2) and (1) are non empty convex and compact. 2. If x  is an optimal solution of (1), then Y is an optimal solution of (2) if and only if Y 2 F and x  A C Y = 0: i i i=1 3. If Y is an optimal solution of (2); then x  is an optimal solution of (1) if and only if x  2 X and x  A C Y = 0: i i i=1 According to the assumptions (A1) and (A2), the solution of problem (1) permits to give the one of problem (2) and vice-versa. We study in the next section, existence and uniqueness of optimal solution of the problem (SDP ) and its convergence to problem (2), in particular the behaviour of its optimal value and its optimal solutions when  ! 0: The k+1 k solution of this problem is of descent type, defined by x = x + d , k k where d is the descent direction and is the displacement step. k k Then, we show in section 3, how to compute the Newton descent direc- tion d. In section 4, we present new three different approximations of G; to compute the displacement step. These approximations are deduced from in- equalities shown in section 4. In section 5, we describe the obtained algorithm. In section 6, we present numerical tests with commentaries on some different examples to illustrate the effectiveness of the three proposed approaches and we compare them with the standard line search method. The paper is finished by conclusions in the last section. The main advantage of (SDP ) resides in the strict convexity of its ob- jective function and the convexity of its feasible domain. Consequently, the conditions of optimality are necessary and sufficient. This, fosters theoretical and numerical studies of the problem. Before this, it is necessary to show that (SDP ) has at least an optimal solution. Logarithmic barrier method via minorant function... 99 2. Existence and uniqueness of optimal solution of problem (SDP ) and its convergence to problem (1) 2.1. Fundamental properties of f For x 2 X, let us introduce the symmetrical definite positive matrix B(x) of rank m; and the lower triangular matrix L(x); such that B(x) = x A C = L(x)L (x); i i i=1 and let us define, for i; j = 1; : : : ; m 1 T 1 A (x) = [L(x)] A [L (x)] ; i i b (x) = trace(A (x)) = trace(A B (x)); i i i 1 1 b b (x) = trace(B (x)A B (x)A ) = trace(A (x)A (x)): ij i j i j Thus, b(x) = (b (x)) is a vector of R and (x) = ( (x)) i ij i=1;:::;m i;j=1;:::;m is a symmetrical matrix of rank m. The previous notation will be used in the expressions of the gradient and the Hessian H of f : To show that problem (SDP ) has a solution, it is sufficient to show that f is inf-compact. Theorem 1 ([4]). The function f is twice continuously differentiable on b b X: Actually, for all x 2 X we have: (a) rf (x) = b b(x): (b) H = r f (x) = (x): (c) The matrix (x) is definite positive. Since f is strictly convex, (SDP ) has at most one optimal solution. 2.2. Problem (SDP ) has one unique optimal solution Firstly, we start with the following definition Definition 1. Let f be a function defined from R to R [ f1g; f is called inf-compact if for all  > 0; the set S (f ) = fx 2 R : f (x)  g is compact, which comes in particular to say that its cone of recession is reduced to zero. 100 Assma Leulmi As the function f takes the value +1 on the boundary of X and is differentiable on X; then it is lower semi-continuous. In order to prove that (SDP ) has one optimal solution, it suffices to prove that recession cone of f S (f ) = fd 2 R : (f ) (d)  0g ; 0   1 is reduced to zero i.e., d = 0 if (f ) (d)  0; where (f ) is defined for x 2 X as f (x + d) f (x) (f ) (d) = lim : !+1 This leads to the following proposition. Proposition 1 ([4]). If b d  0 and d A 2 X then d = 0: i i i=1 As f is inf-compact and strictly convex, therefore the problem (SDP ) admits a unique optimal solution. We denote by x() or x the unique optimal solution of (SDP ) : 2.3. Behavior of the solution when  ! 0 In what follows, we will be interested by the behavior of the optimal value and the optimal solution x() of the problem (SDP ) : For that, let us intro- duce the function f : R  R ! ]1; +1] ; defined by f (x) if  > 0; f (x; ) = b x if  = 0; x 2 X; +1 if not. It is easy to verify that the function f is convex and lower semi-continuous on R  R; see for instance R.T. Rockafellar ([10]). Let us, then, define m : R ! ]1; +1] by m() = inf [f (x; ) : x 2 R ] : This function is convex. Furthermore, we have m(0) = (1) and m() is the optimal value of (SDP ) for  > 0: Logarithmic barrier method via minorant function... 101 It is clear that for  > 0, we get m() = f (x()) = f (x(); ) ; and 0 = rf (x()) = r f (x(); ) = b b(x ): We are now interested in the differentiability of the functions m and x over ]0; +1[. Proposition 2 ([4]). The functions m and x are continuously differen- tiable over ]0; +1[. For any  > 0; we have (x )x () b(x ) = 0; m () = n + n ln() ln det(B(x )): Besides m(0)  b x()  m(0) + n: Denote by S the set of the optimal solutions of the problem (1). We, already, know that this set is non-empty compact convex. The distance of the point x to S is defined by d(x; S ) = inf [kx zk : z 2 S ]: D D The following result concerns the behavior of x and m() when  ! 0: Theorem 2 ([4]). When  ! 0; d(x; S ) ! 0 and m() ! m(0): Remark 1. We know that if one of the problems (1) and (2) has an optimal solution, and the values of their objective functions are equal and finite, the other problem has an optimal solution. 102 Assma Leulmi 3. Newton descent direction and line search With the presence of the barrier function, the problem (SDP ) can be considered as without constraints. So, one can solve it by a classical slope method. As f takes the +1 value on the boundary of X; then the iterates x are in X: Thus, the new proposed method is an interior point method. Let x 2 X be the actual iterate. As a slope direction in x; let us take the Newton’s direction d as a solution of the linear system r f (x)d = rf (x): By virtue of Theorem 1, the precedent linear system is equivalent to the system (3) (x)d = b(x) b; where b(x) and (x) are defined in section 2.1. Since the matrix (x) being symmetrical, positive definite, the linear sys- tem (3) can be effectively solved through the Cholesky decomposition. Evidently, one can admit rf (x) 6= 0 (otherwise, the optimum is reached). It follows that d 6= 0. With calculated direction d, we search  > 0 such that it induces a scharp decrease of f on the semi-line x + d; > 0, and conserving positive definiteness of the matrix B(x + d). Then, the next iterate will be taken equal to x + d. Thus, we can consider the function G( ) = [f (x + d) f (x)]; x + d 2 X; G( ) = b d ln det(B(x + d)) + ln det(B(x)): Since r [f (x)]d = rf (x); we have T 2 T T T d r f (x)d = d rf (x) = d b(x) d b: To simplify the notations, we consider m m X X B = B(x) = x A C and H = d A : i i i i i=1 i=1 Since B is symmetrical and positive definite, there is a lower triangular matrix L so that B = LL . Logarithmic barrier method via minorant function... 103 1 1 T Next, let us put E = L H (L ) ; since d 6= 0; the assumption (A1) implies that H 6= 0 and then E 6= 0: With this notation, for any > 0; such that I + E is positive definite, (4) G( ) = [trace(E) trace(E )] ln det(I + E): Denote by  the eigenvalues of the symmetric matrix E; then (5) G( ) = [ (  ) ln(1 +  )]; 2 [0; b[ ; i i i=1 with (6) b = sup[ : 1 +  > 0 for all i ] = sup[ : x + d 2 X ]: Observe that b = +1 if E is positive semidefinite, and 0 < b < 1 otherwise. It is clear that G is convex on [0; b[ ; G(0) = 0 and 2 00 0 0 <  = G (0) = G (0): Besides, G( ) ! +1 when ! b. It follows that, it exists a unique point such that, G ( ) = 0, where G reaches its minimum in this point. opt opt Unfortunately, it does not exist an explicit formula that gives , and opt the resolution of the equation G ( ) = 0 through iterative methods needs opt at each iteration the computation of G and G . These computations are too expensive because the expression of G in (4) contains the determinant which is difficult to calculate and the expression of (5) necessitates the knowledge of the eigenvalues of E: It is a numerical problem of large size. These difficulties conduct us to look for other new alternatives approaches. Once E is calculated, it is easy to calculate the following quantities X X X X 2 2 2 trace(E) = e =  and trace(E ) = e =  : ii i ij i i i i;j i The following result is caused by H. Wolkowicz et al. ([14]), see also J.P. Crouzeix et al. ([5]) for additional results. Proposition 3 ([14]). x   n 1  min x  x  ; x i n 1 x  + p  max x  x  +  n 1: i x n 1 104 Assma Leulmi Let us recall that, B. Merikhi et al. ([4]) proposed some useful inequalities related to the maximum and to the minimum of x > 0 for any i = 1; : : : ; n; (7) n ln(x   n 1)  A  ln(x )  B  n ln(x ); x i i=1 with A = (n 1) ln(x  + ) + ln(x   n 1); n 1 B = ln(x  +  n 1) + (n 1) ln(x  p ); n 1 where x  and  are respectively, the mean and the standard deviation of a statistical series fx ; x ; : : : ; x g of n real numbers. These quantities are 1 2 n defined as follows n n n X X X 1 1 1 2 2 2 2 x  = x and  = x x  = (x x ) : i i x i n n n i=1 i=1 i=1 The computation of the displacement step by classical line search methods is undesirable and in general impossible. Based on this proposition, we give in the following section new notions of non-expensive minorant functions for G; that offer some variable displacement steps to every iteration with a simple technique. Thanks to definite positivity results in linear algebra, we propose three different alternative minorant functions that offer some variable displacement steps to every iteration. The efficacy of one minorant function compared to the other can be ex- pressed by numerical tests that we will present at the end of this work. 4. Computation of the displacement step Let us go back to the equations (5) and (6) and denote by  and respectively, the mean and the standard deviation of  , respectively, and by kk the Euclidean norm of the vector . So 2 2 2 00 0 kk = n( +  ) = G (0) = G (0) and (8) G( ) = n kk ln(1 +  ): i=1 Logarithmic barrier method via minorant function... 105 Our purpose is to search  2 ]0; b[ that induces a significant decrease of the convex function G. It is to be noted that the best choice with  = , opt where G ( ) = 0, causes numerical complications. However, one can find opt approximately ; but this procedure necessitates, also, too many computations of G and G : However, if we use a line search, it becomes convenient to know the upper bound b of the G domain, which is numerically difficult to solve. Consequently, we will take the upper bound of b given in Proposition 3. b = sup[ : 1 + > 0] with i = 1; 2; i i and =  p ; = kk : 1 2 n 1 4.1. First minorant function This strategy consists to minimize a minorant approximation G of G in- stead to minimize G over [0; b[ : To be efficient, this minorant approximation needs to be simple and sufficiently near G: In our case, it requires 2 00 0 0 = G(0); kk = G (0) = G (0): So, for any x = 1 +  ; i = 1; : : : ; m; we have x  = 1 +  and  =  : i i x By applying inequalities (7), we get ln(1 +  )  (n 1) ln(1 + ) + ln(1 + ); i 1 1 i=1 with =  and =  +  n 1: Then 1 1 n1 2 2 ln(1 +  )kk  (n 1) ln(1 + ) ln(1 + )kk ; i 1 1 i=1 and 2 2 n kk ln(1+ )  n kk (n1) ln(1+ )ln(1+ ): i 1 1 i=1 The logarithms are well defined when < b with if < 0; b = +1 if not: 106 Assma Leulmi Then, we deduce the following minorant function G ( ) =  (n 1) ln(1 + ) ln(1 + ); 1 1 1 1 for any 2 [0; b [ ; with  = nkk . 1 1 00 0 2 G verifies the following proprieties G (0) = G (0) = trace(E ) and 1 1 G (0) = 0; besides G ( ) < 0; 8 2 [0; b [. 1 1 1 G is convex and admits a unique minimum over [0; b [; which can be 1 1 obtained by resolving the equation G ( ) = 0, then we get = b b c; kk 1 n 1 1 where b = and c = . 1 1 1 1 1 1 4.2. Second minorant function We can also think of another more simple functions than G ; that involve only one logarithm. For this, we consider functions of the following type G( ) =   ln(1 + ); 2 [0; [ : The logarithm is well defined over 2 [0; [, with  = sup[ : 1 + > 0]. Then, we have the following minorat function G ( ) =  ln(1 + ); 2 2 2 1 2 2 kk 2 kk G ( ) = kk ln(1 + ); 2 2 1 for any 2 [0; b [, where =  ;  = kk and we take 1 1 2 2 1 n1 kk = which fulfils the following condition kk = =  : 2 2 1 2 00 0 2 G verifies the following proprieties G (0) = G (0) = trace(E ) and 2 2 G (0) = 0; besides G ( ) < 0; 8 2 [0; b [. 2 2 1 G is convex and admits a unique minimum over [0; b [; which can be 2 1 obtained by resolving the equation G ( ) = 0; then we get = : 2 1 Logarithmic barrier method via minorant function... 107 4.3. Third minorant function Another minorant function simpler than G can be extracted from the following known inequality n n X X kk  ln(1 + kk) + ln(1 +  )  0: i i i=1 i=1 Then, we obtain the following minorat function G ( ) =  ln(1 + ); 2 [0; b [ ; 3 3 2 2 with b = ;  = kk(kk 1) and = kk: 2 3 2 00 0 2 G verifies the following proprieties G (0) = G (0) = trace(E ) and 3 3 G (0) = 0; besides G ( ) < 0; 8 2 [0; b [ : 3 3 2 G is convex and admits a unique minimum over [0; b [; which can be 3 2 obtained by resolving the equation G ( ) = 0, then we get = (kk 1) : Proposition 4. G ; i = 1; : : : ; 3, is strictly convex over 2 [0; [ ; with = min ( b; b ; b ) : So we have 1 2 G ( )  G ( )  G ( )  G( ); 8 2 [0; [ : 3 2 1 Proof. The first inequality is obvious. The inequality G( )  G ( ) is a direct consequence of (7). Let’s consider g( ) = G ( )G ( ). Since = 2 1 1 2 and  , we have for any 2 [0; [ 1 1 2 2 2 2 2 (n 1) 00 2 1 1 1 1 g ( ) =   0: 2 2 2 2 (1 + ) (1 + ) (1 + ) (1 + ) 1 1 1 1 Since g(0) = g (0) = 0, it becomes g( )  0 for any > 0. Then, let’s put h( ) = G ( ) G ( ), so 3 2 2 2 0 00 2 1 h(0) = h (0) = 0 and h ( ) = : 2 2 (1 + ) (1 + ) 2 1 Since kk = and so = kk ; then 2 2 1 1 h ( ) = kk  0; 2 2 (1 + ) (1 + ) 2 1 because  . Therefore h( )  0 for any 2 [0; [. 1 2 108 Assma Leulmi Let us recall that the functions G reach their minimum at a unique point which is the root of G ( ) = 0. Thus, the three roots are explicitly calcu- lated, for i = 1; : : : ; 3. So, we have 1 n 1 1 kk = b b c with b = and c = ; 1 1 1 1 1 1 2 1 = ;  = (kk 1) : 2 3 2 1 Thus, the three values  ; i = 1; : : : ; 3 are explicitly computed, then, we take ;  and  belonging to the interval [0; "[ and G ( ) < 0; with " > 0 1 2 3 being a fixed precision. Remark 2. The computation of  is performed by a dichotomous proce- dure, in the cases where  2= (0; b "); and G ( ) > 0; as follows: 1. Put a = 0; b = b ". 2. While jb aj > 10 do 0 a+b a+b a+b if G ( ) < 0 then b = , else a = , so  = b. 2 2 2 This computation guarantees a better approximation of the minimizer of G ( ) while remaining in the domain of G. 5. Description of the algorithm In this section, we present the algorithm of our approach to obtain an optimal solution x  of the problem (1). Begin algorithm Initialization We have to decide for the strategy of the displacement step. " > 0 is a given precision,  > 0;  > 0 and  2 ]0; 1[ are fixed parameters. Start with x 2 X and k = 0: Iteration k k T 1. Take B = B(x ) = x A C and L such that B = LL : i=1 2. Compute k k 1 T k 1 > A (x ) = [L(x )] A [L (x )] ; i i < k k b(x ) = trace(A (x )); k k k b b (x ) = trace(A (x )A (x )); ij i j H = (x ): Logarithmic barrier method via minorant function... 109 3. Solve the linear system Hd = b(x) b: 1 1 T 2 4. Calculate E = L H (L ) ; trace(E) and trace(E ): k+1 k 5. Take the new iterate x = x + d; such that  is obtained by the use of the displacement step strategy of G ; i = 1; : : : ; 3: k k+1 6. If n > "; do x = x ;  =  and go to 1. T k+1 T k k k+1 7. If jb x b x j > n; do x = x and go to 1. 8. Take k = k + 1: k+1 9. Stop: x is an approximate solution of the problem (1). End algorithm We know that the optimal solution of (SDP ) is an approximation of the solution of problem (1). More  is closer to zero, more the approximation will be good. Unfortunately, when  approaches zero, the problem (SDP ) be- comes ill-conditioned. For this reason, we use at the beginning of the iteration the values of  that are not near to zero, and verify n < ". We can explain the interpretation of the update  as follows: if x() is an exact solution of (SDP ) , so b x() 2 [m (0) ; m (0) + n]. It is then not necessary to keep on T k+1 T k the calculus of the iterates when jb x b x j  n: The displacement step  will be determined by classical line search of Armijo-Goldstein-Price type or by one of the three following strategies St i; by minimizing the minorant functions G ; i = 1; : : : ; 3. In the next section, we present comparative numerical tests to prove the effectiveness of our approach over line search method. 6. Numerical tests The following examples are taken from the literature (see for instance [3, 4, 11]) and implemented in MATLAB R2013a on Pentium(R). We have taken " = 1:0e 006;  = 0:125 and two values of ;  = 1 or  = 2: In the table of results, (exp (m; n)) represents the size of the example, (Itrat) represents the number of iterations necessary to obtain an optimal solution, (Time) represents the time of computation in seconds (s), (LS) rep- resents the classical line search of Armijo–Goldstein method and (St i) repre- sents the strategy which uses the minorant function G ; with i = 1; : : : ; 3: Recall that the considered problem is min b x; x A C 2 S ; i i > i=1 x 2 R : 110 Assma Leulmi 6.1. Examples with fixed size In the following examples, diag(x) is the n n diagonal matrix with the components of x as the diagonal entries. Example 1. T T 5 8 8 5 1 1 1 2 C = diag ; A = I; b = ; and the matrices A ; k = 1; : : : ; 3; are defined as follows 1 if i = j = k or i = j = k + 1; 1 if i = k; j = k + 1 or i = k + 1; j = k; A [i; j] = 0 otherwise. We start with an initial point x = 1:5 1:5 1:5 1:5 : The optimal solution is x = 0 1:5 0 5 : The optimal value is b x = 11:5: Table 1. The optimal solution obtained with the differ- ent approaches is St 1 0 1:500011 0 5 St 2 0:000011 1:499992 0 4:999981 St 3 0:000061 1:499982 0 4:999871 LR 0:000003 1:499563 0:000011 4:999968 Example 2. C = diag 4 2 2 0 0 0 ; T T A = diag 1 1 1 1 0 0 ; A = diag 1 1 1 0 1 0 ; 1 2 T T 2 2 1 0 0 1 6 2 4 A = diag ; b = : We start with an initial point x = 1 1 2 : 0 2:41354 0:79323 The optimal solution is x = : The optimal value is b x = 8: Logarithmic barrier method via minorant function... 111 Table 2. The optimal solution obtained with the different approaches is St 1 0 2:413012 0:793232 St 2 0:000001 2:413542 0:794323 St 3 0:000015 2:426911 0:794995 LR 0:000003 2:413613 0:7943256 Example 3. T T 4 5 0 0 0 2 1 1 0 0 C = diag ; A = diag ; T T A = diag 1 2 0 1 0 ; A = diag 0 1 0 0 1 ; 2 3 b = 8 7 3 : 2 1 2 We start with an initial point x = : The optimal solution is x = 1 2 0 : The optimal value is b x = 22: Table 3. The optimal solution obtained with the different approaches is St 1 1 2:000001 0 St 2 0:999968 1:999952 0:000011 St 3 1:000015 2:000011 0:000112 LR 0:999863 2:000113 0:000321 Example 4. 1 1 1 1 1 0 C = ; A = ; A = ; 1 2 1 1 1 1 0 1 1 1 b = : We start with an initial point x = 0 3 : The optimal solution is x = 1 2 : The optimal value is b x = 2: 112 Assma Leulmi Table 4. The optimal solution ob- tained with the different approaches is St 1 0:999999 2 St 2 0:999985 1:999952 St 3 1:000155 2:000021 LR 0:998603 2:000113 Example 5. T T 1 1 0 1 1 0 C = diag ; A = diag ; T T A = diag 1 1 1 ; b = 0 1 : We start with an initial point x = 1 1 : 0:4 0 The optimal solution is x = : The optimal value is b x = 0: Table 5. The optimal solu- tion obtained with the differ- ent approaches is St 1 0:399993 0 St 2 0:399903 0 St 3 0:399855 0 LR 0:399603 0 Table 6. The obtained results exp (m; n) St 1 St 2 St 3 LS Itrat Time Itrat Time Itrat Time Itrat Time exp 1(4; 4) 3 0.012 3 0.014 4 0.19 5 0.25 exp 2(3; 6) 1 0.0016 1 0.0022 1 0.0025 7 0.36 exp 3(3; 5) 4 0.0014 4 0.0023 5 0.0032 6 0.36 exp 4(2; 2) 3 0.0001 4 0.0003 4 0.0006 3 0.068 exp 5(2; 3) 2 0.0001 2 0.0028 3 0.0041 3 0.087 Logarithmic barrier method via minorant function... 113 6.2. Example with variable size Example 6 (Example Cube). n = 2m; C is the n  n identity matrix, T m b = (2; : : : ; 2) 2 R ; a 2 R and the entries of the n  n matrix A ; k = 1; : : : ; m; are given by 1 if i = j = k or i = j = k + m; > 2 a if i = j = k + 1 or i = j = k + m + 1; A [i; j] = a if i = k; j = k + 1 or i = k + m; j = k + m + 1; a if i = k + 1; j = k or i = k + m + 1; j = k + m; 0 otherwise. Test 1: a = 0 and C = I: T m We know that the vector x = (1; : : : ; 1) 2 R is the optimal solution. 0 T m We start with an initial point x = (1:5; : : : ; 1:5) 2 R : The following table resumes the obtained results. Table 7. The obtained results in test 1 Size (m; n) St 1 St 2 St 3 LS Itrat Time Itrat Time Itrat Time Itrat Time (50; 100) 1 16 1 17 1 19 dvg (100; 200) 1 89 1 104 1 112 dvg (200; 400) 1 473 1 545 1 554 dvg dvg means that the algorithm does not terminate within a finite time. Commentary: The results of these tests show that LS do not compete with St 1, St 2 and St 3. In the next experiments, we continue only with St 1, St 2 and St 3. Test 2: a = 2; C = 2I: 0 T m We start with an initial point x = (0; 0; ::; 0) 2 R : The optimal solution is x = (1; : : : ; 1) 2 R : The following table resumes the obtained results. Table 8. The obtained results in test 2 Size (m; n) St 1 St 2 St 3 Itrat Time Itrat Time Itrat Time (50; 100) 1 16 1 18 2 24 (100; 200) 1 72 1 114 2 325 (200; 400) 1 258 1 435 2 623 114 Assma Leulmi Test 3: a = 5 and C = 2I: 0 T m We start with an initial point x = (0; 0; : : : ; 0) 2 R : The optimal solution is x = (1; : : : ; 1) 2 R : The following table resumes the obtained results. Table 9. The obtained results in test 3 Size (m; n) St 1 St 2 St 3 Itrat Time Itrat Time Itrat Time (50; 100) 1 19 1 19 2 21 (100; 200) 1 91 1 102 2 117 (200; 400) 1 235 1 490 2 512 Example 7. C = I; A ; k = 1; : : : ; m; are defined as 1 if i = j = k; A [i; j] = 1 if i = j and i = k + m; 0 otherwise; and b = 2; i = 1; : : : ; m: We start with an initial point x = 2; i = 1; : : : ; m: The optimal solution is x = (1; : : : ;1) 2 R : The optimal value is b x = n = 2m: The following table resumes the obtained results. Table 10. The obtained results Size (m; n) St 1 St 2 St 3 Itrat Time Itrat Time Itrat Time (50; 100) 1 17 1 17 2 25 (100; 200) 1 35 1 89 2 354 (200; 400) 1 223 1 234 2 565 Commentary: We notice that the three strategies converge to the optimal solution. These tests show, clearly, the impact of our three strategies offer an optimal solution of (1) and (2) in a reasonable time and with a small number of iterations. st We also note that the 1 strategy is the best. The obtained comparative numerical results favor this last strategy moreover, it requires a computing time largely low vis-a-vis the other two strategies. This seems to be quite expected, because theoretically the strategy St 1 uses the function G that is the closest (best approximation) of the function G: Logarithmic barrier method via minorant function... 115 7. Conclusion In this paper, we have proposed a new logarithmic barrier approach for solving semidefinite programming problem (SDP), since problem (SDP ) is strictly convex, the KKT conditions are necessary and sufficient. For this, we use Newton’s method that allows us to calculate a good descent direction and determine a new iterate, better than the current iterate. To compute the displacement step, several methods have been proposed by scientists and researchers. Including, line search methods, which are very expensive and unworkable. To overcome this problem, we have proposed in this work a new approach, based on the notion of minorant functions, which allows us to determine the displacement step by a simple, easy and much less costly technique. To improve our contribution, we presented numerical simulations to illus- trate the effectiveness of our approach and the convergence of strategies to the optimal solution of the problem (1). These simulations confirm that the first and second strategies are better in terms of number of iterations, computation time and then reduce the computational cost. Therefore, this work has a very interesting theoretical and numerical value. The digital aspect can be pushed to a level of performance very appreciable for the practice. The technique of minorant functions to determine the displacement step in the direction of descent is a very reliable alternative that is confirmed as the technique of choice for (SDP) and other classes of optimization problems: Quadratic Programming (QP) and Non-Linear Programming (NLP).... Acknowledgements. The author is very grateful and would like to thank the Editor-in-Chief and the anonymous referee for their suggestions and help- ful comments which significantly improved the presentation of this paper. References [1] F. Alizadeh, J.-P.A. Haeberly, and M.L. Overton, Primal-dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results, SIAM J. Optim. 8 (1998), no. 3, 746–768. [2] D. Benterki, J.-P. Crouzeix, and B. Merikhi, A numerical implementation of an interior point method for semidefinite programming, Pesqui. Oper. 23 (2003), no. 1, 49–59. [3] D. Benterki, J.-P. Crouzeix, and B. Merikhi, A numerical feasible interior point method for linear semidefinite programs, RAIRO Oper. Res. 41 (2007), no. 1, 49–59. [4] J.-P. Crouzeix and B. Merikhi, A logarithm barrier method for semi-definite program- ming, RAIRO Oper. Res. 42 (2008), no. 2, 123–139. 116 Assma Leulmi [5] J.-P. Crouzeix and A. Seeger, New bounds for the extreme values of a finite sample of real numbers, J. Math. Anal. Appl. 197 (1996), no. 2, 411–426. [6] J. Ji, F.A. Potra, and R. Sheng, On the local convergence of a predictor-corrector method for semidefinite programming, SIAM J. Optim. 10 (1999), no. 1, 195–210. [7] B. Merikhi, Extension de Quelques Méthodes de Points Intérieurs pour la Program- mation Semi-Définie, Thèse de Doctorat, Département de Mathématiques, Université Ferhat Abbas, Sétif, 2006. [8] R.D.C. Monteiro, Primal-dual path-following algorithms for semidefinite programming, SIAM J. Optim. 7 (1997), no. 3, 663–678. [9] Y.E. Nesterov and A.S. Nemirovskii, Optimization Over Positive Semidefinite Matri- ces: Mathematical Background and User’s Manual, Technical report, Central Economic & Mathematical Institute, USSR Academy of Science, Moscow, 1990. [10] R.T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, N.J., 1970. [11] K. Samia and D. Benterki, A relaxed logarithmic barrier method for semidefinite pro- gramming, RAIRO Oper. Res. 49 (2015), no. 3, 555–568. [12] K.-C. Toh, Some new search directions for primal-dual interior point methods in semi- definite programming, SIAM J. Optim. 11 (2000), no. 1, 223–242. [13] I. Touil, D. Benterki, and A. Yassine, A feasible primal-dual interior point method for linear semidefinite programming, J. Comput. Appl. Math. 312 (2017), 216–230. [14] H. Wolkowicz and G.P.H. Styan, Bounds for eigenvalues using traces, Linear Algebra Appl. 29 (1980), 471–506. Department of Mathematics Ferhat Abbas University of Setif-1 Algeria e-mail: smaleulm@yahoo.fr http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Annales Mathematicae Silesianae de Gruyter

Logarithmic Barrier Method Via Minorant Function for Linear Semidefinite Programming

Leulmi, Assma
Annales Mathematicae Silesianae , Volume 37 (1): 22 – Mar 1, 2023
Download PDF
22 pages

Loading next page...
 
/lp/de-gruyter/logarithmic-barrier-method-via-minorant-function-for-linear-arttDAw07F
Publisher
de Gruyter
Copyright
© 2023 Assma Leulmi, published by Sciendo
eISSN
0860-2107
DOI
10.2478/amsil-2022-0021
Publisher site
See Article on Publisher Site

Abstract

Annales Mathematicae Silesianae 37 (2023), no. 1, 95–116 DOI: 10.2478/amsil-2022-0021 LOGARITHMIC BARRIER METHOD VIA MINORANT FUNCTION FOR LINEAR SEMIDEFINITE PROGRAMMING Assma Leulmi Abstract. We propose in this study, a new logarithmic barrier approach to solve linear semidefinite programming problem. We are interested in computa- tion of the direction by Newton’s method and of the displacement step using minorant functions instead of line search methods in order to reduce the com- putation cost. Our new approach is even more beneficial than classical line search meth- ods. This purpose is confirmed by some numerical simulations showing the effectiveness of the algorithm developed in this work, which are presented in the last section of this paper. 1. Introduction Semidefinite programming (SDP) problem is an optimization model of a linear function of a symmetric matrix subject to linear equality constraints and the additional condition that the matrix be positive semidefinite. SDPs include as special cases linear programmings (LP), when all the symmetric matrices involved are diagonal. General SDP is perhaps one of the most pow- erful forms of convex optimization. Semidefinite programming permits to solve numerous problems, as non-linear programming (NLP) problems, quadratic programming (QP) problems,.... Received: 21.02.2021. Accepted: 27.12.2022. Published online: 07.02.2023. (2020) Mathematics Subject Classification: 90C22, 90C51. Key words and phrases: linear semidefinite programming, barrier methods, line search. c 2022 The Author(s). This is an Open Access article distributed under the terms of the Creative Commons Attribution License CC BY (http://creativecommons.org/licenses/by/4.0/). 96 Assma Leulmi Many algorithms have been suggested to resolve linear SDP problem such that, Interior-point methods (IPMs) for SDP have been pioneered by Nesterov and Nemirovskii ([9]) as well as Alizadeh et al. ([1]). Several methods have been proposed to solve SDP such that, projective IPMs and their alternatives ([12, 7]), central trajectory methods ([13]), loga- rithmic barrier methods ([4]). Our work is based on the latter type of IPMs, the obstacle for establishing an iteration is the determination and computation of the displacement step. Unfortunately, the computation of the displacement step, especially when us- ing line search methods, is costly and even more difficult in the case of semi- definite problems ([4]). In this paper, we are interested in solving SDP by IPMs. The idea of this method consists to solve SDP with a new logarithmic barrier approach. Then, we use Newton’s method to treat the associated perturbed equations to obtain a descent direction. We propose an alternative ways to determine the displacement step along the direction which are more efficient than classical line searches. We consider the following SDP problem min b x; x A C 2 S ; (1) i i i=1 x 2 R : Here S designs the cone of the symmetrical semidefinite positive nn matrix, matrices C, A ; with i = 1; : : : ; m; are the given symmetrical matrices and b 2 R : The problem (1) is the dual of the following semidefinite problem maxhC; Yi; hA ; Yi = b ; 8i = 1; : : : ; m; (2) i i Y 2 S : We denote by hC; Yi the trace of the matrix (C Y ), and recall that h;i corresponds to an inner product on the space of n n matrices. Their feasible sets involving a non polyhedral convex cone of positive semi- definite matrices are called linear semidefinite programs. A priory, one of the advantages of the problem (1) with respect to its dual problem (2) is that variable of the objective function is a vector instead to be a matrix in the type problem (2). Furthermore, under certain convenient hypothesis, the resolution of the problem (1) is equivalent to the problem (2) in the sense that the optimal solution of one of the two problems can be Logarithmic barrier method via minorant function... 97 reduced directly from the other through the application of the theorem of the slackness complementary, see for instance [1, 6, 8]. In all which follows, we denote by m + 1. X = fx 2 R : x A C 2 S g; the set of feasible solutions of (1). i i i=1 m + 2. X = fx 2 R : x A C 2 int(S )g; the set of strictly feasible i i i=1 n solutions of (1). 3. F = fY 2 S : hA ; Yi = b ; 8i = 1; : : : ; mg; the set of feasible solutions i i of (2). 4. F = fY 2 F : Y 2 int(S )g; the set of strictly feasible solutions of (2). int(S ) is the set of the symmetrical definite positive n n matrices. The problem (1) is approximated by the following problem (SDP ) min f (x); (SDP ) x 2 R ; with the penalty parameter  > 0 and f : R ! ]1; +1] is the barrier function defined by b x + n ln   ln[det( x A C )] if x 2 X; i i f (x) = i=1 +1 if not. The problem (SDP ) can be solved via a classical Newton descent method. The difficulty with the line search is the presence of the determinant in the definition of the logarithmic barrier function which leads to a very high cost in the classical procedures of exact or approximate line search. In our approach, instead of minimizing f ; along the descent direction at a current point x; we propose minorants functions G for which the optimal solution of the displacement step is obtained explicitly. Let us minimize the function G such that [f (x + d) f (x)] = G( )  G( ); 8 > 0; 0 0 with G(0) = G(0) = 0; G (0) = G (0) < 0: The best quality of the approximations G of G is ensured by the condition 00 00 G (0) = G (0): The idea of this new approach is to introduce one original process to calculate the displacement step based on minorants functions. Then, we obtain an explicit approximation which leads to reducing the objective, adding to this, it is economical and robust, contrary to the traditional methods of line search. 98 Assma Leulmi Backdrop and brief information in linear semidefinite programming. Let us state the following necessary assumptions (A1) The system of equations hA ; Yi = b ; i = 1; : : : ; m is of rank m: i i b b (A2) The sets X and F are not empty. We know that (see [1, 2]) 1. The sets of optimal solutions of problems (2) and (1) are non empty convex and compact. 2. If x  is an optimal solution of (1), then Y is an optimal solution of (2) if and only if Y 2 F and x  A C Y = 0: i i i=1 3. If Y is an optimal solution of (2); then x  is an optimal solution of (1) if and only if x  2 X and x  A C Y = 0: i i i=1 According to the assumptions (A1) and (A2), the solution of problem (1) permits to give the one of problem (2) and vice-versa. We study in the next section, existence and uniqueness of optimal solution of the problem (SDP ) and its convergence to problem (2), in particular the behaviour of its optimal value and its optimal solutions when  ! 0: The k+1 k solution of this problem is of descent type, defined by x = x + d , k k where d is the descent direction and is the displacement step. k k Then, we show in section 3, how to compute the Newton descent direc- tion d. In section 4, we present new three different approximations of G; to compute the displacement step. These approximations are deduced from in- equalities shown in section 4. In section 5, we describe the obtained algorithm. In section 6, we present numerical tests with commentaries on some different examples to illustrate the effectiveness of the three proposed approaches and we compare them with the standard line search method. The paper is finished by conclusions in the last section. The main advantage of (SDP ) resides in the strict convexity of its ob- jective function and the convexity of its feasible domain. Consequently, the conditions of optimality are necessary and sufficient. This, fosters theoretical and numerical studies of the problem. Before this, it is necessary to show that (SDP ) has at least an optimal solution. Logarithmic barrier method via minorant function... 99 2. Existence and uniqueness of optimal solution of problem (SDP ) and its convergence to problem (1) 2.1. Fundamental properties of f For x 2 X, let us introduce the symmetrical definite positive matrix B(x) of rank m; and the lower triangular matrix L(x); such that B(x) = x A C = L(x)L (x); i i i=1 and let us define, for i; j = 1; : : : ; m 1 T 1 A (x) = [L(x)] A [L (x)] ; i i b (x) = trace(A (x)) = trace(A B (x)); i i i 1 1 b b (x) = trace(B (x)A B (x)A ) = trace(A (x)A (x)): ij i j i j Thus, b(x) = (b (x)) is a vector of R and (x) = ( (x)) i ij i=1;:::;m i;j=1;:::;m is a symmetrical matrix of rank m. The previous notation will be used in the expressions of the gradient and the Hessian H of f : To show that problem (SDP ) has a solution, it is sufficient to show that f is inf-compact. Theorem 1 ([4]). The function f is twice continuously differentiable on b b X: Actually, for all x 2 X we have: (a) rf (x) = b b(x): (b) H = r f (x) = (x): (c) The matrix (x) is definite positive. Since f is strictly convex, (SDP ) has at most one optimal solution. 2.2. Problem (SDP ) has one unique optimal solution Firstly, we start with the following definition Definition 1. Let f be a function defined from R to R [ f1g; f is called inf-compact if for all  > 0; the set S (f ) = fx 2 R : f (x)  g is compact, which comes in particular to say that its cone of recession is reduced to zero. 100 Assma Leulmi As the function f takes the value +1 on the boundary of X and is differentiable on X; then it is lower semi-continuous. In order to prove that (SDP ) has one optimal solution, it suffices to prove that recession cone of f S (f ) = fd 2 R : (f ) (d)  0g ; 0   1 is reduced to zero i.e., d = 0 if (f ) (d)  0; where (f ) is defined for x 2 X as f (x + d) f (x) (f ) (d) = lim : !+1 This leads to the following proposition. Proposition 1 ([4]). If b d  0 and d A 2 X then d = 0: i i i=1 As f is inf-compact and strictly convex, therefore the problem (SDP ) admits a unique optimal solution. We denote by x() or x the unique optimal solution of (SDP ) : 2.3. Behavior of the solution when  ! 0 In what follows, we will be interested by the behavior of the optimal value and the optimal solution x() of the problem (SDP ) : For that, let us intro- duce the function f : R  R ! ]1; +1] ; defined by f (x) if  > 0; f (x; ) = b x if  = 0; x 2 X; +1 if not. It is easy to verify that the function f is convex and lower semi-continuous on R  R; see for instance R.T. Rockafellar ([10]). Let us, then, define m : R ! ]1; +1] by m() = inf [f (x; ) : x 2 R ] : This function is convex. Furthermore, we have m(0) = (1) and m() is the optimal value of (SDP ) for  > 0: Logarithmic barrier method via minorant function... 101 It is clear that for  > 0, we get m() = f (x()) = f (x(); ) ; and 0 = rf (x()) = r f (x(); ) = b b(x ): We are now interested in the differentiability of the functions m and x over ]0; +1[. Proposition 2 ([4]). The functions m and x are continuously differen- tiable over ]0; +1[. For any  > 0; we have (x )x () b(x ) = 0; m () = n + n ln() ln det(B(x )): Besides m(0)  b x()  m(0) + n: Denote by S the set of the optimal solutions of the problem (1). We, already, know that this set is non-empty compact convex. The distance of the point x to S is defined by d(x; S ) = inf [kx zk : z 2 S ]: D D The following result concerns the behavior of x and m() when  ! 0: Theorem 2 ([4]). When  ! 0; d(x; S ) ! 0 and m() ! m(0): Remark 1. We know that if one of the problems (1) and (2) has an optimal solution, and the values of their objective functions are equal and finite, the other problem has an optimal solution. 102 Assma Leulmi 3. Newton descent direction and line search With the presence of the barrier function, the problem (SDP ) can be considered as without constraints. So, one can solve it by a classical slope method. As f takes the +1 value on the boundary of X; then the iterates x are in X: Thus, the new proposed method is an interior point method. Let x 2 X be the actual iterate. As a slope direction in x; let us take the Newton’s direction d as a solution of the linear system r f (x)d = rf (x): By virtue of Theorem 1, the precedent linear system is equivalent to the system (3) (x)d = b(x) b; where b(x) and (x) are defined in section 2.1. Since the matrix (x) being symmetrical, positive definite, the linear sys- tem (3) can be effectively solved through the Cholesky decomposition. Evidently, one can admit rf (x) 6= 0 (otherwise, the optimum is reached). It follows that d 6= 0. With calculated direction d, we search  > 0 such that it induces a scharp decrease of f on the semi-line x + d; > 0, and conserving positive definiteness of the matrix B(x + d). Then, the next iterate will be taken equal to x + d. Thus, we can consider the function G( ) = [f (x + d) f (x)]; x + d 2 X; G( ) = b d ln det(B(x + d)) + ln det(B(x)): Since r [f (x)]d = rf (x); we have T 2 T T T d r f (x)d = d rf (x) = d b(x) d b: To simplify the notations, we consider m m X X B = B(x) = x A C and H = d A : i i i i i=1 i=1 Since B is symmetrical and positive definite, there is a lower triangular matrix L so that B = LL . Logarithmic barrier method via minorant function... 103 1 1 T Next, let us put E = L H (L ) ; since d 6= 0; the assumption (A1) implies that H 6= 0 and then E 6= 0: With this notation, for any > 0; such that I + E is positive definite, (4) G( ) = [trace(E) trace(E )] ln det(I + E): Denote by  the eigenvalues of the symmetric matrix E; then (5) G( ) = [ (  ) ln(1 +  )]; 2 [0; b[ ; i i i=1 with (6) b = sup[ : 1 +  > 0 for all i ] = sup[ : x + d 2 X ]: Observe that b = +1 if E is positive semidefinite, and 0 < b < 1 otherwise. It is clear that G is convex on [0; b[ ; G(0) = 0 and 2 00 0 0 <  = G (0) = G (0): Besides, G( ) ! +1 when ! b. It follows that, it exists a unique point such that, G ( ) = 0, where G reaches its minimum in this point. opt opt Unfortunately, it does not exist an explicit formula that gives , and opt the resolution of the equation G ( ) = 0 through iterative methods needs opt at each iteration the computation of G and G . These computations are too expensive because the expression of G in (4) contains the determinant which is difficult to calculate and the expression of (5) necessitates the knowledge of the eigenvalues of E: It is a numerical problem of large size. These difficulties conduct us to look for other new alternatives approaches. Once E is calculated, it is easy to calculate the following quantities X X X X 2 2 2 trace(E) = e =  and trace(E ) = e =  : ii i ij i i i i;j i The following result is caused by H. Wolkowicz et al. ([14]), see also J.P. Crouzeix et al. ([5]) for additional results. Proposition 3 ([14]). x   n 1  min x  x  ; x i n 1 x  + p  max x  x  +  n 1: i x n 1 104 Assma Leulmi Let us recall that, B. Merikhi et al. ([4]) proposed some useful inequalities related to the maximum and to the minimum of x > 0 for any i = 1; : : : ; n; (7) n ln(x   n 1)  A  ln(x )  B  n ln(x ); x i i=1 with A = (n 1) ln(x  + ) + ln(x   n 1); n 1 B = ln(x  +  n 1) + (n 1) ln(x  p ); n 1 where x  and  are respectively, the mean and the standard deviation of a statistical series fx ; x ; : : : ; x g of n real numbers. These quantities are 1 2 n defined as follows n n n X X X 1 1 1 2 2 2 2 x  = x and  = x x  = (x x ) : i i x i n n n i=1 i=1 i=1 The computation of the displacement step by classical line search methods is undesirable and in general impossible. Based on this proposition, we give in the following section new notions of non-expensive minorant functions for G; that offer some variable displacement steps to every iteration with a simple technique. Thanks to definite positivity results in linear algebra, we propose three different alternative minorant functions that offer some variable displacement steps to every iteration. The efficacy of one minorant function compared to the other can be ex- pressed by numerical tests that we will present at the end of this work. 4. Computation of the displacement step Let us go back to the equations (5) and (6) and denote by  and respectively, the mean and the standard deviation of  , respectively, and by kk the Euclidean norm of the vector . So 2 2 2 00 0 kk = n( +  ) = G (0) = G (0) and (8) G( ) = n kk ln(1 +  ): i=1 Logarithmic barrier method via minorant function... 105 Our purpose is to search  2 ]0; b[ that induces a significant decrease of the convex function G. It is to be noted that the best choice with  = , opt where G ( ) = 0, causes numerical complications. However, one can find opt approximately ; but this procedure necessitates, also, too many computations of G and G : However, if we use a line search, it becomes convenient to know the upper bound b of the G domain, which is numerically difficult to solve. Consequently, we will take the upper bound of b given in Proposition 3. b = sup[ : 1 + > 0] with i = 1; 2; i i and =  p ; = kk : 1 2 n 1 4.1. First minorant function This strategy consists to minimize a minorant approximation G of G in- stead to minimize G over [0; b[ : To be efficient, this minorant approximation needs to be simple and sufficiently near G: In our case, it requires 2 00 0 0 = G(0); kk = G (0) = G (0): So, for any x = 1 +  ; i = 1; : : : ; m; we have x  = 1 +  and  =  : i i x By applying inequalities (7), we get ln(1 +  )  (n 1) ln(1 + ) + ln(1 + ); i 1 1 i=1 with =  and =  +  n 1: Then 1 1 n1 2 2 ln(1 +  )kk  (n 1) ln(1 + ) ln(1 + )kk ; i 1 1 i=1 and 2 2 n kk ln(1+ )  n kk (n1) ln(1+ )ln(1+ ): i 1 1 i=1 The logarithms are well defined when < b with if < 0; b = +1 if not: 106 Assma Leulmi Then, we deduce the following minorant function G ( ) =  (n 1) ln(1 + ) ln(1 + ); 1 1 1 1 for any 2 [0; b [ ; with  = nkk . 1 1 00 0 2 G verifies the following proprieties G (0) = G (0) = trace(E ) and 1 1 G (0) = 0; besides G ( ) < 0; 8 2 [0; b [. 1 1 1 G is convex and admits a unique minimum over [0; b [; which can be 1 1 obtained by resolving the equation G ( ) = 0, then we get = b b c; kk 1 n 1 1 where b = and c = . 1 1 1 1 1 1 4.2. Second minorant function We can also think of another more simple functions than G ; that involve only one logarithm. For this, we consider functions of the following type G( ) =   ln(1 + ); 2 [0; [ : The logarithm is well defined over 2 [0; [, with  = sup[ : 1 + > 0]. Then, we have the following minorat function G ( ) =  ln(1 + ); 2 2 2 1 2 2 kk 2 kk G ( ) = kk ln(1 + ); 2 2 1 for any 2 [0; b [, where =  ;  = kk and we take 1 1 2 2 1 n1 kk = which fulfils the following condition kk = =  : 2 2 1 2 00 0 2 G verifies the following proprieties G (0) = G (0) = trace(E ) and 2 2 G (0) = 0; besides G ( ) < 0; 8 2 [0; b [. 2 2 1 G is convex and admits a unique minimum over [0; b [; which can be 2 1 obtained by resolving the equation G ( ) = 0; then we get = : 2 1 Logarithmic barrier method via minorant function... 107 4.3. Third minorant function Another minorant function simpler than G can be extracted from the following known inequality n n X X kk  ln(1 + kk) + ln(1 +  )  0: i i i=1 i=1 Then, we obtain the following minorat function G ( ) =  ln(1 + ); 2 [0; b [ ; 3 3 2 2 with b = ;  = kk(kk 1) and = kk: 2 3 2 00 0 2 G verifies the following proprieties G (0) = G (0) = trace(E ) and 3 3 G (0) = 0; besides G ( ) < 0; 8 2 [0; b [ : 3 3 2 G is convex and admits a unique minimum over [0; b [; which can be 3 2 obtained by resolving the equation G ( ) = 0, then we get = (kk 1) : Proposition 4. G ; i = 1; : : : ; 3, is strictly convex over 2 [0; [ ; with = min ( b; b ; b ) : So we have 1 2 G ( )  G ( )  G ( )  G( ); 8 2 [0; [ : 3 2 1 Proof. The first inequality is obvious. The inequality G( )  G ( ) is a direct consequence of (7). Let’s consider g( ) = G ( )G ( ). Since = 2 1 1 2 and  , we have for any 2 [0; [ 1 1 2 2 2 2 2 (n 1) 00 2 1 1 1 1 g ( ) =   0: 2 2 2 2 (1 + ) (1 + ) (1 + ) (1 + ) 1 1 1 1 Since g(0) = g (0) = 0, it becomes g( )  0 for any > 0. Then, let’s put h( ) = G ( ) G ( ), so 3 2 2 2 0 00 2 1 h(0) = h (0) = 0 and h ( ) = : 2 2 (1 + ) (1 + ) 2 1 Since kk = and so = kk ; then 2 2 1 1 h ( ) = kk  0; 2 2 (1 + ) (1 + ) 2 1 because  . Therefore h( )  0 for any 2 [0; [. 1 2 108 Assma Leulmi Let us recall that the functions G reach their minimum at a unique point which is the root of G ( ) = 0. Thus, the three roots are explicitly calcu- lated, for i = 1; : : : ; 3. So, we have 1 n 1 1 kk = b b c with b = and c = ; 1 1 1 1 1 1 2 1 = ;  = (kk 1) : 2 3 2 1 Thus, the three values  ; i = 1; : : : ; 3 are explicitly computed, then, we take ;  and  belonging to the interval [0; "[ and G ( ) < 0; with " > 0 1 2 3 being a fixed precision. Remark 2. The computation of  is performed by a dichotomous proce- dure, in the cases where  2= (0; b "); and G ( ) > 0; as follows: 1. Put a = 0; b = b ". 2. While jb aj > 10 do 0 a+b a+b a+b if G ( ) < 0 then b = , else a = , so  = b. 2 2 2 This computation guarantees a better approximation of the minimizer of G ( ) while remaining in the domain of G. 5. Description of the algorithm In this section, we present the algorithm of our approach to obtain an optimal solution x  of the problem (1). Begin algorithm Initialization We have to decide for the strategy of the displacement step. " > 0 is a given precision,  > 0;  > 0 and  2 ]0; 1[ are fixed parameters. Start with x 2 X and k = 0: Iteration k k T 1. Take B = B(x ) = x A C and L such that B = LL : i=1 2. Compute k k 1 T k 1 > A (x ) = [L(x )] A [L (x )] ; i i < k k b(x ) = trace(A (x )); k k k b b (x ) = trace(A (x )A (x )); ij i j H = (x ): Logarithmic barrier method via minorant function... 109 3. Solve the linear system Hd = b(x) b: 1 1 T 2 4. Calculate E = L H (L ) ; trace(E) and trace(E ): k+1 k 5. Take the new iterate x = x + d; such that  is obtained by the use of the displacement step strategy of G ; i = 1; : : : ; 3: k k+1 6. If n > "; do x = x ;  =  and go to 1. T k+1 T k k k+1 7. If jb x b x j > n; do x = x and go to 1. 8. Take k = k + 1: k+1 9. Stop: x is an approximate solution of the problem (1). End algorithm We know that the optimal solution of (SDP ) is an approximation of the solution of problem (1). More  is closer to zero, more the approximation will be good. Unfortunately, when  approaches zero, the problem (SDP ) be- comes ill-conditioned. For this reason, we use at the beginning of the iteration the values of  that are not near to zero, and verify n < ". We can explain the interpretation of the update  as follows: if x() is an exact solution of (SDP ) , so b x() 2 [m (0) ; m (0) + n]. It is then not necessary to keep on T k+1 T k the calculus of the iterates when jb x b x j  n: The displacement step  will be determined by classical line search of Armijo-Goldstein-Price type or by one of the three following strategies St i; by minimizing the minorant functions G ; i = 1; : : : ; 3. In the next section, we present comparative numerical tests to prove the effectiveness of our approach over line search method. 6. Numerical tests The following examples are taken from the literature (see for instance [3, 4, 11]) and implemented in MATLAB R2013a on Pentium(R). We have taken " = 1:0e 006;  = 0:125 and two values of ;  = 1 or  = 2: In the table of results, (exp (m; n)) represents the size of the example, (Itrat) represents the number of iterations necessary to obtain an optimal solution, (Time) represents the time of computation in seconds (s), (LS) rep- resents the classical line search of Armijo–Goldstein method and (St i) repre- sents the strategy which uses the minorant function G ; with i = 1; : : : ; 3: Recall that the considered problem is min b x; x A C 2 S ; i i > i=1 x 2 R : 110 Assma Leulmi 6.1. Examples with fixed size In the following examples, diag(x) is the n n diagonal matrix with the components of x as the diagonal entries. Example 1. T T 5 8 8 5 1 1 1 2 C = diag ; A = I; b = ; and the matrices A ; k = 1; : : : ; 3; are defined as follows 1 if i = j = k or i = j = k + 1; 1 if i = k; j = k + 1 or i = k + 1; j = k; A [i; j] = 0 otherwise. We start with an initial point x = 1:5 1:5 1:5 1:5 : The optimal solution is x = 0 1:5 0 5 : The optimal value is b x = 11:5: Table 1. The optimal solution obtained with the differ- ent approaches is St 1 0 1:500011 0 5 St 2 0:000011 1:499992 0 4:999981 St 3 0:000061 1:499982 0 4:999871 LR 0:000003 1:499563 0:000011 4:999968 Example 2. C = diag 4 2 2 0 0 0 ; T T A = diag 1 1 1 1 0 0 ; A = diag 1 1 1 0 1 0 ; 1 2 T T 2 2 1 0 0 1 6 2 4 A = diag ; b = : We start with an initial point x = 1 1 2 : 0 2:41354 0:79323 The optimal solution is x = : The optimal value is b x = 8: Logarithmic barrier method via minorant function... 111 Table 2. The optimal solution obtained with the different approaches is St 1 0 2:413012 0:793232 St 2 0:000001 2:413542 0:794323 St 3 0:000015 2:426911 0:794995 LR 0:000003 2:413613 0:7943256 Example 3. T T 4 5 0 0 0 2 1 1 0 0 C = diag ; A = diag ; T T A = diag 1 2 0 1 0 ; A = diag 0 1 0 0 1 ; 2 3 b = 8 7 3 : 2 1 2 We start with an initial point x = : The optimal solution is x = 1 2 0 : The optimal value is b x = 22: Table 3. The optimal solution obtained with the different approaches is St 1 1 2:000001 0 St 2 0:999968 1:999952 0:000011 St 3 1:000015 2:000011 0:000112 LR 0:999863 2:000113 0:000321 Example 4. 1 1 1 1 1 0 C = ; A = ; A = ; 1 2 1 1 1 1 0 1 1 1 b = : We start with an initial point x = 0 3 : The optimal solution is x = 1 2 : The optimal value is b x = 2: 112 Assma Leulmi Table 4. The optimal solution ob- tained with the different approaches is St 1 0:999999 2 St 2 0:999985 1:999952 St 3 1:000155 2:000021 LR 0:998603 2:000113 Example 5. T T 1 1 0 1 1 0 C = diag ; A = diag ; T T A = diag 1 1 1 ; b = 0 1 : We start with an initial point x = 1 1 : 0:4 0 The optimal solution is x = : The optimal value is b x = 0: Table 5. The optimal solu- tion obtained with the differ- ent approaches is St 1 0:399993 0 St 2 0:399903 0 St 3 0:399855 0 LR 0:399603 0 Table 6. The obtained results exp (m; n) St 1 St 2 St 3 LS Itrat Time Itrat Time Itrat Time Itrat Time exp 1(4; 4) 3 0.012 3 0.014 4 0.19 5 0.25 exp 2(3; 6) 1 0.0016 1 0.0022 1 0.0025 7 0.36 exp 3(3; 5) 4 0.0014 4 0.0023 5 0.0032 6 0.36 exp 4(2; 2) 3 0.0001 4 0.0003 4 0.0006 3 0.068 exp 5(2; 3) 2 0.0001 2 0.0028 3 0.0041 3 0.087 Logarithmic barrier method via minorant function... 113 6.2. Example with variable size Example 6 (Example Cube). n = 2m; C is the n  n identity matrix, T m b = (2; : : : ; 2) 2 R ; a 2 R and the entries of the n  n matrix A ; k = 1; : : : ; m; are given by 1 if i = j = k or i = j = k + m; > 2 a if i = j = k + 1 or i = j = k + m + 1; A [i; j] = a if i = k; j = k + 1 or i = k + m; j = k + m + 1; a if i = k + 1; j = k or i = k + m + 1; j = k + m; 0 otherwise. Test 1: a = 0 and C = I: T m We know that the vector x = (1; : : : ; 1) 2 R is the optimal solution. 0 T m We start with an initial point x = (1:5; : : : ; 1:5) 2 R : The following table resumes the obtained results. Table 7. The obtained results in test 1 Size (m; n) St 1 St 2 St 3 LS Itrat Time Itrat Time Itrat Time Itrat Time (50; 100) 1 16 1 17 1 19 dvg (100; 200) 1 89 1 104 1 112 dvg (200; 400) 1 473 1 545 1 554 dvg dvg means that the algorithm does not terminate within a finite time. Commentary: The results of these tests show that LS do not compete with St 1, St 2 and St 3. In the next experiments, we continue only with St 1, St 2 and St 3. Test 2: a = 2; C = 2I: 0 T m We start with an initial point x = (0; 0; ::; 0) 2 R : The optimal solution is x = (1; : : : ; 1) 2 R : The following table resumes the obtained results. Table 8. The obtained results in test 2 Size (m; n) St 1 St 2 St 3 Itrat Time Itrat Time Itrat Time (50; 100) 1 16 1 18 2 24 (100; 200) 1 72 1 114 2 325 (200; 400) 1 258 1 435 2 623 114 Assma Leulmi Test 3: a = 5 and C = 2I: 0 T m We start with an initial point x = (0; 0; : : : ; 0) 2 R : The optimal solution is x = (1; : : : ; 1) 2 R : The following table resumes the obtained results. Table 9. The obtained results in test 3 Size (m; n) St 1 St 2 St 3 Itrat Time Itrat Time Itrat Time (50; 100) 1 19 1 19 2 21 (100; 200) 1 91 1 102 2 117 (200; 400) 1 235 1 490 2 512 Example 7. C = I; A ; k = 1; : : : ; m; are defined as 1 if i = j = k; A [i; j] = 1 if i = j and i = k + m; 0 otherwise; and b = 2; i = 1; : : : ; m: We start with an initial point x = 2; i = 1; : : : ; m: The optimal solution is x = (1; : : : ;1) 2 R : The optimal value is b x = n = 2m: The following table resumes the obtained results. Table 10. The obtained results Size (m; n) St 1 St 2 St 3 Itrat Time Itrat Time Itrat Time (50; 100) 1 17 1 17 2 25 (100; 200) 1 35 1 89 2 354 (200; 400) 1 223 1 234 2 565 Commentary: We notice that the three strategies converge to the optimal solution. These tests show, clearly, the impact of our three strategies offer an optimal solution of (1) and (2) in a reasonable time and with a small number of iterations. st We also note that the 1 strategy is the best. The obtained comparative numerical results favor this last strategy moreover, it requires a computing time largely low vis-a-vis the other two strategies. This seems to be quite expected, because theoretically the strategy St 1 uses the function G that is the closest (best approximation) of the function G: Logarithmic barrier method via minorant function... 115 7. Conclusion In this paper, we have proposed a new logarithmic barrier approach for solving semidefinite programming problem (SDP), since problem (SDP ) is strictly convex, the KKT conditions are necessary and sufficient. For this, we use Newton’s method that allows us to calculate a good descent direction and determine a new iterate, better than the current iterate. To compute the displacement step, several methods have been proposed by scientists and researchers. Including, line search methods, which are very expensive and unworkable. To overcome this problem, we have proposed in this work a new approach, based on the notion of minorant functions, which allows us to determine the displacement step by a simple, easy and much less costly technique. To improve our contribution, we presented numerical simulations to illus- trate the effectiveness of our approach and the convergence of strategies to the optimal solution of the problem (1). These simulations confirm that the first and second strategies are better in terms of number of iterations, computation time and then reduce the computational cost. Therefore, this work has a very interesting theoretical and numerical value. The digital aspect can be pushed to a level of performance very appreciable for the practice. The technique of minorant functions to determine the displacement step in the direction of descent is a very reliable alternative that is confirmed as the technique of choice for (SDP) and other classes of optimization problems: Quadratic Programming (QP) and Non-Linear Programming (NLP).... Acknowledgements. The author is very grateful and would like to thank the Editor-in-Chief and the anonymous referee for their suggestions and help- ful comments which significantly improved the presentation of this paper. References [1] F. Alizadeh, J.-P.A. Haeberly, and M.L. Overton, Primal-dual interior-point methods for semidefinite programming: convergence rates, stability and numerical results, SIAM J. Optim. 8 (1998), no. 3, 746–768. [2] D. Benterki, J.-P. Crouzeix, and B. Merikhi, A numerical implementation of an interior point method for semidefinite programming, Pesqui. Oper. 23 (2003), no. 1, 49–59. [3] D. Benterki, J.-P. Crouzeix, and B. Merikhi, A numerical feasible interior point method for linear semidefinite programs, RAIRO Oper. Res. 41 (2007), no. 1, 49–59. [4] J.-P. Crouzeix and B. Merikhi, A logarithm barrier method for semi-definite program- ming, RAIRO Oper. Res. 42 (2008), no. 2, 123–139. 116 Assma Leulmi [5] J.-P. Crouzeix and A. Seeger, New bounds for the extreme values of a finite sample of real numbers, J. Math. Anal. Appl. 197 (1996), no. 2, 411–426. [6] J. Ji, F.A. Potra, and R. Sheng, On the local convergence of a predictor-corrector method for semidefinite programming, SIAM J. Optim. 10 (1999), no. 1, 195–210. [7] B. Merikhi, Extension de Quelques Méthodes de Points Intérieurs pour la Program- mation Semi-Définie, Thèse de Doctorat, Département de Mathématiques, Université Ferhat Abbas, Sétif, 2006. [8] R.D.C. Monteiro, Primal-dual path-following algorithms for semidefinite programming, SIAM J. Optim. 7 (1997), no. 3, 663–678. [9] Y.E. Nesterov and A.S. Nemirovskii, Optimization Over Positive Semidefinite Matri- ces: Mathematical Background and User’s Manual, Technical report, Central Economic & Mathematical Institute, USSR Academy of Science, Moscow, 1990. [10] R.T. Rockafellar, Convex Analysis, Princeton University Press, Princeton, N.J., 1970. [11] K. Samia and D. Benterki, A relaxed logarithmic barrier method for semidefinite pro- gramming, RAIRO Oper. Res. 49 (2015), no. 3, 555–568. [12] K.-C. Toh, Some new search directions for primal-dual interior point methods in semi- definite programming, SIAM J. Optim. 11 (2000), no. 1, 223–242. [13] I. Touil, D. Benterki, and A. Yassine, A feasible primal-dual interior point method for linear semidefinite programming, J. Comput. Appl. Math. 312 (2017), 216–230. [14] H. Wolkowicz and G.P.H. Styan, Bounds for eigenvalues using traces, Linear Algebra Appl. 29 (1980), 471–506. Department of Mathematics Ferhat Abbas University of Setif-1 Algeria e-mail: smaleulm@yahoo.fr

Journal

Annales Mathematicae Silesianae – de Gruyter

Published: Mar 1, 2023

Keywords: linear semidefinite programming; barrier methods; line search; 90C22; 90C51

There are no references for this article.