本文共 2488 字，大约阅读时间需要 8 分钟。

规则化Logistic回归，利用了牛顿法。

代码如下：

%Regularized logistic regression%算法：Newton's Method close all, clear clc xx= load('ex5Logx.dat');  yy = load('ex5Logy.dat'); y=yy; x = map_feature(xx(:,1),xx(:,2));    %x=[1,x1,x2] map to [1,u,v,u^2,...,v^6] ->28-feature vector m=size(x,2);           %x的维度 lambda=[0,1,10];       %lambda的取值  theta = zeros(m,size(lambda,2)); iter = 12;             %迭代次数 J=zeros(iter,size(lambda,2));                  %代价函数 for i=1:size(lambda,2)     for t = 1:iter          tmp=(lambda(i)/m)*[0;theta(2:28,i)];         h = 1./(1+exp(-(x*theta(:,i)))); %逻辑回归函数         gradient = (1/m).*x'*(h-y)+tmp; %梯度         hession = (1/m).*x'*diag(h)*diag(1-h)*x + (lambda(i)/m).*diag([0;ones(m-1,1)]); %hession         theta(:,i) = theta(:,i) - hession^(-1)*gradient; %更新规则            J(t,i)=-(1/m)*(y'*log(h) + (1-y)'*log(1-h)) + lambda(i)/(2*m)*(theta(2:end,i)'*theta(2:end,i));         %ad=(1/m)*sum(-y.*log(h) - (1-y).*log(1-h)) +(lambda(i)/(2*m))*(theta(2:end,i)'*theta(2:end,i));     end end %**************************** %**************************** % Define the ranges of the grid u = linspace(-1, 1.5, 200); v = linspace(-1, 1.5, 200);% Initialize space for the values to be plotted z = zeros(length(u), length(v));% Evaluate z = theta*x over the gridfor n=1:size(lambda,2)    for i = 1:length(u)        for j = 1:length(v)            % Notice the order of j, i here!            z(j,i) = map_feature(u(i), v(j))*theta(:,n);        end    end    % Because of the way that contour plotting works    % in Matlab, we need to transpose z, or    % else the axis orientation will be flipped!        figure    % Find the indices for the 2 classes    pos = find(yy); neg = find(yy == 0);    plot(xx(pos, 1), xx(pos, 2), 'k+')    %标签为1    hold on    plot(xx(neg, 1), xx(neg, 2), 'o','MarkerEdgeColor','b','MarkerFaceColor','yellow');  %标签为0       xlabel('U');    ylabel('V');    %%%%%%%%%%%%%%%%%%%%%%%%%%%        %z = z';    % Plot z = 0 by specifying the range [0, 0]    contour(u,v,z, [0, 0], 'LineWidth', 2)    titlestyle={'\lambda = 0','\lambda = 1','\lambda = 10'};    legend('y = 1','y = 0','Decision bondary');    title(char(titlestyle(n)),'FontSize',14);    hold offend%Plot Jfigure plotcolor={'r','g','b'}; for i=1:size(lambda,2)    plot([1:iter]',J(:,i),'o--','MarkerFaceColor', char(plotcolor(i)), 'MarkerSize', 8)     hold on end legend('\lambda = 0','\lambda = 1','\lambda = 10'); xlabel('Iteration'); ylabel('J')  title('cost function') grid on axis([1,12,0,4])

参考1：

参考2：

转载地址：http://wbpti.baihongyu.com/