{"id":3093,"date":"2022-02-27T11:25:11","date_gmt":"2022-02-27T03:25:11","guid":{"rendered":"https:\/\/egonlin.com\/?p=3093"},"modified":"2022-02-27T11:25:11","modified_gmt":"2022-02-27T03:25:11","slug":"%e7%ac%ac%e4%b8%80%e8%8a%82%ef%bc%9a%e7%ba%bf%e6%80%a7%e5%8f%af%e5%88%86%e6%94%af%e6%8c%81%e5%90%91%e9%87%8f%e6%9c%ba","status":"publish","type":"post","link":"https:\/\/egonlin.com\/?p=3093","title":{"rendered":"\u7b2c\u4e00\u8282\uff1a\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a"},"content":{"rendered":"

\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a<\/h1>\n

\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u5b66\u4e60\u76ee\u6807<\/h1>\n
    \n
  1. \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u3001\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\u3001\u975e\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\u533a\u522b<\/li>\n
  2. \u51fd\u6570\u95f4\u9694\u4e0e\u51e0\u4f55\u95f4\u9694<\/li>\n
  3. \u76ee\u6807\u51fd\u6570\u4e0e\u76ee\u6807\u51fd\u6570\u7684\u4f18\u5316\u95ee\u9898<\/li>\n
  4. \u652f\u6301\u5411\u91cf\u548c\u95f4\u9694\u8fb9\u754c<\/li>\n
  5. \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u7684\u6b65\u9aa4<\/li>\n<\/ol>\n

    \u652f\u6301\u5411\u91cf\u673a\u5f15\u5165<\/h1>\n

      \u652f\u6301\u5411\u91cf\u673a\uff08support vector machines\uff0cSVM\uff09\u8bde\u751f\u4e8c\u5341\u591a\u5e74\uff0c\u7531\u4e8e\u5b83\u826f\u597d\u7684\u5206\u7c7b\u6027\u80fd\u5e2d\u5377\u4e86\u673a\u5668\u5b66\u4e60\u9886\u57df\uff0c\u5982\u679c\u4e0d\u8003\u8651\u96c6\u6210\u5b66\u4e60\u3001\u4e0d\u8003\u8651\u7279\u5b9a\u7684\u8bad\u7ec3\u6570\u636e\u96c6\uff0cSVM\u7531\u4e8e\u6cdb\u5316\u80fd\u529b\u5f3a\uff0c\u56e0\u6b64\u5728\u5206\u7c7b\u7b97\u6cd5\u4e2d\u7684\u8868\u73b0\u6392\u7b2c\u4e00\u662f\u6ca1\u6709\u4ec0\u4e48\u5f02\u8bae\u7684\uff0c\u901a\u5e38\u60c5\u51b5\u4e0b\u5b83\u4e5f\u662f\u9996\u9009\u5206\u7c7b\u5668\u3002\u8fd1\u51e0\u5e74SVM\u5728\u56fe\u50cf\u5206\u7c7b\u9886\u57df\u867d\u6709\u88ab\u6df1\u5ea6\u5b66\u4e60\u8d76\u8d85\u7684\u8d8b\u52bf\uff0c\u4f46\u7531\u4e8e\u6df1\u5ea6\u5b66\u4e60\u9700\u8981\u5927\u91cf\u7684\u6570\u636e\u9a71\u52a8\u56e0\u6b64\u5728\u67d0\u4e9b\u9886\u57dfSVM\u8fd8\u662f\u65e0\u6cd5\u66ff\u4ee3\u7684\u3002<\/p>\n

      SVM\u662f\u4e00\u79cd\u4e8c\u5206\u7c7b\u6a21\u578b\uff0c\u5b83\u7684\u57fa\u672c\u6a21\u578b\u662f\u5b9a\u4e49\u5728\u7279\u5f81\u7a7a\u95f4\u4e0a\u7684\u95f4\u9694\u6700\u5927\u7684\u7ebf\u6027\u5206\u7c7b\u5668\uff0c\u95f4\u9694\u6700\u5927\u5316\u4e5f\u4f7f\u5b83\u4e0d\u540c\u4e8e\u611f\u77e5\u673a\uff1bSVM\u8fd8\u6709\u6838\u6280\u5de7\uff0c\u4e5f\u56e0\u6b64\u5b83\u4e5f\u662f\u4e00\u4e2a\u975e\u7ebf\u6027\u5206\u7c7b\u5668\u3002<\/p>\n

      SVM\u901a\u8fc7\u5b66\u5f97\u6a21\u578b\u6784\u5efa\u7684\u96be\u5ea6\u7531\u7b80\u81f3\u7e41\u53ef\u4ee5\u5212\u5206\u4e3a\u4ee5\u4e0b\u4e09\u79cd\uff1a<\/p>\n

      \n
    1. \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\uff08linear support vector machine in linearly separable case\uff09\uff1a\u5f53\u8bad\u7ec3\u6570\u636e\u7ebf\u6027\u53ef\u5206\u65f6\uff0c\u901a\u8fc7\u786c\u95f4\u9694\u6700\u5927\u5316\uff08hard margin maximization\uff09\u5b66\u4e60\u4e00\u4e2a\u7ebf\u6027\u7684\u5206\u7c7b\u5668\uff0c\u5373\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\uff0c\u4e5f\u79f0\u4f5c\u786c\u95f4\u9694\u652f\u6301\u5411\u91cf\u673a<\/li>\n
    2. \u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\uff08linear support vector machine\uff09\uff1a\u5f53\u8bad\u7ec3\u6570\u636e\u8fd1\u4f3c\u7ebf\u6027\u53ef\u5206\u65f6\uff0c\u901a\u8fc7\u8f6f\u95f4\u9694\u6700\u5927\u5316\uff08soft margin maximization\uff09\u4e5f\u5b66\u4e60\u4e00\u4e2a\u7ebf\u6027\u5206\u7c7b\u5668\uff0c\u5373\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\uff0c\u4e5f\u79f0\u4f5c\u8f6f\u95f4\u9694\u652f\u6301\u5411\u91cf\u673a<\/li>\n
    3. \u975e\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\uff08non-linear support vector machine\uff09\uff1a\u5f53\u8bad\u7ec3\u6570\u636e\u7ebf\u6027\u4e0d\u53ef\u5206\u65f6\uff0c\u901a\u8fc7\u4f7f\u7528\u6838\u6280\u5de7\uff08kernel trick\uff09\u53ca\u8f6f\u95f4\u9694\u6700\u5927\u5316\uff0c\u5b66\u4e60\u4e00\u4e2a\u975e\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a<\/li>\n<\/ol>\n

      \u7ebf\u6027\u53ef\u5206\u548c\u7ebf\u6027\u4e0d\u53ef\u5206<\/h2>\n

        \u7531\u4e8e\u611f\u77e5\u673a\u4e00\u6587\u4e2d\u8be6\u7ec6\u7684\u4ecb\u7ecd\u8fc7\u7ebf\u6027\u53ef\u5206\u4e0e\u7ebf\u6027\u4e0d\u53ef\u5206\u7684\u533a\u522b\uff0c\u8fd9\u91cc\u53ea\u7ed9\u51fa\u56fe\u4f8b\u4fbf\u4e8e\u7406\u89e3\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\u548c\u975e\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\u3002<\/p>\n

      # \u7ebf\u6027\u53ef\u5206\u4e0e\u7ebf\u6027\u4e0d\u53ef\u5206\u56fe\u4f8b\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.font_manager import FontProperties\n\n%matplotlib inline\nfont = FontProperties(fname='\/Library\/Fonts\/Heiti.ttc')\n\nnp.random.seed(1)\nx1 = np.random.random(20)+1.5\ny1 = np.random.random(20)+0.5\nx2 = np.random.random(20)+3\ny2 = np.random.random(20)+0.5\n\n# \u4e00\u884c\u4e8c\u5217\u7b2c\u4e00\u4e2a\nplt.subplot(121)\nplt.scatter(x1, y1, s=50, color='b', label='\u7537\u5b69(+1)')\nplt.scatter(x2, y2, s=50, color='r', label='\u5973\u5b69(-1)')\nplt.vlines(2.8, 0, 2, colors="r", linestyles="-", label='$wx+b=0$')\nplt.title('\u7ebf\u6027\u53ef\u5206', fontproperties=font, fontsize=20)\nplt.xlabel('x')\nplt.legend(prop=font)\n\n# \u4e00\u884c\u4e8c\u5217\u7b2c\u4e8c\u4e2a\nplt.subplot(122)\nplt.scatter(x1, y1, s=50, color='b', label='\u7537\u5b69(+1)')\nplt.scatter(x2, y2, s=50, color='r', label='\u5973\u5b69(-1)')\nplt.scatter(3.5, 1, s=50, color='b')\nplt.scatter(3.6, 1.2, s=50, color='b')\nplt.scatter(3.9, 1.3, s=50, color='b')\nplt.scatter(3.8, 1.3, s=50, color='b')\nplt.scatter(3.7, 0.6, s=50, color='b')\nplt.title('\u7ebf\u6027\u4e0d\u53ef\u5206', fontproperties=font, fontsize=20)\nplt.xlabel('x')\nplt.legend(prop=font)\nplt.show()<\/code><\/pre>\n
      \"\"<\/div><\/p>\n
      \u611f\u77e5\u673a\u6a21\u578b\u548c\u652f\u6301\u5411\u91cf\u673a<\/h2>\n
        \u611f\u77e5\u673a\u4e00\u6587\u4e2d\u8be6\u7ec6\u8bb2\u89e3\u8fc7\u611f\u77e5\u673a\u6a21\u578b\u7684\u539f\u7406\uff0c\u6b64\u5904\u4e0d\u591a\u8d58\u8ff0\uff0c\u7b80\u5355\u6982\u62ec\u3002<\/p>\n
        \u5728\u4e8c\u7ef4\u7a7a\u95f4\u4e2d\uff0c\u611f\u77e5\u673a\u6a21\u578b\u8bd5\u56fe\u627e\u5230\u4e00\u6761\u76f4\u7ebf\u80fd\u591f\u628a\u4e8c\u5143\u6570\u636e\u5206\u9694\u5f00\uff1b\u5728\u9ad8\u7ef4\u7a7a\u95f4\u4e2d\u611f\u77e5\u673a\u6a21\u578b\u8bd5\u56fe\u627e\u5230\u4e00\u4e2a\u8d85\u5e73\u9762$S$\uff0c\u80fd\u591f\u628a\u4e8c\u5143\u6570\u636e\u9694\u79bb\u5f00\u3002\u8fd9\u4e2a\u8d85\u5e73\u9762$S$\u4e3a$\\omega{x}+b=0$\uff0c\u5728\u8d85\u5e73\u9762$S$\u4e0a\u65b9\u7684\u6570\u636e\u5b9a\u4e49\u4e3a$1$\uff0c\u5728\u8d85\u5e73\u9762$S$\u4e0b\u65b9\u7684\u6570\u636e\u5b9a\u4e49\u4e3a$-1$\uff0c\u5373\u5f53$\\omega{x}>0$\uff0c$\\hat{y}=+1$\uff1b\u5f53$\\omega{x}<0$\uff0c$\\hat{y}=-1$\u3002<\/p>\n
        \u4e0a\u5f20\u7ebf\u6027\u53ef\u5206\u548c\u7ebf\u6027\u4e0d\u53ef\u5206\u7684\u533a\u522b\u56fe\u7b2c\u4e00\u5f20\u56fe\u5219\u627e\u5230\u4e86\u4e00\u6761\u76f4\u7ebf\u80fd\u591f\u628a\u4e8c\u5143\u6570\u636e\u5206\u9694\u5f00\uff0c\u4f46\u662f\u80fd\u591f\u53d1\u73b0\u4e8b\u5b9e\u4e0a\u53ef\u80fd\u4e0d\u53ea\u5b58\u5728\u4e00\u6761\u76f4\u7ebf\u5c06\u6570\u636e\u5212\u5206\u4e3a\u4e24\u7c7b\uff0c\u56e0\u6b64\u518d\u627e\u5230\u8fd9\u4e9b\u76f4\u7ebf\u540e\u8fd8\u9700\u8981\u627e\u5230\u4e00\u6761\u6700\u4f18\u76f4\u7ebf\uff0c\u5bf9\u4e8e\u8fd9\u4e00\u70b9\u611f\u77e5\u673a\u6a21\u578b\u4f7f\u7528\u7684\u7b56\u7565\u662f\u8ba9\u6240\u6709\u8bef\u5206\u7c7b\u70b9\u5230\u8d85\u5e73\u9762\u7684\u8ddd\u79bb\u548c\u6700\u5c0f\uff0c\u5373\u6700\u5c0f\u5316\u8be5\u5f0f
      \n$$
      \nJ(\\omega)=\\sum_{{x_i}\\in{M}} {\\frac{- y_i(\\omega{x_i}+b)}{||\\omega||_2}}
      \n$$
      \n  \u4e0a\u5f0f\u4e2d\u53ef\u4ee5\u770b\u51fa\u5982\u679c$\\omega$\u548c$b$\u6210\u6bd4\u4f8b\u7684\u589e\u52a0\uff0c\u5219\u5206\u5b50\u7684$\\omega$\u548c$b$\u6269\u5927$n$\u500d\u65f6\uff0c\u5206\u6bcd\u7684L2\u8303\u6570\u4e5f\u5c06\u6269\u5927$n$\u500d\uff0c\u4e5f\u5c31\u662f\u8bf4\u5206\u5b50\u548c\u5206\u6bcd\u6709\u56fa\u5b9a\u7684\u500d\u6570\u5173\u7cfb\uff0c\u65e2\u53ef\u4ee5\u5206\u6bcd$||\\omega||2$\u56fa\u5b9a\u4e3a$1$\uff0c\u7136\u540e\u6c42\u5206\u5b50\u7684\u6700\u5c0f\u5316\u4f5c\u4e3a\u4ee3\u4ef7\u51fd\u6570\uff0c\u56e0\u6b64\u7ed9\u5b9a\u611f\u77e5\u673a\u7684\u76ee\u6807\u51fd\u6570\u4e3a
      \n$$
      \nJ(\\omega)=\\sum<\/em>{{x_i}\\in{M}} – y_i(\\omega{x_i}+b)
      \n$$
      \n  \u65e2\u7136\u5206\u5b50\u548c\u5206\u6bcd\u6709\u56fa\u5b9a\u500d\u6570\uff0c\u90a3\u4e48\u53ef\u4e0d\u53ef\u4ee5\u56fa\u5b9a\u5206\u5b50\uff0c\u628a\u5206\u6bcd\u7684\u5012\u6570\u4f5c\u4e3a\u76ee\u6807\u51fd\u6570\u5462\uff1f\u4e00\u5b9a\u662f\u53ef\u4ee5\u7684\uff0c\u56fa\u5b9a\u5206\u5b50\u5c31\u662f\u652f\u6301\u5411\u91cf\u673a\u4f7f\u7528\u7684\u7b56\u7565\u3002<\/p>\n
      \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u8be6\u89e3<\/h1>\n
      \u786e\u4fe1\u5ea6<\/h2>\n
      # \u786e\u4fe1\u5ea6\u56fe\u4f8b\nimport matplotlib.pyplot as plt\nfrom matplotlib.font_manager import FontProperties\n%matplotlib inline\nfont = FontProperties(fname='\/Library\/Fonts\/Heiti.ttc')\n\nx1 = [1, 2, 2.5, 3.2]\nx11 = [4.5, 5, 6]\nx2 = [1, 1.2, 1.4, 1.5]\nx22 = [1.5, 1.3, 1]\n\nplt.scatter(x1, x2, s=50, color='b', label='+1')\nplt.scatter(x11, x22, s=50, color='r', label='-1')\nplt.vlines(3.5, 0.8, 2, colors="g", linestyles="-", label='$w*x+b=0$')\nplt.text(2, 1.3, s='A', fontsize=15, color='k', ha='center')\nplt.text(2.5, 1.5, s='B', fontsize=15, color='k', ha='center')\nplt.text(3.2, 1.6, s='C', fontsize=15, color='k', ha='center')\nplt.legend()\nplt.show()<\/code><\/pre>\n
      \"\"<\/div><\/p>\n
        \u4e0a\u56fe\u6709\u5747\u5728\u8d85\u5e73\u9762\u6b63\u7c7b\u7684A\uff0cB\uff0cC\u4e09\u4e2a\u70b9\u3002\u56e0\u4e3a\u70b9A\u8ddd\u79bb\u8d85\u5e73\u9762\u8fdc\uff0c\u5982\u679c\u9884\u6d4b\u4e3a\u6b63\u7c7b\u70b9\uff0c\u5c31\u6bd4\u8f83\u786e\u4fe1\u9884\u6d4b\u65f6\u6b63\u786e\u7684\uff1b\u70b9C\u8ddd\u79bb\u8d85\u5e73\u9762\u8f83\u8fd1\uff0c\u5982\u679c\u9884\u6d4b\u4e3a\u6b63\u7c7b\u70b9\u5c31\u4e0d\u90a3\u4e48\u786e\u4fe1\uff0c\u56e0\u4e3a\u8d85\u5e73\u9762\u53ef\u80fd\u5b58\u5728\u7740\u591a\u6761\uff0c\u6709\u53ef\u80fd\u6709\u53e6\u4e00\u6761\u66f4\u4f18\u7684\u8d85\u5e73\u9762$\\omega{x}+b=0$\uff1b\u70b9B\u4ecb\u4e8eA\u548cC\u4e4b\u95f4\uff0c\u5219\u5176\u9884\u6d4b\u4e3a\u6b63\u7c7b\u70b9\u7684\u786e\u4fe1\u5ea6\u4ecb\u4e8eA\u548cC\u4e4b\u95f4\u3002<\/p>\n
      \u51fd\u6570\u95f4\u9694\u548c\u51e0\u4f55\u95f4\u9694<\/h2>\n
      \u51fd\u6570\u95f4\u9694<\/h3>\n
        \u4e00\u4e2a\u70b9\u8ddd\u79bb\u8d85\u5e73\u9762\u7684\u8fdc\u8fd1\u53ef\u4ee5\u8868\u793a\u5206\u7c7b\u9884\u6d4b\u7684\u786e\u4fe1\u7a0b\u5ea6\u3002\u5728\u8d85\u5e73\u9762\u56fa\u5b9a\u4f4d$\\omega{x}+b=0$\u7684\u60c5\u51b5\u4e0b\uff0c$|\\omega+b=0|$\u8868\u793a\u70b9$x$\u5230\u8d85\u5e73\u9762\u7684\u76f8\u5bf9\u8ddd\u79bb\uff0c\u800c$\\omega{x}$\u548c$y$\u662f\u5426\u540c\u53f7\u80fd\u591f\u5224\u65ad\u5206\u7c7b\u662f\u5426\u6b63\u786e\uff0c\u6240\u4ee5\u53ef\u4ee5\u7528\u91cf$y(\\omega{x}+b)$\u8868\u793a\u5206\u7c7b\u7684\u6b63\u786e\u6027\u548c\u786e\u4fe1\u5ea6\uff0c\u8fd9\u5c31\u662f\u51fd\u6570\u95f4\u9694\uff08functional margin\uff09\u7684\u6982\u5ff5\u3002<\/p>\n
        \u7ed9\u5b9a\u6570\u636e\u96c6$T$\u548c\u8d85\u5e73\u9762$(\\omega,b)$\uff0c\u5b9a\u4e49\u8d85\u5e73\u9762$(\\omega,b)$\u5173\u4e8e\u6837\u672c\u70b9$(x_i,y_i)$\u7684\u51fd\u6570\u95f4\u9694\u4e3a
      \n$$
      \n\\hat{\\gamma_i} = y_i(\\omega{xi}+b)
      \n$$
      \n  \u5bf9\u4e8e\u8bad\u7ec3\u96c6$T$\u4e2d$m$\u4e2a\u6837\u672c\u70b9\u5bf9\u5e94\u7684$m$\u4e2a\u51fd\u6570\u95f4\u9694\u7684\u6700\u5c0f\u503c\uff0c\u5c31\u662f\u6574\u4e2a\u8bad\u7ec3\u96c6\u7684\u51fd\u6570\u95f4\u9694\uff0c\u5373
      \n$$
      \n\\hat{\\gamma} = \\underbrace{min}<\/em>{i=1,\\ldots,m}\\hat{\\gamma_i}
      \n$$
      \n  \u51fd\u6570\u95f4\u9694\u5e76\u4e0d\u80fd\u6b63\u5e38\u53cd\u5e94\u70b9\u5230\u8d85\u5e73\u9762\u7684\u8ddd\u79bb\uff0c\u56e0\u4e3a\u53ea\u8981\u6210\u6bd4\u4f8b\u7684\u6539\u53d8$\\omega$\u548c$b$\uff0c\u8d85\u5e73\u9762\u5374\u5e76\u6ca1\u6709\u6539\u53d8\uff0c\u4f46\u51fd\u6570\u95f4\u9694\u5374\u4f1a\u53d8\u4e3a\u539f\u6765\u7684\u4e24\u500d\u3002<\/p>\n
      \u51e0\u4f55\u95f4\u9694<\/h3>\n
        \u7531\u4e8e\u51fd\u6570\u95f4\u9694\u4e0d\u80fd\u53cd\u5e94\u70b9\u5230\u8d85\u5e73\u9762\u7684\u8ddd\u79bb\uff0c\u56e0\u6b64\u53ef\u4ee5\u5bf9\u8d85\u5e73\u9762\u7684\u6cd5\u5411\u91cf$\\omega$\u52a0\u4e0a\u7ea6\u675f\u6761\u4ef6\uff0c\u4f8b\u5982\u611f\u77e5\u673a\u6a21\u578b\u4e2d\u89c4\u8303\u5316$||\\omega||=1$\uff0c\u4f7f\u5f97\u95f4\u9694\u662f\u786e\u5b9a\u7684\u3002\u6b64\u65f6\u7684\u51fd\u6570\u95f4\u9694\u5c06\u4f1a\u53d8\u6210\u51e0\u4f55\u95f4\u9694\uff08geometric margin\uff09\u3002<\/p>\n
        \u5bf9\u4e8e\u67d0\u4e00\u5b9e\u4f8b$x_i$\uff0c\u5176\u7c7b\u6807\u8bb0\u4e3a$y_i$\uff0c\u5219\u6539\u70b9\u7684\u51e0\u4f55\u95f4\u9694\u7684\u5b9a\u4e49\u4e3a
      \n$$
      \n\\gamma_i = {\\frac{y_i(\\omega{xi}+b)}{||\\omega||}}
      \n$$
      \n  \u5bf9\u4e8e\u8bad\u7ec3\u96c6$T$\u4e2d$m$\u4e2a\u6837\u672c\u70b9\u5bf9\u5e94\u7684$m$\u4e2a\u51fd\u6570\u95f4\u9694\u7684\u6700\u5c0f\u503c\uff0c\u5c31\u662f\u6574\u4e2a\u8bad\u7ec3\u96c6\u7684\u51e0\u4f55\u95f4\u9694\uff0c\u5373
      \n$$
      \n\\gamma = \\underbrace{min}<\/em>{i=1,\\ldots,m}\\gamma_i
      \n$$
      \n  \u51e0\u4f55\u95f4\u9694\u624d\u662f\u70b9\u5230\u8d85\u5e73\u9762\u7684\u771f\u6b63\u8ddd\u79bb\uff0c\u611f\u77e5\u673a\u6a21\u578b\u7528\u5230\u7684\u8ddd\u79bb\u5c31\u662f\u51e0\u4f55\u95f4\u9694\u3002<\/p>\n
      \u51fd\u6570\u95f4\u9694\u548c\u51e0\u4f55\u95f4\u9694\u7684\u5173\u7cfb<\/h3>\n
        \u7531\u51fd\u6570\u95f4\u9694\u548c\u51e0\u4f55\u95f4\u9694\u7684\u5b9a\u4e49\u53ef\u77e5\u51fd\u6570\u95f4\u9694\u548c\u51e0\u4f55\u95f4\u9694\u6709\u4ee5\u4e0b\u7684\u5173\u7cfb
      \n$$
      \n\\gamma = {\\frac{\\hat{\\gamma}}{||\\omega||}}
      \n$$<\/p>\n
      \u652f\u6301\u5411\u91cf\u548c\u95f4\u9694\u8fb9\u754c<\/h2>\n
        \u7531\u4e8e\u53ef\u4ee5\u627e\u5230\u591a\u4e2a\u8d85\u5e73\u9762\u5c06\u6570\u636e\u5206\u5f00\u5bfc\u81f4\u79bb\u8d85\u5e73\u9762\u8fd1\u7684\u70b9\u4e0d\u786e\u4fe1\u5ea6\u9ad8\uff0c\u56e0\u6b64\u611f\u77e5\u673a\u6a21\u578b\u4f18\u5316\u65f6\u5e0c\u671b\u6240\u6709\u7684\u70b9\u90fd\u79bb\u8d85\u5e73\u9762\u8fdc\u3002\u4f46\u662f\u79bb\u8d85\u5e73\u9762\u8f83\u8fdc\u7684\u70b9\u5df2\u7ecf\u88ab\u6b63\u786e\u5206\u7c7b\uff0c\u8ba9\u5b83\u4eec\u79bb\u8d85\u5e73\u9762\u66f4\u8fdc\u6beb\u65e0\u610f\u4e49\u3002\u7531\u4e8e\u662f\u79bb\u8d85\u5e73\u9762\u8fd1\u7684\u70b9\u5bb9\u6613\u88ab\u8bef\u5206\u7c7b\uff0c\u56e0\u6b64\u6b21\u53ef\u4ee5\u8ba9\u79bb\u8d85\u5e73\u9762\u8f83\u8fd1\u7684\u70b9\u5c3d\u53ef\u80fd\u7684\u8fdc\u79bb\u8d85\u5e73\u9762\uff0c\u8fd9\u6837\u624d\u80fd\u63d0\u5347\u6a21\u578b\u7684\u5206\u7c7b\u6548\u679c\uff0c\u8fd9\u6b63\u662fSVM\u601d\u60f3\u7684\u8d77\u6e90\u3002<\/p>\n
      # \u95f4\u9694\u6700\u5927\u5316\u56fe\u4f8b\nimport numpy as np\nimport matplotlib.pyplot as plt\nfrom matplotlib.font_manager import FontProperties\nfrom sklearn import svm\n%matplotlib inline\nfont = FontProperties(fname='\/Library\/Fonts\/Heiti.ttc')\n\nnp.random.seed(8)  # \u4fdd\u8bc1\u6570\u636e\u968f\u673a\u7684\u552f\u4e00\u6027\n\n# \u6784\u9020\u7ebf\u6027\u53ef\u5206\u6570\u636e\u70b9\narray = np.random.randn(20, 2)\nX = np.r_[array-[3, 3], array+[3, 3]]\ny = [0]*20+[1]*20\n\n# \u5efa\u7acbsvm\u6a21\u578b\nclf = svm.SVC(kernel='linear')\nclf.fit(X, y)\n\n# \u6784\u9020\u7b49\u7f51\u4e2a\u65b9\u9635\nx1_min, x1_max = X[:, 0].min(), X[:, 0].max(),\nx2_min, x2_max = X[:, 1].min(), X[:, 1].max(),\nx1, x2 = np.meshgrid(np.linspace(x1_min, x1_max),\n                     np.linspace(x2_min, x2_max))\n\n# \u5f97\u5230\u5411\u91cfw: w_0x_1+w_1x_2+b=0\nw = clf.coef_[0]\n# \u52a01\u540e\u624d\u53ef\u7ed8\u5236 -1 \u7684\u7b49\u9ad8\u7ebf [-1,0,1] + 1 = [0,1,2]\nf = w[0]*x1 + w[1]*x2 + clf.intercept_[0] + 1\n\n# \u7ed8\u5236H1\uff0c\u5373wx+b=-1\nplt.contour(x1, x2, f, [0], colors='k', linestyles='--')\nplt.text(2, -4, s='$H_2={\\omega}x+b=-1$', fontsize=10, color='r', ha='center')\n\n# \u7ed8\u5236\u5206\u9694\u8d85\u5e73\u9762\uff0c\u5373wx+b=0\nplt.contour(x1, x2, f, [1], colors='k')\nplt.text(2.5, -2, s='$\\omega{x}+b=0$', fontsize=10, color='r', ha='center')\nplt.text(2.5, -2.5, s='\u5206\u79bb\u8d85\u5e73\u9762', fontsize=10,\n         color='r', ha='center', fontproperties=font)\n\n# \u7ed8\u5236H2\uff0c\u5373wx+b=1\nplt.contour(x1, x2, f, [2], colors='k', linestyles='--')\nplt.text(3, 0, s='$H_1=\\omega{x}+b=1$', fontsize=10, color='r', ha='center')\n\n# \u7ed8\u5236\u6570\u636e\u6563\u70b9\u56fe\nplt.scatter(X[0:20, 0], X[0:20, 1], cmap=plt.cm.Paired, marker='x')\nplt.text(1, 1.8, s='\u652f\u6301\u5411\u91cf', fontsize=10, color='gray',\n         ha='center', fontproperties=font)\n\nplt.scatter(X[20:40, 0], X[20:40, 1], cmap=plt.cm.Paired, marker='o')\nplt.text(-1.5, -0.5, s='\u652f\u6301\u5411\u91cf', fontsize=10,\n         color='gray', ha='center', fontproperties=font)\n# plt.scatter(clf.support_vectors_[:,0],clf.support_vectors_[:,1) # \u7ed8\u5236\u652f\u6301\u5411\u91cf\u70b9\n\nplt.xlim(x1_min-1, x1_max+1)\nplt.ylim(x2_min-1, x2_max+1)\nplt.show()<\/code><\/pre>\n
      \"\"<\/div><\/p>\n
        \u5982\u4e0a\u56fe\u6240\u793a\uff0c\u5206\u79bb\u8d85\u5e73\u9762\u4e3a$\\omega{x}+b=0$\u3002\u5982\u679c\u6240\u6709\u7684\u6837\u672c\u4e0d\u5149\u53ef\u4ee5\u88ab\u5206\u79bb\u8d85\u5e73\u9762\u5206\u5f00\uff0c\u8fd8\u548c\u5206\u79bb\u8d85\u5e73\u9762\u4fdd\u6301\u4e00\u5b9a\u7684\u51fd\u6570\u95f4\u9694\uff08\u4e0a\u56fe\u7684\u51fd\u6570\u95f4\u9694\u4e3a1\uff09\u3002<\/p>\n
        \u5bf9\u4e8e$y_i=1$\u7684\u6b63\u4f8b\u70b9\uff0c\u652f\u6301\u5411\u91cf\u5728\u8d85\u5e73\u9762$H_1\uff1a\\omega{x}+b=1$\u4e0a\uff1b\u5bf9\u4e8e$y_i=-1$\u7684\u8d1f\u4f8b\u70b9\uff0c\u652f\u6301\u5411\u91cf\u5728\u8d85\u5e73\u9762$H_2\uff1a\\omega{x}+b=-1$\u4e0a\uff0c\u5373\u5728$H_1$\u548c$H_2$\u4e0a\u7684\u70b9\u5c31\u662f\u652f\u6301\u5411\u91cf\uff08support vector\uff09\u3002<\/p>\n
        \u56fe\u4e2d\u865a\u7ebf\u6240\u793a\u7684\u4e24\u4e2a\u5e73\u884c\u7684\u8d85\u5e73\u9762$H_1$\u548c$H_2$\u4e4b\u95f4\u7684\u8ddd\u79bb\u79f0\u4e3a\u95f4\u9694\uff08margin\uff09\uff0c\u95f4\u9694\u4f9d\u8d56\u4e8e\u5206\u79bb\u8d85\u5e73\u9762\u7684\u6cd5\u5411\u91cf$\\omega$\uff0c\u7b49\u4e8e${\\frac{2}{||\\omega||}}$\u3002<\/p>\n
        \u7531\u6b64\u53ef\u4ee5\u770b\u51fa\u53ea\u6709\u652f\u6301\u5411\u91cf\u51b3\u5b9a\u5206\u79bb\u8d85\u5e73\u9762\u7684\u4f4d\u7f6e\uff0c\u5373\u5176\u4ed6\u5b9e\u4f8b\u70b9\u5bf9\u5206\u79bb\u8d85\u5e73\u9762\u6ca1\u6709\u5f71\u54cd\u3002\u6b63\u5f0f\u7531\u4e8e\u652f\u6301\u5411\u91cf\u5728\u786e\u5b9a\u5206\u79bb\u8d85\u5e73\u9762\u7684\u65f6\u5019\u8d77\u7740\u51b3\u5b9a\u6027\u7684\u4f5c\u7528\uff0c\u6240\u4ee5\u5c06\u8fd9\u79cd\u5206\u7c7b\u6a21\u578b\u79f0\u4f5c\u652f\u6301\u5411\u91cf\u673a\u3002\u7531\u4e8e\u652f\u6301\u5411\u91cf\u7684\u4e2a\u6570\u4e00\u822c\u5f88\u5c11\uff0c\u56e0\u6b64\u652f\u6301\u5411\u91cf\u673a\u7531\u5f88\u5c11\u7684\u91cd\u8981\u7684\u6837\u672c\u786e\u5b9a\u3002<\/p>\n
      \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u76ee\u6807\u51fd\u6570\u5373\u786c\u95f4\u9694\u6700\u5927\u5316<\/h2>\n
        \u4e0a\u4e00\u8282\u8bb2\u5230\u4e86SVM\u7684\u6a21\u578b\u5176\u5b9e\u5c31\u662f\u8ba9\u6240\u6709\u70b9\u5230\u5206\u79bb\u8d85\u5e73\u9762\u7684\u8ddd\u79bb\u5927\u4e8e\u4e00\u5b9a\u7684\u8ddd\u79bb\uff0c\u5373\u6240\u6709\u5df2\u88ab\u5206\u7c7b\u7684\u70b9\u8981\u5728\u5404\u81ea\u7c7b\u522b\u7684\u652f\u6301\u5411\u91cf\u7684\u4e24\u8fb9\uff0c\u5373\u5e0c\u671b\u6700\u5927\u5316\u8d85\u5e73\u9762$(\\omega,b)$\u5173\u4e8e\u8bad\u7ec3\u6570\u636e\u96c6\u7684\u51e0\u4f55\u95f4\u9694$\\gamma$\uff0c\u8fd9\u4e2a\u95ee\u9898\u53ef\u4ee5\u8868\u793a\u4e3a\u4e0b\u9762\u7684\u7ea6\u675f\u6700\u4f18\u5316\u95ee\u9898
      \n$$
      \n\\begin{align}
      \n& \\underbrace{\\max}_{\\omega,b} \\gamma \\
      \n& s.t. \\quad {\\frac{y_i(\\omega{x_i}+b)}{||\\omega||}}\\geq\\gamma, \\quad  i=1,2,\\ldots,m
      \n\\end{align}
      \n$$
      \n\u5176\u4e2d$m$\u8868\u793a$m$\u4e2a\u6837\u672c\uff0c$s.t.$\u8868\u793a\u201csubject to\uff08\u4f7f\u5f97\u2026\u6ee1\u8db3\u2026\uff09\u201d\uff0c\u5373\u7ea6\u675f\u6761\u4ef6\uff0c\u8be5\u7ea6\u675f\u6761\u4ef6\u6307\u7684\u662f\u8d85\u5e73\u9762$(\\omega,b)$\u5173\u4e8e\u6bcf\u4e2a\u8bad\u7ec3\u6837\u672c\u7684\u96c6\u5408\u95f4\u9694\u81f3\u5c11\u662f$\\gamma$\u3002<\/p>\n
        \u901a\u8fc7\u51fd\u6570\u95f4\u9694\u548c\u51e0\u4f55\u95f4\u9694\u7684\u5173\u7cfb\uff0c\u53ef\u4ee5\u628a\u4e0a\u8ff0\u5f0f\u5b50\u6539\u5199\u6210
      \n$$
      \n\\begin{align}
      \n& \\underbrace{\\max}_{\\omega,b} {\\frac{\\hat{\\gamma}}{||\\omega||}} \\
      \n& s.t. \\quad y_i(\\omega{x_i}+b)\\geq\\hat{\\gamma}, \\quad  i=1,2,\\ldots,m
      \n\\end{align}
      \n$$
      \n  \u5982\u679c\u5c06$\\omega$\u548c$b$\u6309\u6bd4\u4f8b\u6539\u53d8\u6210$\\lambda{\\omega}$\u548c$\\lambda{b}$\uff0c\u6b64\u65f6\u7684\u51fd\u6570\u95f4\u9694\u4e3a$\\lambda{\\hat{\\gamma}}$\uff0c\u5373$y_i(\\omega{xi}+b)\\geq\\hat{\\gamma}$\u4e00\u5b9a\u6210\u7acb\uff0c\u56e0\u6b64\u51fd\u6570\u95f4\u9694$\\hat{\\gamma}$\u5e76\u4e0d\u5f71\u54cd\u6700\u4f18\u5316\u95ee\u9898\u7684\u89e3\uff0c\u5373\u51fd\u6570\u95f4\u9694\u5bf9\u4e0a\u9762\u7684\u6700\u4f18\u5316\u95ee\u9898\u7684\u4e0d\u7b49\u5f0f\u7ea6\u675f\u6ca1\u6709\u5f71\u54cd\uff0c\u56e0\u6b64\u53ef\u4ee5\u53d6$\\hat{\\gamma}=1$\u3002\u8fd9\u6837\u6700\u4f18\u5316\u95ee\u9898\u53d8\u6210\u4e86
      \n$$
      \n\\begin{align}
      \n& \\underbrace{\\max}<\/em>{\\omega,b} {\\frac{1}{||\\omega||}} \\
      \n& s.t. \\quad y_i(\\omega{x_i}+b)\\geq1, \\quad  i=1,2,\\ldots,m
      \n\\end{align}
      \n$$
      \n  \u53ef\u4ee5\u770b\u51fa\u8fd9\u4e2a\u6700\u4f18\u5316\u95ee\u9898\u548c\u611f\u77e5\u673a\u7684\u4f18\u5316\u95ee\u9898\u662f\u4e0d\u540c\u7684\uff0c\u611f\u77e5\u673a\u662f\u56fa\u5b9a\u5206\u6bcd\u4f18\u5316\u5206\u5b50\uff0c\u800cSVM\u5728\u52a0\u4e0a\u4e86\u652f\u6301\u5411\u91cf\u7684\u540c\u65f6\u56fa\u5b9a\u5206\u5b50\u4f18\u5316\u5206\u6bcd\u3002<\/p>\n
        \u6ce8\u610f\u6700\u5927\u5316${\\frac{1}{||\\omega||}}$\u5373\u6700\u5c0f\u5316$||\\omega|||$\uff0c\u8003\u8651\u5230\u4e8c\u8303\u6570\u7684\u6027\u8d28\uff0c\u56e0\u6b64\u52a0\u4e2a\u5e73\u65b9\uff0c\u5373\u6700\u5c0f\u5316${\\frac{1}{2}}{||\\omega||}^2$\uff0c\u5219\u53ef\u4ee5\u5f97\u5230\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u7684\u6700\u4f18\u5316\u95ee\u9898\uff0c\u5373\u76ee\u6807\u51fd\u6570\u7684\u6700\u4f18\u5316\u95ee\u9898\uff0c\u5373\u786c\u95f4\u9694\u6700\u5927\u5316\u4e3a
      \n$$
      \n\\begin{align}
      \n& \\underbrace{\\min}_{\\omega,b} {\\frac{1}{2}}{||\\omega||}^2 \\
      \n& s.t. \\quad y_i(\\omega{x_i}+b)\\geq1, \\quad  i=1,2,\\ldots,m
      \n\\end{align}
      \n$$
      \n\u5176\u4e2d${\\frac{1}{2}}{||\\omega||}^2$\u4e3a\u76ee\u6807\u51fd\u6570<\/p>\n
      \u51f8\u6700\u4f18\u5316\u95ee\u9898<\/h2>\n
        \u7531\u4e8e\u76ee\u6807\u51fd\u6570\u7684\u6700\u4f18\u5316\u95ee\u9898\u4e2d\u7684\u76ee\u6807\u51fd\u6570\u662f\u8fde\u7eed\u53ef\u5fae\u7684\u51f8\u51fd\u6570\uff0c\u7ea6\u675f\u51fd\u6570\u662f\u4eff\u5c04\u51fd\u6570\uff08\u6ce8\uff1a\u5982\u679c$f(x)=a*x+b$\uff0c\u5219$f(x)$\u79f0\u4e3a\u4eff\u5c04\u51fd\u6570\uff09\uff0c\u5219\u8be5\u95ee\u9898\u662f\u4e00\u4e2a\u51f8\u6700\u4f18\u5316\u95ee\u9898\u3002\u53c8\u7531\u4e8e\u76ee\u6807\u51fd\u6570\u662f\u4e8c\u6b21\u51fd\u6570\uff0c\u5219\u8be5\u51f8\u6700\u4f18\u5316\u95ee\u9898\u53d8\u6210\u4e86\u51f8\u4e8c\u6b21\u89c4\u5212\u95ee\u9898\u3002<\/p>\n
        \u5982\u679c\u6c42\u51fa\u4e86\u76ee\u6807\u51fd\u6570\u6700\u4f18\u5316\u95ee\u9898\u4e2d\u7684\u89e3$\\omega^,b^<\/em>$\uff0c\u5219\u53ef\u4ee5\u5f97\u5230\u6700\u5927\u95f4\u9694\u5206\u79bb\u8d85\u5e73\u9762${\\omega^}^Tx+b^<\/em>=0$\u548c\u5206\u7c7b\u51b3\u7b56\u51fd\u6570$f(x)=sign({\\omega^}^Tx+b^<\/em>)$\uff08\u6ce8\uff1asign\u51fd\u6570\u5373\u7b26\u53f7\u51fd\u6570\uff0c\u7c7b\u4f3c\u4e8eSigmoid\u51fd\u6570\u7684\u56fe\u5f62\uff09\uff0c\u5373\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u6a21\u578b\u3002<\/p>\n
      \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u7684\u6700\u4f18\u5316\u95ee\u9898<\/h2>\n
        \u6839\u636e\u51f8\u4f18\u5316\u7406\u8bba\uff0c\u53ef\u4ee5\u901a\u8fc7\u62c9\u683c\u6717\u65e5\u51fd\u6570\u628a\u4f18\u5316\u6709\u7ea6\u675f\u7684\u76ee\u6807\u51fd\u6570\u8f6c\u5316\u4e3a\u4f18\u5316\u65e0\u7ea6\u675f\u7684\u76ee\u6807\u51fd\u6570\u3002\u65e2\u5e94\u7528\u62c9\u683c\u6717\u65e5\u5bf9\u5076\u6027\uff0c\u901a\u8fc7\u6c42\u89e3\u5bf9\u5076\u95ee\u9898\u5f97\u5230\u539f\u59cb\u95ee\u9898\u7684\u6700\u4f18\u89e3\uff0c\u8fdb\u800c\u628a\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u7684\u6700\u4f18\u5316\u95ee\u9898\u4f5c\u4e3a\u539f\u59cb\u6700\u4f18\u5316\u95ee\u9898\uff0c\u6709\u65f6\u4e5f\u79f0\u8be5\u65b9\u6cd5\u4e3a\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u7684\u5bf9\u5076\u7b97\u6cd5\uff08dual algorithm\uff09\u3002<\/p>\n
        \u9996\u5148\u5f15\u8fdb\u62c9\u683c\u6717\u65e5\u4e58\u5b50\uff08Lagrange multiplier\uff09$\\alphai\\geq0, \\quad i=1,2,\\ldots,m$\uff0c\u7136\u540e\u6784\u5efa\u62c9\u683c\u6717\u65e5\u51fd\u6570\uff08Lagrange function\uff09
      \n$$
      \nL(\\omega,b,\\alpha) = {\\frac{1}{2}}{||\\omega||}^2-\\sum<\/em>{i=1}^m\\alpha_iy_i(\\omega{xi}+b)+\\sum<\/em>{i=1}^m\\alpha_i
      \n$$
      \n\u5176\u4e2d$\\alpha=(\\alpha_1,\\alpha_2,\\ldots,\\alpha_m)^T$\u4e3a\u62c9\u683c\u6717\u65e5\u4e58\u5b50\u5411\u91cf\u3002<\/p>\n
        \u56e0\u6b64\u4f18\u5316\u95ee\u9898\u53d8\u6210
      \n$$
      \n\\underbrace{\\min}{\\omega,b} \\underbrace{\\max}<\/em>{\\alphai\\geq0} L(\\omega,b,\\alpha)
      \n$$
      \n  \u7531\u4e8e\u8fd9\u4e2a\u4f18\u5316\u95ee\u9898\u6ee1\u8db3Karush-Kuhn-Tucker(KKT)\u6761\u4ef6,\u65e2\u53ef\u4ee5\u901a\u8fc7\u62c9\u683c\u6717\u65e5\u5bf9\u5076\u6027\u628a\u4e0a\u8ff0\u7684\u4f18\u5316\u95ee\u9898\u8f6c\u5316\u4e3a\u7b49\u4ef7\u7684\u5bf9\u5076\u95ee\u9898\uff0c\u5373\u4f18\u5316\u76ee\u6807\u53d8\u6210
      \n$$
      \n\\underbrace{\\max}<\/em>{\\alphai\\geq0} \\underbrace{\\min}<\/em>{\\omega,b} L(\\omega,b,\\alpha)
      \n$$
      \n\u4ece\u4e0a\u5f0f\u4e2d\uff0c\u5219\u53ef\u4ee5\u5148\u6c42\u4f18\u5316\u51fd\u6570\u5bf9\u4e8e$\\omega$\u548c$b$\u7684\u6781\u5c0f\u503c\uff0c\u63a5\u7740\u518d\u6c42\u62c9\u683c\u6717\u65e5\u4e58\u5b50$\\alpha$\u7684\u6781\u5927\u503c\u3002<\/p>\n
        \n
      1. \u6c42$\\underbrace{\\min}_{\\omega,b}L(\\omega,b,\\alpha)$<\/li>\n<\/ol>\n
          \u901a\u8fc7\u5bf9$\\omega$\u548c$b$\u5206\u522b\u6c42\u504f\u5bfc\u5e76\u4ee4\u5176\u7b49\u4e8e$0$\u5373\u53ef\u5f97$L(\\omega,b,a)$\u7684\u6781\u5c0f\u503c
        \n$$
        \n\\begin{align}
        \n& \\nabla\\omega{L}(\\omega,b,\\alpha) = \\omega – \\sum<\/em>{i=1}^m\\alpha_iy_ix_i = 0 \\
        \n& \\nablabL(\\omega,b,\\alpha) = -\\sum<\/em>{i=1}^m\\alpha_iyi =0
        \n\\end{align}
        \n$$
        \n\u5f97
        \n$$
        \n\\begin{align}
        \n& \\omega = \\sum<\/em>{i=1}^m \\alpha_iy_ixi \\
        \n& \\sum<\/em>{i=1}^m \\alpha_iy_i = 0
        \n\\end{align}
        \n$$
        \n  \u4ece$\\omega$\u548c$b$\u6c42\u504f\u5bfc\u7b49\u4e8e$0$\u7684\u7ed3\u679c\u53ef\u4ee5\u770b\u51fa$\\omega$\u548c$\\alpha$\u7684\u5173\u7cfb\uff0c\u53ea\u8981\u540e\u9762\u80fd\u63a5\u7740\u6c42\u51fa\u4f18\u5316\u51fd\u6570\u6781\u5927\u5316\u5bf9\u5e94\u7684$\\alpha$\uff0c\u5373\u53ef\u4ee5\u6c42\u51fa$\\omega$\uff0c\u7531\u4e8e\u4e0a\u8ff0\u4e0a\u5f0f\u5df2\u7ecf\u6ca1\u6709\u4e86$b$\uff0c\u56e0\u6b64\u6700\u540e\u7684$b$\u53ef\u80fd\u6709\u591a\u4e2a\u3002<\/p>\n
          \u5c06\u4e0a\u8ff0\u5f0f\u5b50\u5373\u53ef\u4ee3\u5165\u62c9\u683c\u6717\u65e5\u51fd\u6570\uff08\u6ce8\uff1a\u7531\u4e8e\u63a8\u5bfc\u8fc7\u7a0b\u5341\u5206\u590d\u6742\uff0c\u5bf9\u63a5\u4e0b\u6765\u7684\u8bb2\u89e3\u65e0\u610f\uff0c\u6b64\u5904\u4e0d\u7ed9\u51fa\u63a8\u5bfc\u8fc7\u7a0b\uff0c\u6709\u5174\u8da3\u7684\u540c\u5b66\u53ef\u4ee5\u81ea\u884c\u5c1d\u8bd5\uff0c\u5176\u4e2d\u4f1a\u7528\u5230\u8303\u6570\u7684\u5b9a\u4e49\u5373${||\\omega||}^2=\\omega\\omega$\u4ee5\u53ca\u4e58\u6cd5\u8fd0\u7b97\u6cd5\u5219$(a+b+c+\\ldots)(a+b+c+\\ldots)=aa+ab+ac+ba+bb+bc+\\ldots$\u4ee5\u53ca\u4e00\u4e9b\u77e9\u9635\u7684\u8fd0\u7b97\uff09\uff0c\u5373\u5f97
        \n$$
        \n\\begin{align}
        \n\\underbrace{\\min}{\\omega,b}L(\\omega,b,\\alpha) & = {\\frac{1}{2}}\\sum<\/em>{i=1}^m\\sum_{j=1}^m\\alpha_i\\alpha_jy_iy_j(x_ixj) – \\sum<\/em>{i=1}^m\\alpha_iyi((\\sum<\/em>{i=1}^m\\alpha_jy_jx_j)xi+b)+\\sum<\/em>{i=1}^m\\alphai \\
        \n& = – {\\frac{1}{2}}\\sum<\/em>{i=1}^m\\sum_{j=1}^m\\alpha_i\\alpha_jy_iy_j(x_ixj)+\\sum<\/em>{i=1}^m\\alphai
        \n\\end{align}
        \n$$
        \n  \u4ece\u4e0a\u5f0f\u53ef\u4ee5\u770b\u51fa\u901a\u8fc7\u5bf9$\\omega$\u548c$b$\u6781\u5c0f\u5316\u4ee5\u540e\uff0c\u4f18\u5316\u51fd\u6570\u53ea\u5269\u4e0b$\\alpha$\u505a\u53c2\u6570\uff0c\u53ea\u8981\u80fd\u591f\u6781\u5927\u5316$\\underbrace{\\min}<\/em>{\\omega,b}L(\\omega,b,\\alpha)$\u5373\u53ef\u6c42\u51fa\u76f8\u5e94\u7684$\\alpha$\uff0c\u8fdb\u800c\u6c42\u51fa$\\omega$\u548c$b$\u3002<\/p>\n
          \n
        1. \u6c42$\\underbrace{\\min}_{\\omega,b}L(\\omega,b,\\alpha)$\u5bf9$\\alpha$\u7684\u6781\u5927\u5316<\/li>\n<\/ol>\n
            \u5bf9$\\underbrace{\\min}{\\omega,b}L(\\omega,b,\\alpha)$\u6c42\u6781\u5927\u5316\u7684\u6570\u5b66\u8868\u8fbe\u5f0f\u4e3a
          \n$$
          \n\\begin{align}
          \n& \\underbrace{\\max}<\/em>{\\alpha} – {\\frac{1}{2}} \\sum{i=1}^m\\sum<\/em>{j=1}^m\\alpha_i\\alpha_jy_iy_j(x_ixj)+\\sum<\/em>{i=1}^m\\alphai \\
          \n& s.t. \\sum<\/em>{i=1}^m \\alpha_iy_i =0 \\
          \n& \\quad \\alphai\\geq0, \\quad i=1,2,\\ldots,m
          \n\\end{align}
          \n$$
          \n\u901a\u8fc7\u53bb\u6389\u8d1f\u53f7\uff0c\u5373\u53ef\u8f6c\u5316\u4e3a\u7b49\u4ef7\u7684\u6781\u5c0f\u95ee\u9898\u5982\u4e0b
          \n$$
          \n\\begin{align}
          \n& \\underbrace{\\min}<\/em>{\\alpha} {\\frac{1}{2}} \\sum{i=1}^m\\sum<\/em>{j=1}^m\\alpha_i\\alpha_jy_iy_j(x_ixj) – \\sum<\/em>{i=1}^m\\alphai \\
          \n& s.t. \\sum<\/em>{i=1}^m \\alpha_iy_i =0 \\
          \n& \\quad \\alphai\\geq0, \\quad i=1,2,\\ldots,m
          \n\\end{align}
          \n$$
          \n  \u4e00\u822c\u901a\u8fc7SMO\u7b97\u6cd5\u6c42\u51fa\u4e0a\u5f0f\u6781\u5c0f\u5316\u5bf9\u5e94\u7684$\\alpha$\uff0c\u5047\u8bbe\u901a\u8fc7SMO\u7b97\u6cd5\u5f97\u5230\u4e86\u8be5$\\alpha$\u503c\u8bb0\u4f5c$\\alpha^*$\uff0c\u5373\u53ef\u6839\u636e$\\omega=\\sum<\/em>{i=1}^m\\alpha_iy_ixi$\u6c42\u5f97\u539f\u59cb\u6700\u4f18\u5316\u95ee\u9898\u7684\u89e3$\\omega^$\u4e3a
          \n$$
          \n\\omega^<\/em> = \\sum<\/em>{i=1}^m {\\alpha_i}^ y_ixi
          \n$$
          \n  \u7531\u4e8e\u5bf9$b$\u6c42\u504f\u5bfc\u5f97$\\sum<\/em>{i=1}^m \\alpha_iy_i = 0$\uff0c\u56e0\u6b64\u6709\u591a\u5c11\u4e2a\u652f\u6301\u5411\u91cf\u5219\u6709\u591a\u5c11\u4e2a$b^<\/em>$\uff0c\u5e76\u4e14\u8fd9\u4e9b$b^$\u90fd\u53ef\u4ee5\u4f5c\u4e3a\u6700\u7ec8\u7684\u7ed3\u679c\uff0c\u4f46\u662f\u5bf9\u4e8e\u4e25\u683c\u7684\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\uff0c$b^<\/em>$\u7684\u503c\u662f\u552f\u4e00\u7684\uff0c\u5373\u8fd9\u91cc\u6240\u6709\u7684$b^*$\u90fd\u662f\u4e00\u6837\u7684\u3002<\/p>\n
            \u6839\u636eKKT\u6761\u4ef6\u4e2d\u7684\u5bf9\u5076\u4e92\u8865\u6761\u4ef6${\\alpha_j}^(y_j(\\omega^<\/em>x_j+b)-1)=0$\uff0c\u5982\u679c${\\alpha_j}^>0$\uff0c\u5219\u6709$y_j(\\omega^<\/em>x_j+b^)-1=0$\u5373\u70b9\u90fd\u5728\u652f\u6301\u5411\u91cf\u673a\u4e0a\uff0c\u5426\u5219\u5982\u679c${\\alpha_j}^<\/em>=0$\uff0c\u5219\u6709$y_j(\\omega^x_j+b^<\/em>)-1\\geq0$\u5373\u5df2\u88ab\u6b63\u786e\u5206\u7c7b\u3002\u7531\u4e8e\u5bf9\u4e8e\u4efb\u610f\u652f\u6301\u5411\u91cf$(x_j,y_j)$\u90fd\u6709$y_j(\\omega^x_j+b^<\/em>)-1=0$\u5373$y_j(\\omega^x_j+b^<\/em>)y_j-y_i=0$\uff0c\u4ee3\u5165$\\omega^$\u5373\u53ef\u5f97$b^<\/em>$\u4e3a
          \n$$
          \nb^ = yj – \\sum<\/em>{i=1}^m{\\alpha_i}^<\/em>y_i(x_ix_j)
          \n$$<\/p>\n
          \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u6d41\u7a0b<\/h1>\n
          \u8f93\u5165<\/h2>\n
            \u6709$m$\u4e2a\u6837\u672c\u7684\u7ebf\u6027\u53ef\u5206\u8bad\u7ec3\u96c6$T={(x_1,y_1),(x_2,y_2),\\cdots,(x_m,y_m)}$\uff0c\u5176\u4e2d$x_i$\u4e3a$n$\u7ef4\u7279\u5f81\u5411\u91cf\uff0c$y_i$\u4e3a\u4e8c\u5143\u8f93\u51fa\u5373\u503c\u4e3a$1$\u6216\u8005$-1$\u3002<\/p>\n
          \u8f93\u51fa<\/h2>\n
            \u5206\u79bb\u8d85\u5e73\u9762\u7684\u53c2\u6570$w^$\u548c$b^<\/em>$\u4ee5\u53ca\u5206\u7c7b\u51b3\u7b56\u51fd\u6570<\/p>\n
          \u6d41\u7a0b<\/h2>\n
            \n
          1. \u6784\u9020\u7ea6\u675f\u4f18\u5316\u95ee\u9898\u4e3a
            \n$$
            \n\\begin{align}
            \n& \\underbrace{\\min}{\\alpha} {\\frac{1}{2}} \\sum<\/em>{i=1}^m\\sum_{j=1}^m\\alpha_i\\alpha_jy_iy_j(x_ixj) – \\sum<\/em>{i=1}^m\\alphai \\
            \n& s.t. \\sum<\/em>{i=1}^m \\alpha_iy_i =0 \\
            \n& \\quad \\alpha_i\\geq0, \\quad i=1,2,\\ldots,m
            \n\\end{align}
            \n$$<\/li>\n
          2. \u4f7f\u7528SMO\u7b97\u6cd5\u6c42\u51fa\u4e0a\u5f0f\u6700\u5c0f\u65f6\u5bf9\u5e94\u7684$\\alpha^*$<\/li>\n
          3. \u8ba1\u7b97$w^$\u4e3a
            \n$$
            \nw^<\/em> = \\sum_{i=1}^m \\alpha^*y_ix_i
            \n$$<\/li>\n
          4. \u627e\u5230\u4e00\u4e2a\u652f\u6301\u5411\u91cf\uff0c\u5373\u6ee1\u8db3$\\alpha_s>0$\u5bf9\u5e94\u7684\u6837\u672c$(x_s,y_s)$\uff0c\u8ba1\u7b97$b^$\u4e3a
            \n$$
            \nb^<\/em> = yj – \\sum<\/em>{i=1}^m \\alpha^*y_i(x_ix_j)
            \n$$<\/li>\n
          5. \u6c42\u5f97\u5206\u79bb\u8d85\u5e73\u9762\u4e3a
            \n$$
            \nw^x+b^<\/em> = 0
            \n$$<\/li>\n
          6. \u6c42\u5f97\u5206\u7c7b\u51b3\u7b56\u51fd\u6570\u4e3a
            \n$$
            \nf(x) = sign(w^x+b^<\/em>)
            \n$$
            \n  \u5728\u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u4e2d\u53ef\u4ee5\u53d1\u73b0$w^$\u548c$b^<\/em>$\u53ea\u4f9d\u8d56\u4e8e\u8bad\u7ec3\u6570\u636e\u4e2d\u5bf9\u5e94\u7684$\\alpha^>0$\u7684\u6837\u672c\u70b9$(x_j,y_j)$\uff0c\u800c\u5176\u4ed6\u6837\u672c\u70b9\u5bf9$w^<\/em>$\u548c$b^$\u6ca1\u6709\u5f71\u54cd\uff0c\u800c$\\alpha^<\/em>>0$\u7684\u6837\u672c\u70b9\u5373\u4e3a\u652f\u6301\u5411\u91cf\u3002<\/li>\n<\/ol>\n
              \u7531KKT\u4e92\u8865\u6761\u4ef6\u53ef\u5f97\u5bf9\u4e8e$\\alpha^>0$\u6837\u672c\u70b9$(x_j,y_j)$\u6709$y_j(w^<\/em>x_j+b)-1=0$\uff0c\u5e76\u4e14\u95f4\u9694\u8fb9\u754c$H_1$\u548c$H_2$\u5206\u522b\u4e3a$w^x_j+b^<\/em>=1$\u548c$w^x_j+b^<\/em>=-1$\uff0c\u5373\u652f\u6301\u5411\u91cf\u4e00\u5b9a\u5728\u95f4\u9694\u8fb9\u754c\u4e0a\u3002<\/p>\n
            \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u4f18\u7f3a\u70b9<\/h1>\n
            \u4f18\u70b9<\/h2>\n
              \n
            1. \u51b3\u7b56\u51fd\u6570\u7684\u8ba1\u7b97\u53d6\u51b3\u4e8e\u652f\u6301\u5411\u91cf\u7684\u6570\u91cf\uff0c\u800c\u4e0d\u662f\u6837\u672c\u7a7a\u95f4\u7684\u7ef4\u6570<\/li>\n
            2. \u589e\u5220\u975e\u652f\u6301\u5411\u91cf\u5bf9\u6a21\u578b\u6ca1\u6709\u5f71\u54cd<\/li>\n<\/ol>\n
              \u7f3a\u70b9<\/h2>\n
                \n
              1. \u65e0\u6cd5\u5904\u7406\u5f02\u5e38\u70b9<\/li>\n
              2. \u4e0d\u652f\u6301\u7ebf\u6027\u4e0d\u53ef\u5206\u6570\u636e\u7684\u5206\u7c7b<\/li>\n
              3. \u5c5e\u4e8e\u4e8c\u5206\u7c7b\u5206\u7c7b\u5668\uff0c\u5982\u679c\u9700\u8981\u591a\u5206\u7c7b\u9700\u8981\u4f7f\u7528OVR\u7b49\u65b9\u6cd5<\/li>\n<\/ol>\n
                \u5c0f\u7ed3<\/h1>\n
                  \u652f\u6301\u5411\u91cf\u673a\u662f\u57fa\u4e8e\u611f\u77e5\u673a\u6a21\u578b\u6f14\u5316\u800c\u6765\u7684\uff0c\u89e3\u51b3\u4e86\u611f\u77e5\u673a\u6a21\u578b\u53ef\u80fd\u4f1a\u5f97\u5230\u591a\u4e2a\u5206\u7c7b\u76f4\u7ebf\u7684\u95ee\u9898\uff0c\u7531\u4e8e\u4f7f\u7528\u4e86\u786c\u95f4\u9694\u6700\u5927\u5316\u652f\u6301\u5411\u91cf\u673a\u7684\u76ee\u6807\u51fd\u6570\u53ea\u548c\u652f\u6301\u5411\u91cf\u7684\u4f4d\u7f6e\u6709\u5173\uff0c\u5927\u5927\u964d\u4f4e\u4e86\u8ba1\u7b97\u91cf\u3002<\/p>\n
                  \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u662f\u652f\u6301\u5411\u91cf\u673a\u6700\u539f\u59cb\u7684\u5f62\u5f0f\uff0c\u7531\u4e8e\u4f7f\u7528\u4e86\u786c\u95f4\u9694\u6700\u5927\u5316\uff0c\u56e0\u6b64\u65e0\u6cd5\u505a\u5230\u5bf9\u5f02\u5e38\u503c\u548c\u975e\u7ebf\u6027\u53ef\u5206\u6570\u636e\u7684\u5904\u7406\uff0c\u6b64\u5904\u4e0d\u591a\u8d58\u8ff0\uff0c\u8ba9\u6211\u4eec\u770b\u4ed6\u7684\u5347\u7ea7\u7248\u2014\u2014\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"
                \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u5b66\u4e60\u76ee\u6807 \u7ebf\u6027\u53ef\u5206\u652f\u6301\u5411\u91cf\u673a\u3001\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\u3001\u975e\u7ebf\u6027\u652f\u6301\u5411\u91cf\u673a\u533a\u522b \u51fd\u6570 […]<\/p>\n","protected":false},"author":3,"featured_media":3094,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[276,291],"tags":[],"_links":{"self":[{"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/posts\/3093"}],"collection":[{"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3093"}],"version-history":[{"count":0,"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/posts\/3093\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/media\/3094"}],"wp:attachment":[{"href":"https:\/\/egonlin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3093"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3093"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3093"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}