{"id":3151,"date":"2022-02-27T12:28:32","date_gmt":"2022-02-27T04:28:32","guid":{"rendered":"https:\/\/egonlin.com\/?p=3151"},"modified":"2022-02-27T12:36:09","modified_gmt":"2022-02-27T04:36:09","slug":"%e7%ac%ac%e4%b8%80%e7%af%87%ef%bc%9a%e6%8f%90%e5%8d%87%e6%a0%91","status":"publish","type":"post","link":"https:\/\/egonlin.com\/?p=3151","title":{"rendered":"\u7b2c\u4e00\u7bc7\uff1a\u63d0\u5347\u6811"},"content":{"rendered":"

\u63d0\u5347\u6811<\/h1>\n

  \u63d0\u5347\u6811(boosting tree)\u662f\u4ee5\u5206\u7c7b\u6811\u6216\u56de\u5f52\u6811\u4f5c\u4e3a\u5f31\u5b66\u4e60\u5668\u7684\u5f3a\u5b66\u4e60\u5668\u3002<\/p>\n

  \u63d0\u5347\u6811\u6a21\u578b\u7528\u7684\u662f\u52a0\u6cd5\u6a21\u578b\uff0c\u7b97\u6cd5\u7528\u7684\u662f\u524d\u5411\u5206\u6b65\u7b97\u6cd5\uff0c\u5f31\u5b66\u4e60\u5668\u662f\u51b3\u7b56\u6811\u7684\u96c6\u6210\u5b66\u4e60\u65b9\u6cd5\u3002<\/p>\n

\u63d0\u5347\u6811\u5b66\u4e60\u76ee\u6807<\/h1>\n
    \n
  1. \u52a0\u6cd5\u6a21\u578b<\/li>\n
  2. \u524d\u5411\u5206\u6b65\u7b97\u6cd5<\/li>\n
  3. \u63d0\u5347\u6811\u4e0eAdaBoost\u7b97\u6cd5<\/li>\n
  4. \u56de\u5f52\u63d0\u5347\u6811\u6d41\u7a0b<\/li>\n
  5. \u63d0\u5347\u6811\u4f18\u7f3a\u70b9<\/li>\n<\/ol>\n

    \u63d0\u5347\u6811\u5f15\u5165<\/h1>\n

      \u5047\u8bbeNick\u7684\u5e74\u9f84\u662f25\u5c81\u3002<\/p>\n

      \n
    1. \u7b2c1\u68f5\u51b3\u7b56\u6811<\/li>\n<\/ol>\n

      \u628aNick\u7684\u5e74\u9f84\u8bbe\u7f6e\u6210\u521d\u59cb\u503c0\u5c81\u53bb\u5b66\u4e60\uff0c\u5982\u679c\u7b2c1\u68f5\u51b3\u7b56\u6811\u9884\u6d4bNick\u7684\u5e74\u9f84\u662f12\u5c81\uff0c\u5373\u6b8b\u5dee\u503c\u4e3a$25-12=13$<\/p>\n

        \n
      1. \u7b2c2\u8bfe\u51b3\u7b56\u6811\n
          \n
        1. \u628aNick\u7684\u5e74\u9f84\u8bbe\u7f6e\u6210\u6b8b\u5dee\u503c13\u5c81\u53bb\u5b66\u4e60\uff0c\u5982\u679c\u7b2c2\u68f5\u51b3\u7b56\u6811\u80fd\u628aNick\u5206\u523013\u5c81\u7684\u53f6\u5b50\u8282\u70b9\uff0c\u7d2f\u52a0\u4e24\u68f5\u51b3\u7b56\u6811\u7684\u9884\u6d4b\u503c\u52a0\u548c$12+13=25$\uff0c\u5c31\u662fNick\u7684\u771f\u5b9e\u5e74\u9f8425\u5c81<\/li>\n
        2. \u5982\u679c\u7b2c2\u68f5\u51b3\u7b56\u6811\u7684\u5f97\u5230\u7684\u662f10\u5c81\uff0c\u6b8b\u5dee\u503c\u4e3a$25-12-10=3$<\/li>\n<\/ol>\n<\/li>\n
        3. \u7b2c3\u8bfe\u51b3\u7b56\u6811<\/li>\n<\/ol>\n

          \u628aNick\u7684\u5e74\u9f84\u8bbe\u7f6e\u6210\u6b8b\u5dee\u503c3\u5c81\u53bb\u5b66\u4e60\u2026\u2026<\/p>\n

            \n
          1. \u7ee7\u7eed\u91cd\u590d\u4e0a\u8ff0\u8fc7\u7a0b\u5b66\u4e60\uff0c\u4e0d\u65ad\u903c\u8fd1Nick\u7684\u771f\u5b9e\u5e74\u9f84<\/li>\n<\/ol>\n

            \u63d0\u5347\u6811\u8be6\u89e3<\/h1>\n

            \u52a0\u6cd5\u6a21\u578b<\/h2>\n

              \u63d0\u5347\u6811\u6a21\u578b\u53ef\u4ee5\u8868\u793a\u4e3a\u51b3\u7b56\u6811\u7684\u52a0\u6cd5\u6a21\u578b
            \n$$
            \nfM(x)=\\sum<\/em>{i=1}^MT(x;\\theta_m)
            \n$$
            \n\u5176\u4e2d$T(x;\\theta_m)$\u8868\u793a\u51b3\u7b56\u6811\uff1b$\\theta_m$\u8868\u793a\u51b3\u7b56\u6811\u7684\u53c2\u6570\uff1b$M$\u4e3a\u6811\u7684\u4e2a\u6570\u3002<\/p>\n

            \u524d\u5411\u5206\u6b65\u7b97\u6cd5<\/h2>\n

              \u63d0\u5347\u6811\u6a21\u578b\u4f7f\u7528\u7684\u662f\u524d\u5411\u5206\u5e03\u7b97\u6cd5\uff0c\u5373\u5047\u8bbe\u521d\u59cb\u63d0\u5347\u6811$f_0(x)=0$\uff0c\u7b2c$m$\u6b65\u7684\u6a21\u578b\u662f
            \n$$
            \nfm(x)=f<\/em>{m-1}(x)+T(x;\\thetam)
            \n$$
            \n\u5176\u4e2d$f<\/em>{m-1}(x)$\u4e3a\u5f53\u524d\u6a21\u578b\uff0c\u901a\u8fc7\u7ecf\u9a8c\u98ce\u9669\u6781\u5c0f\u5316\u786e\u5b9a\u4e00\u4e0b\u8bfe\u51b3\u7b56\u6811\u7684\u53c2\u6570$\\theta_m$
            \n$$
            \n\\hat{\\thetam}=\\underbrace{arg\\,min}<\/em>{\\thetam}\\sum<\/em>{i=1}^mL(yi,f<\/em>{m-1}(x_i)+T(x_i;\\theta_m))
            \n$$<\/p>\n

            \u63d0\u5347\u6811\u4e0eAdaBoost\u7b97\u6cd5<\/h2>\n

              AdaBoost\u7b97\u6cd5\u4f7f\u7528\u7684\u662f\u524d\u5411\u5206\u6b65\u7b97\u6cd5\uff0c\u5229\u7528\u524d\u4e00\u8f6e\u5f31\u5b66\u4e60\u5668\u7684\u8bef\u5dee\u7387\u66f4\u65b0\u8bad\u7ec3\u6570\u636e\u7684\u6743\u91cd\uff1b\u63d0\u5347\u6811\u4f7f\u7528\u7684\u4e5f\u662f\u524d\u5411\u5206\u6b65\u7b97\u6cd5\uff0c\u4f46\u662f\u63d0\u5347\u6811\u5982\u5176\u540d\uff0c\u4ed6\u7684\u5f31\u5b66\u4e60\u5668\u53ea\u80fd\u4f7f\u7528\u51b3\u7b56\u6811\uff0c\u4e00\u822c\u4f7f\u7528CART\u6811\uff0c\u7136\u540e\u4ed6\u7684\u8fed\u4ee3\u601d\u8def\u4e5f\u4e0eAdaBoost\u7b97\u6cd5\u4e0d\u540c<\/p>\n

              \u5047\u8bbe\u63d0\u5347\u6811\u5728$t-1$\u8f6e\u7684\u5f3a\u5b66\u4e60\u5668\u4e3a$f{t-1}(x)$\uff0c\u76ee\u6807\u51fd\u6570\u662f
            \n$$
            \nL(y,f<\/em>{m-1}(x))
            \n$$
            \n  \u5728\u7b2c$t$\u8f6e\u7684\u76ee\u6807\u5219\u662f\u627e\u5230\u4e00\u4e2a\u5f31\u5b66\u4e60\u5668(\u51b3\u7b56\u6811)$h_t(x)$\uff0c\u6700\u5c0f\u5316\u7b2c$t$\u8f6e\u7684\u76ee\u6807\u51fd\u6570
            \n$$
            \nL(y,f_m(x))=L(yi,f<\/em>{m-1}(x)+T(x;\\theta_m))
            \n$$
            \n  \u4f46\u662f\u5f53AdaBoost\u7b97\u6cd5\u4e2d\u7684\u5f31\u5b66\u4e60\u5668\u4e3a\u4e8c\u7c7b\u5206\u7c7b\u6811\u7684\u65f6\u5019\uff0c\u5176\u5b9eAdaBoost\u5c31\u662f\u63d0\u5347\u6811\uff0c\u5373\u53ef\u4ee5\u8bf4\u5206\u7c7b\u63d0\u5347\u6811\u7b97\u6cd5\u662fAdaBoost\u7b97\u6cd5\u7684\u4e00\u79cd\u7279\u6b8a\u60c5\u51b5\u3002<\/p>\n

            \u56de\u5f52\u63d0\u5347\u6811<\/h2>\n

              \u6709$m$\u4e2a\u6570\u636e$n$\u4e2a\u7279\u5f81\u7684\u8bad\u7ec3\u6570\u636e\u96c6$T={(x_,y_1),(x_2,y_2),\\cdots,(x_m,y_m)}$\uff0c\u5982\u679c\u5c06\u8f93\u5165\u7a7a\u95f4\u5212\u5206\u4e3a$k$\u4e92\u4e0d\u76f8\u4ea4\u7684\u533a\u57df$R_1,R_2,\\cdots,R_j$\uff0c\u5e76\u4e14\u5728\u6bcf\u4e2a\u533a\u57df\u4e0a\u786e\u5b9a\u8f93\u51fa\u7684\u5e38\u91cf$cj$\uff0c\u51b3\u7b56\u6811\u53ef\u4ee5\u8868\u793a\u4e3a
            \n$$
            \nT(x;\\theta)=\\sum<\/em>{j=1}^Jc_jI(x\\in{R_j})
            \n$$
            \n\u5176\u4e2d\uff0c$\\theta={(R_1,c_1),(R_2,c_2),\\cdots,(R_J,c_J)}\u8868\u793a\u6811\u7684\u533a\u57df\u5212\u5206\u548c\u5404\u533a\u57df\u4e0a\u7684\u5e38\u6570\uff0c$J$\u662f\u56de\u5f52\u6811\u7684\u53f6\u8282\u70b9\u4e2a\u6570\u3002<\/p>\n

            \u524d\u5411\u5206\u6b65\u7b97\u6cd5<\/h3>\n

            $$
            \n\\begin{align}
            \n& f_0(x)=0 \\
            \n& f_1(x)=f_0(x)+T(x;\\theta_1) \\
            \n& \\cdots \\
            \n& fm(x)=f<\/em>{m-1}(x)+T(x,\\theta_m),m=1,2,\\cdots,M \\
            \n& fM(x)=\\sum<\/em>{m=1}^MT(x;\\theta_m)
            \n\\end{align}
            \n$$
            \n  \u5728\u7b2c$m$\u6b65$fm(x)=f<\/em>{m-1}(x)+T(x,\\thetam)$\u7684\u65f6\u5019\uff0c\u7ed9\u5b9a\u4e86$f<\/em>{m-1}(x)$\uff0c\u9700\u8981\u6c42\u89e3\u7b2c$m$\u68f5\u7684\u53c2\u6570$\\hat{\\theta_m}$
            \n$$
            \n\\hat{\\thetam} = \\underbrace{arg\\,min}<\/em>{\\thetam}\\sum<\/em>{i=1}^mL(yi,f<\/em>{m-1}(x_i)+T(x_i;\\theta_m))
            \n$$<\/p>\n

            \u5e73\u65b9\u8bef\u5dee\u635f\u5931\u51fd\u6570<\/h3>\n

              \u5bf9\u4e8e\u7b2c$m$\u68f5\u6811\u7684\u53c2\u6570$\\hat{\\thetam}$\uff0c\u53ef\u4ee5\u91c7\u7528\u5e73\u65b9\u8bef\u5dee\u635f\u5931\u51fd\u6570$L(y,f(x))=(y-f(x))^2$\u6c42\u89e3\uff0c\u6811\u7684\u635f\u5931\u53d8\u4e3a
            \n$$
            \n\\begin{align}
            \nL(y,f<\/em>{m-1}(x)+T(x;\\thetam)) & = [y-f<\/em>{m-1}(x)-T(x;\\theta_m)]^2 \\
            \n& = [r-T(x;\\thetam)]^2
            \n\\end{align}
            \n$$
            \n\u5176\u4e2d$r=y-f<\/em>{m-1}(x)$\u662f\u5f53\u524d\u6a21\u578b\u62df\u5408\u6570\u636e\u7684\u6b8b\u5dee\u3002<\/p>\n

              \u5bf9\u4e8e\u56de\u5f52\u63d0\u5347\u6811\uff0c\u53ea\u9700\u7b80\u5355\u5730\u62df\u5408\u5f53\u524d\u6a21\u578b\u7684\u6b8b\u5dee\u3002<\/p>\n

            \u56de\u5f52\u63d0\u5347\u6811\u6d41\u7a0b<\/h1>\n

            \u8f93\u5165<\/h2>\n

              \u6709$m$\u4e2a\u6570\u636e$n$\u4e2a\u7279\u5f81\u7684\u8bad\u7ec3\u6570\u636e\u96c6$T={(x_,y_1),(x_2,y_2),\\cdots,(x_m,y_m)}$\u3002<\/p>\n

            \u8f93\u51fa<\/h2>\n

              \u56de\u5f52\u63d0\u5347\u6811$f_M(x)$\u3002<\/p>\n

            \u6d41\u7a0b<\/h1>\n
              \n
            1. \u521d\u59cb\u5316$f_0(x)=0$<\/li>\n
            2. \u5bf9$m=1,2,\\cdots,M$\n
                \n
              1. \u8ba1\u7b97\u6b8b\u5dee$r_{mi}=yi-f<\/em>{m-1}(x_i),\\quad{i=1,2,\\cdots,m}$<\/li>\n
              2. \u62df\u5408\u6b8b\u5dee$r_{mi}$\u5b66\u4e60\u4e00\u4e2a\u56de\u5f52\u6811\uff0c\u5f97\u5230$T(x;\\theta_m)$<\/li>\n
              3. \u66f4\u65b0$fm(x)=f<\/em>{m-1}(x)+T(x;\\theta_m)$<\/li>\n<\/ol>\n<\/li>\n
              4. \u5f97\u5230\u56de\u5f52\u63d0\u5347\u6811
                \n$$
                \nfM(x)=\\sum<\/em>{i=1}^MT(x;\\theta_m)
                \n$$<\/li>\n<\/ol>\n

                \u63d0\u5347\u6811\u4f18\u7f3a\u70b9<\/h1>\n

                \u4f18\u70b9<\/h2>\n
                  \n
                1. \u65e2\u53ef\u4ee5\u89e3\u51b3\u5206\u7c7b\u95ee\u9898\uff0c\u53c8\u53ef\u4ee5\u89e3\u51b3\u56de\u5f52\u95ee\u9898<\/li>\n<\/ol>\n

                  \u7f3a\u70b9<\/h2>\n
                    \n
                  1. \u5f31\u5b66\u4e60\u5668\u4e4b\u95f4\u5b58\u5728\u4f9d\u8d56\u5173\u7cfb\uff0c\u96be\u4ee5\u5e76\u884c\u8bad\u7ec3<\/li>\n
                  2. \u63d0\u5347\u6811\u53ea\u662f\u7b80\u5355\u7684\u62df\u5408\u6a21\u578b\u7684\u6b8b\u5dee\uff0c\u5e76\u4e0d\u51c6\u786e<\/li>\n<\/ol>\n

                    \u5c0f\u7ed3<\/h1>\n

                      \u63d0\u5347\u6811\u5c5e\u4e8eBoosting\u7cfb\u5217\u7b97\u6cd5\uff0c\u4ed6\u548cAdaBoost\u6709\u76f8\u4f3c\u4e4b\u5904\u7684\uff0c\u5e76\u4e14\u5f53AdaBoost\u7b97\u6cd5\u4e2d\u7684\u5f31\u5b66\u4e60\u5668\u4e3a\u4e8c\u7c7b\u5206\u7c7b\u6811\u7684\u65f6\u5019\uff0c\u68af\u5ea6\u63d0\u5347\u6811\u5c31\u662f\u4e00\u79cd\u7279\u6b8a\u7684AdaBoost\u7b97\u6cd5\u3002<\/p>\n

                      \u7531\u4e8e\u63d0\u5347\u6811\u662f\u7531\u7b80\u5355\u7684\u6b8b\u5dee\u8ba1\u7b97\u5f97\u5230\u7684\uff0c\u6240\u4ee5\u5728\u67d0\u79cd\u7a0b\u5ea6\u4e0a\u6765\u8bf4\uff0c\u63d0\u5347\u6811\u662f\u6709\u4e00\u5b9a\u7f3a\u9677\u7684\uff0c\u4e3a\u4e86\u89e3\u51b3\u8fd9\u4e2a\u95ee\u9898\uff0c\u4e00\u822c\u4f1a\u91c7\u7528\u68af\u5ea6\u63d0\u5347\u6811\u6765\u5f25\u8865\u3002<\/p>\n","protected":false},"excerpt":{"rendered":"

                    \u63d0\u5347\u6811   \u63d0\u5347\u6811(boosting tree)\u662f\u4ee5\u5206\u7c7b\u6811\u6216\u56de\u5f52\u6811\u4f5c\u4e3a\u5f31\u5b66\u4e60\u5668\u7684\u5f3a\u5b66\u4e60\u5668\u3002 […]<\/p>\n","protected":false},"author":3,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":[],"categories":[276,296],"tags":[],"_links":{"self":[{"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/posts\/3151"}],"collection":[{"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/users\/3"}],"replies":[{"embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=3151"}],"version-history":[{"count":0,"href":"https:\/\/egonlin.com\/index.php?rest_route=\/wp\/v2\/posts\/3151\/revisions"}],"wp:attachment":[{"href":"https:\/\/egonlin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=3151"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=3151"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/egonlin.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=3151"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}