{"id":188,"date":"2025-07-01T08:58:25","date_gmt":"2025-07-01T00:58:25","guid":{"rendered":"https:\/\/mianka.xyz\/?p=188"},"modified":"2025-07-01T09:08:51","modified_gmt":"2025-07-01T01:08:51","slug":"pytorch","status":"publish","type":"post","link":"https:\/\/www.mianka.xyz\/?p=188","title":{"rendered":"PyTorch\u81ea\u5b9a\u4e49\u6a21\u578b\u8bbe\u8ba1\u4e0e\u4f18\u5316\u5668\u9009\u62e9\u5168\u653b\u7565\uff1a\u4ece\u67b6\u6784\u8bbe\u8ba1\u5230\u8bad\u7ec3\u7b56\u7565"},"content":{"rendered":"<h2 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 24px; border: none; line-height: 1.6; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">\u4e00\u3001\u81ea\u5b9a\u4e49\u6a21\u578b\u8bbe\u8ba1\uff1a\u4ece\u9700\u6c42\u5230\u67b6\u6784<\/span><\/h2>\n<h3 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 20px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">1.1 \u6a21\u578b\u8bbe\u8ba1\u6838\u5fc3\u539f\u5219<\/span><\/h3>\n<ol style=\"box-sizing: border-box; padding: 0px 0px 0px 30px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style-position: outside; list-style-image: none; margin-top: 14px; margin-bottom: 1em; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\" class=\" list-paddingleft-2\">\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u4efb\u52a1\u9002\u914d\u6027<\/span><\/p>\n<\/li>\n<ul style=\"list-style-type: disc;\" class=\" list-paddingleft-2\">\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">CV\u4efb\u52a1<\/span>\uff1a\u4f18\u5148\u9009\u62e9\u5377\u79ef\u795e\u7ecf\u7f51\u7edc\uff08CNN\uff09\u6216\u89c6\u89c9Transformer\uff08ViT\uff09\uff0c\u5229\u7528\u5c40\u90e8\u611f\u53d7\u91ce\u4e0e\u5e73\u79fb\u4e0d\u53d8\u6027\u3002<\/p>\n<\/li>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">NLP\u4efb\u52a1<\/span>\uff1a\u91c7\u7528Transformer\u6216\u5faa\u73af\u795e\u7ecf\u7f51\u7edc\uff08RNN\uff09\uff0c\u6355\u6349\u957f\u8ddd\u79bb\u4f9d\u8d56\u5173\u7cfb\u3002<\/p>\n<\/li>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u751f\u6210\u6a21\u578b<\/span>\uff1aGAN\u6216\u6269\u6563\u6a21\u578b\u9700\u8bbe\u8ba1\u751f\u6210\u5668-\u5224\u522b\u5668\u5bf9\u79f0\u7ed3\u6784\uff0c\u6216U-Net\u7b49\u7f16\u7801\u5668-\u89e3\u7801\u5668\u67b6\u6784\u3002<\/p>\n<\/li>\n<\/ul>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u6a21\u5757\u5316\u4e0e\u53ef\u6269\u5c55\u6027<\/span><\/p>\n<\/li>\n<ul style=\"list-style-type: disc;\" class=\" list-paddingleft-2\">\n<li>\n<p>\u4f7f\u7528<code class=\" inline\" style=\"box-sizing: border-box; padding: 1px 4px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin: 0px 2px; scrollbar-width: none; font-size: 12.75px; tab-size: 4; background-color: rgba(27, 31, 35, 0.05); border-radius: 3px;\">nn.Module<\/code>\u5c01\u88c5\u53ef\u590d\u7528\u7ec4\u4ef6\uff08\u5982\u6b8b\u5dee\u5757\u3001\u6ce8\u610f\u529b\u5c42\uff09\uff0c\u901a\u8fc7\u7ee7\u627f\u4e0e\u7ec4\u5408\u5feb\u901f\u6784\u5efa\u590d\u6742\u6a21\u578b\u3002<\/p>\n<\/li>\n<li>\n<p><span style=\"color: #06071F; font-family: PingFang-SC-Regular; font-size: 15px; text-wrap-mode: wrap; background-color: #FDFDFE;\">\u793a\u4f8b\uff1a\u81ea\u5b9a\u4e49\u6b8b\u5dee\u5757<\/span><\/p>\n<\/li>\n<li>\n<pre class=\"prism-highlight prism-language-python\">class&nbsp;ResidualBlock(nn.Module):\n&nbsp;&nbsp;&nbsp;&nbsp;def&nbsp;__init__(self,&nbsp;in_channels):\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;super().__init__()\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.conv1&nbsp;=&nbsp;nn.Conv2d(in_channels,&nbsp;in_channels,&nbsp;kernel_size=3,&nbsp;padding=1)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.bn1&nbsp;=&nbsp;nn.BatchNorm2d(in_channels)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.relu&nbsp;=&nbsp;nn.ReLU(inplace=True)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.conv2&nbsp;=&nbsp;nn.Conv2d(in_channels,&nbsp;in_channels,&nbsp;kernel_size=3,&nbsp;padding=1)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.bn2&nbsp;=&nbsp;nn.BatchNorm2d(in_channels)\n&nbsp;&nbsp;&nbsp;&nbsp;\n&nbsp;&nbsp;&nbsp;&nbsp;def&nbsp;forward(self,&nbsp;x):\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;identity&nbsp;=&nbsp;x\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out&nbsp;=&nbsp;self.conv1(x)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out&nbsp;=&nbsp;self.bn1(out)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out&nbsp;=&nbsp;self.relu(out)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out&nbsp;=&nbsp;self.conv2(out)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out&nbsp;=&nbsp;self.bn2(out)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;out&nbsp;+=&nbsp;identity&nbsp;&nbsp;#&nbsp;\u6b8b\u5dee\u8fde\u63a5\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;out<\/pre>\n<\/li>\n<\/ul>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u8ba1\u7b97\u6548\u7387\u4e0e\u5185\u5b58\u4f18\u5316<\/span><\/p>\n<\/li>\n<ul style=\"list-style-type: disc;\" class=\" list-paddingleft-2\">\n<li>\n<p>\u4f7f\u7528<code class=\" inline\" style=\"box-sizing: border-box; padding: 1px 4px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin: 0px 2px; scrollbar-width: none; font-size: 12.75px; tab-size: 4; background-color: rgba(27, 31, 35, 0.05); border-radius: 3px;\">nn.Sequential<\/code>\u7b80\u5316\u524d\u5411\u4f20\u64ad\uff0c\u907f\u514d\u5197\u4f59\u8ba1\u7b97\u3002<\/p>\n<\/li>\n<li>\n<p>\u5bf9\u9ad8\u5206\u8fa8\u7387\u8f93\u5165\uff08\u5982\u533b\u5b66\u56fe\u50cf\uff09\uff0c\u91c7\u7528\u5206\u7ec4\u5377\u79ef\uff08<code class=\" inline\" style=\"box-sizing: border-box; padding: 1px 4px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin: 0px 2px; scrollbar-width: none; font-size: 12.75px; tab-size: 4; background-color: rgba(27, 31, 35, 0.05); border-radius: 3px;\">nn.GroupConv<\/code>\uff09\u6216\u6df1\u5ea6\u53ef\u5206\u79bb\u5377\u79ef\uff08<code class=\" inline\" style=\"box-sizing: border-box; padding: 1px 4px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin: 0px 2px; scrollbar-width: none; font-size: 12.75px; tab-size: 4; background-color: rgba(27, 31, 35, 0.05); border-radius: 3px;\">nn.SeparableConv2d<\/code>\uff09\u964d\u4f4e\u53c2\u6570\u91cf\u3002<\/p>\n<\/li>\n<\/ul>\n<\/ol>\n<p><\/p>\n<h3 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 20px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">1.2 \u5178\u578b\u573a\u666f\u6a21\u578b\u8bbe\u8ba1\u6848\u4f8b<\/span><\/h3>\n<h4 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 18px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">\u6848\u4f8b1\uff1a\u8f7b\u91cf\u5316\u56fe\u50cf\u5206\u7c7b\u6a21\u578b<\/span><\/h4>\n<p id=\"\" style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style: none; margin-top: 0px; margin-bottom: 0px; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u9700\u6c42<\/span>\uff1a\u5728\u5d4c\u5165\u5f0f\u8bbe\u5907\u4e0a\u90e8\u7f72MNIST\u5206\u7c7b\u6a21\u578b\uff0c\u8981\u6c42\u53c2\u6570\u91cf&lt;100K\u3002<br style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 6px; scrollbar-width: none; content: &quot; &quot;; display: block;\"\/><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u8bbe\u8ba1<\/span>\uff1a<\/p>\n<ul style=\"box-sizing: border-box; padding: 0px 0px 0px 30px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style-position: outside; list-style-image: none; margin-top: 14px; margin-bottom: 1em; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\" class=\" list-paddingleft-2\">\n<li>\n<p>\u4f7f\u7528\u6df1\u5ea6\u53ef\u5206\u79bb\u5377\u79ef\u66ff\u4ee3\u6807\u51c6\u5377\u79ef\uff0c\u51cf\u5c11\u53c2\u6570\u91cf\u3002<\/p>\n<\/li>\n<li>\n<p>\u63d2\u5165\u901a\u9053\u6ce8\u610f\u529b\u6a21\u5757\uff08SE Block\uff09\u63d0\u5347\u7279\u5f81\u8868\u8fbe\u80fd\u529b\u3002<\/p>\n<p><\/p>\n<\/li>\n<li>\n<pre class=\"prism-highlight prism-language-python\">class&nbsp;LightCNN(nn.Module):\n&nbsp;&nbsp;&nbsp;&nbsp;def&nbsp;__init__(self):\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;super().__init__()\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.conv1&nbsp;=&nbsp;nn.Conv2d(1,&nbsp;32,&nbsp;kernel_size=3,&nbsp;padding=1)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.dw_conv&nbsp;=&nbsp;nn.Sequential(\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nn.Conv2d(32,&nbsp;32,&nbsp;kernel_size=3,&nbsp;padding=1,&nbsp;groups=32),&nbsp;&nbsp;#&nbsp;\u6df1\u5ea6\u5377\u79ef\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nn.Conv2d(32,&nbsp;64,&nbsp;kernel_size=1),&nbsp;&nbsp;#&nbsp;\u70b9\u5377\u79ef\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nn.BatchNorm2d(64),\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;nn.ReLU()\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.se_block&nbsp;=&nbsp;SEBlock(64)&nbsp;&nbsp;#&nbsp;\u81ea\u5b9a\u4e49SE\u6ce8\u610f\u529b\u6a21\u5757\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.fc&nbsp;=&nbsp;nn.Linear(64*7*7,&nbsp;10)\n&nbsp;&nbsp;&nbsp;&nbsp;\n&nbsp;&nbsp;&nbsp;&nbsp;def&nbsp;forward(self,&nbsp;x):\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=&nbsp;torch.relu(self.conv1(x))\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=&nbsp;self.dw_conv(x)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=&nbsp;self.se_block(x)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=&nbsp;torch.flatten(x,&nbsp;1)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;self.fc(x)<\/pre>\n<\/li>\n<\/ul>\n<h4 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 18px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">\u6848\u4f8b2\uff1a\u57fa\u4e8eTransformer\u7684\u6587\u672c\u751f\u6210\u6a21\u578b<\/span><\/h4>\n<p id=\"\" style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style: none; margin-top: 0px; margin-bottom: 0px; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u9700\u6c42<\/span>\uff1a\u6784\u5efa\u4e00\u4e2a\u53ef\u751f\u6210\u77ed\u6587\u672c\u7684Transformer\u89e3\u7801\u5668\u6a21\u578b\u3002<br style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 6px; scrollbar-width: none; content: &quot; &quot;; display: block;\"\/><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u8bbe\u8ba1<\/span>\uff1a<\/p>\n<ul style=\"box-sizing: border-box; padding: 0px 0px 0px 30px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style-position: outside; list-style-image: none; margin-top: 14px; margin-bottom: 1em; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\" class=\" list-paddingleft-2\">\n<li>\n<p>\u91c7\u7528\u81ea\u56de\u5f52\u7ed3\u6784\uff0c\u63a9\u7801\u591a\u5934\u6ce8\u610f\u529b\u9632\u6b62\u4fe1\u606f\u6cc4\u9732\u3002<\/p>\n<\/li>\n<li>\n<p>\u4f7f\u7528\u76f8\u5bf9\u4f4d\u7f6e\u7f16\u7801\u66ff\u4ee3\u7edd\u5bf9\u4f4d\u7f6e\u7f16\u7801\uff0c\u63d0\u5347\u957f\u5e8f\u5217\u5efa\u6a21\u80fd\u529b\u3002<\/p>\n<\/li>\n<li>\n<pre class=\"prism-highlight prism-language-python\">class&nbsp;TextGenerator(nn.Module):\n&nbsp;&nbsp;&nbsp;&nbsp;def&nbsp;__init__(self,&nbsp;vocab_size,&nbsp;d_model=512,&nbsp;nhead=8,&nbsp;num_layers=6):\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;super().__init__()\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;encoder_layer&nbsp;=&nbsp;nn.TransformerEncoderLayer(\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;d_model=d_model,&nbsp;nhead=nhead,&nbsp;dim_feedforward=2048\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.transformer&nbsp;=&nbsp;nn.TransformerEncoder(encoder_layer,&nbsp;num_layers=num_layers)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.embedding&nbsp;=&nbsp;nn.Embedding(vocab_size,&nbsp;d_model)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.pos_encoder&nbsp;=&nbsp;PositionalEncoding(d_model)&nbsp;&nbsp;#&nbsp;\u81ea\u5b9a\u4e49\u4f4d\u7f6e\u7f16\u7801\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.decoder&nbsp;=&nbsp;nn.Linear(d_model,&nbsp;vocab_size)\n&nbsp;&nbsp;&nbsp;&nbsp;\n&nbsp;&nbsp;&nbsp;&nbsp;def&nbsp;forward(self,&nbsp;src):\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;src&nbsp;=&nbsp;self.embedding(src)&nbsp;*&nbsp;math.sqrt(self.d_model)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;src&nbsp;=&nbsp;self.pos_encoder(src)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;output&nbsp;=&nbsp;self.transformer(src)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;self.decoder(output)<\/pre>\n<\/li>\n<\/ul>\n<hr style=\"box-sizing: content-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 20px 0px; scrollbar-width: none; height: 0px; overflow: visible; border-right: 0px; border-bottom: 0px; border-left: 0px; border-image: initial; border-top-style: solid; border-top-color: rgb(230, 230, 235); color: rgb(6, 7, 31); font-size: 15px; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"\/>\n<h2 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 24px; border: none; line-height: 1.6; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">\u4e8c\u3001\u4f18\u5316\u5668\u9009\u62e9\uff1a\u7406\u8bba\u3001\u5b9e\u8df5\u4e0e\u8c03\u53c2<\/span><\/h2>\n<h3 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 20px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">2.1 \u4f18\u5316\u5668\u6838\u5fc3\u7279\u6027\u5bf9\u6bd4<\/span><\/h3>\n<p class=\"XurfUklW table-scroll\" style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style: none; margin-top: 20px; margin-bottom: 0px; overflow-x: auto; width: 741.806px; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254); scrollbar-width: thin !important;\"><\/p>\n<table class=\"OxEWNITQ\" width=\"NaN\">\n<thead style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none;\">\n<tr style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none;\" class=\"firstRow\">\n<th style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Medium; list-style: none; margin: 0px; scrollbar-width: none; color: rgb(0, 24, 70); font-weight: 500; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; border-left-color: rgba(231, 229, 238, 0.7); border-top-left-radius: 10px; background: rgba(207, 222, 254, 0.2) !important; text-align: left !important; vertical-align: top !important;\">\u4f18\u5316\u5668<\/th>\n<th style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Medium; list-style: none; margin: 0px; scrollbar-width: none; color: rgb(0, 24, 70); font-weight: 500; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; background: rgba(207, 222, 254, 0.2) !important; text-align: left !important; vertical-align: top !important;\">\u9002\u7528\u573a\u666f<\/th>\n<th style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Medium; list-style: none; margin: 0px; scrollbar-width: none; color: rgb(0, 24, 70); font-weight: 500; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; background: rgba(207, 222, 254, 0.2) !important; text-align: left !important; vertical-align: top !important;\">\u4f18\u52bf<\/th>\n<th style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Medium; list-style: none; margin: 0px; scrollbar-width: none; color: rgb(0, 24, 70); font-weight: 500; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; border-top-right-radius: 10px; background: rgba(207, 222, 254, 0.2) !important; text-align: left !important; vertical-align: top !important;\">\u52a3\u52bf<\/th>\n<\/tr>\n<\/thead>\n<tbody style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none;\">\n<tr style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none; background-color: rgb(253, 253, 254);\">\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-left-color: rgba(231, 229, 238, 0.7); vertical-align: top !important;\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">SGD<\/span><\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u5927\u89c4\u6a21\u6570\u636e\u3001\u7b80\u5355\u6a21\u578b\uff08\u5982ResNet\uff09<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u8bad\u7ec3\u7a33\u5b9a\uff0c\u6cdb\u5316\u80fd\u529b\u5f3a<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u9700\u7cbe\u7ec6\u8c03\u53c2\uff0c\u6536\u655b\u6162<\/td>\n<\/tr>\n<tr style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none; background-color: rgb(253, 253, 254);\">\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-left-color: rgba(231, 229, 238, 0.7); vertical-align: top !important;\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">Adam<\/span><\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u590d\u6742\u6a21\u578b\uff08\u5982Transformer\u3001GAN\uff09<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u81ea\u9002\u5e94\u5b66\u4e60\u7387\uff0c\u52a0\u901f\u6536\u655b<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u53ef\u80fd\u8fc7\u62df\u5408\uff0c\u6cdb\u5316\u6027\u7a0d\u5f31<\/td>\n<\/tr>\n<tr style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none; background-color: rgb(253, 253, 254);\">\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-left-color: rgba(231, 229, 238, 0.7); vertical-align: top !important;\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">AdamW<\/span><\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u9884\u8bad\u7ec3\u6a21\u578b\uff08\u5982BERT\u3001GPT\uff09<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u89e3\u8026\u6743\u91cd\u8870\u51cf\uff0c\u7a33\u5b9a\u8bad\u7ec3<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u5bf9\u5b66\u4e60\u7387\u654f\u611f<\/td>\n<\/tr>\n<tr style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none; background-color: rgb(253, 253, 254);\">\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-left-color: rgba(231, 229, 238, 0.7); vertical-align: top !important;\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">RAdam<\/span><\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u8bad\u7ec3\u521d\u671f\u68af\u5ea6\u4e0d\u7a33\u5b9a\uff08\u5982GAN\u751f\u6210\u5668\uff09<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u52a8\u6001\u8c03\u6574\u52a8\u91cf\u8303\u56f4\uff0c\u89e3\u51b3\u51b7\u542f\u52a8<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u8ba1\u7b97\u5f00\u9500\u7565\u9ad8<\/td>\n<\/tr>\n<tr style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none; background-color: rgb(253, 253, 254);\">\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-left-color: rgba(231, 229, 238, 0.7); vertical-align: top !important;\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">LAMB<\/span><\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u8d85\u5927\u89c4\u6a21\u6a21\u578b\uff08\u5982GPT-3\u3001ViT\uff09<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u5206\u5c42\u81ea\u9002\u5e94\u5b66\u4e60\u7387\uff0c\u652f\u6301\u767e\u4ebf\u53c2\u6570<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); vertical-align: top !important;\">\u9700\u8c03\u6574\u03b2\u53c2\u6570<\/td>\n<\/tr>\n<tr style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; list-style: none; margin: 0px; scrollbar-width: none; background-color: rgb(253, 253, 254);\">\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-bottom-left-radius: 10px; vertical-align: top !important;\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">Adafactor<\/span><\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-bottom-color: rgba(231, 229, 238, 0.7); vertical-align: top !important;\">\u5185\u5b58\u53d7\u9650\u573a\u666f\uff08\u5982\u957f\u5e8f\u5217RNN\uff09<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-bottom-color: rgba(231, 229, 238, 0.7); vertical-align: top !important;\">\u56e0\u5f0f\u5206\u89e3\u68af\u5ea6\u77e9\u9635\uff0c\u51cf\u5c11\u5b58\u50a8<\/td>\n<td style=\"box-sizing: border-box; padding: 10px 20px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Regular; list-style: none; margin: 0px; scrollbar-width: none; border-right-color: rgba(231, 229, 238, 0.7); border-top-color: rgba(231, 229, 238, 0.7); letter-spacing: 0px; max-width: 300px; color: rgb(18, 6, 73); border-bottom-color: rgba(231, 229, 238, 0.7); border-bottom-right-radius: 10px; vertical-align: top !important;\">\u6536\u655b\u901f\u5ea6\u8f83\u6162<\/td>\n<\/tr>\n<\/tbody>\n<\/table>\n<p><\/p>\n<h3 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 20px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 20px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">2.2 \u573a\u666f\u5316\u4f18\u5316\u5668\u9009\u62e9\u7b56\u7565<\/span><\/h3>\n<h4 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 18px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">\u573a\u666f1\uff1a\u8ba1\u7b97\u673a\u89c6\u89c9\uff08ResNet-50\u8bad\u7ec3\uff09<\/span><\/h4>\n<ul style=\"box-sizing: border-box; padding: 0px 0px 0px 30px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style-position: outside; list-style-image: none; margin-top: 14px; margin-bottom: 1em; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\" class=\" list-paddingleft-2\">\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u63a8\u8350\u4f18\u5316\u5668<\/span>\uff1aSGD + Momentum\uff08\u5b66\u4e60\u73870.1\uff0c\u52a8\u91cf0.9\uff09 + \u4f59\u5f26\u9000\u706b\u8c03\u5ea6\u5668<\/p>\n<\/li>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u539f\u56e0<\/span>\uff1a<\/p>\n<\/li>\n<ul style=\"list-style-type: square;\" class=\" list-paddingleft-2\">\n<li>\n<p>SGD\u7684\u968f\u673a\u68af\u5ea6\u4e0b\u964d\u7279\u6027\u53ef\u907f\u514d\u9677\u5165\u5c40\u90e8\u6700\u4f18\uff0c\u63d0\u5347\u6cdb\u5316\u80fd\u529b\u3002<\/p>\n<\/li>\n<li>\n<p>\u4f59\u5f26\u9000\u706b\u52a8\u6001\u8c03\u6574\u5b66\u4e60\u7387\uff0c\u5e73\u8861\u8bad\u7ec3\u521d\u671f\u4e0e\u672b\u671f\u7684\u6536\u655b\u901f\u5ea6\u3002<\/p>\n<\/li>\n<\/ul>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u4ee3\u7801\u793a\u4f8b<\/span>\uff1a<\/p>\n<pre style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin-top: 8px; margin-bottom: 8px; scrollbar-width: none; font-size: 1em; overflow: auto; position: relative;\">optimizer&nbsp;=&nbsp;torch.optim.SGD(model.parameters(),&nbsp;lr=0.1,&nbsp;momentum=0.9,&nbsp;weight_decay=1e-4)\nscheduler&nbsp;=&nbsp;torch.optim.lr_scheduler.CosineAnnealingLR(optimizer,&nbsp;T_max=90,&nbsp;eta_min=0)<\/pre>\n<\/li>\n<\/ul>\n<h4 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 18px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">\u573a\u666f2\uff1a\u81ea\u7136\u8bed\u8a00\u5904\u7406\uff08BERT\u5fae\u8c03\uff09<\/span><\/h4>\n<ul style=\"box-sizing: border-box; padding: 0px 0px 0px 30px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style-position: outside; list-style-image: none; margin-top: 14px; margin-bottom: 1em; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\" class=\" list-paddingleft-2\">\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u63a8\u8350\u4f18\u5316\u5668<\/span>\uff1aAdamW\uff08\u5b66\u4e60\u73875e-5\uff0c\u03b21=0.9\uff0c\u03b22=0.999\uff09 + \u7ebf\u6027\u9884\u70ed\u8c03\u5ea6\u5668<\/p>\n<\/li>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u539f\u56e0<\/span>\uff1a<\/p>\n<\/li>\n<ul style=\"list-style-type: square;\" class=\" list-paddingleft-2\">\n<li>\n<p>AdamW\u7684\u81ea\u9002\u5e94\u5b66\u4e60\u7387\u53ef\u7f13\u89e3\u5fae\u8c03\u9636\u6bb5\u68af\u5ea6\u6d88\u5931\u95ee\u9898\u3002<\/p>\n<\/li>\n<li>\n<p>\u7ebf\u6027\u9884\u70ed\u9010\u6b65\u63d0\u5347\u5b66\u4e60\u7387\uff0c\u907f\u514d\u521d\u671f\u8bad\u7ec3\u4e0d\u7a33\u5b9a\u3002<\/p>\n<\/li>\n<\/ul>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u4ee3\u7801\u793a\u4f8b<\/span>\uff1a<\/p>\n<pre style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin-top: 8px; margin-bottom: 8px; scrollbar-width: none; font-size: 1em; overflow: auto; position: relative;\">optimizer&nbsp;=&nbsp;torch.optim.AdamW(model.parameters(),&nbsp;lr=5e-5,&nbsp;weight_decay=0.01)\nscheduler&nbsp;=&nbsp;torch.optim.lr_scheduler.LinearLR(optimizer,&nbsp;start_factor=0.01,&nbsp;end_factor=1.0,&nbsp;total_iters=1000)<\/pre>\n<\/li>\n<\/ul>\n<h4 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 18px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">\u573a\u666f3\uff1a\u751f\u6210\u5bf9\u6297\u7f51\u7edc\uff08DCGAN\u8bad\u7ec3\uff09<\/span><\/h4>\n<ul style=\"box-sizing: border-box; padding: 0px 0px 0px 30px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style-position: outside; list-style-image: none; margin-top: 14px; margin-bottom: 1em; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\" class=\" list-paddingleft-2\">\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u63a8\u8350\u4f18\u5316\u5668<\/span>\uff1a<\/p>\n<\/li>\n<ul style=\"list-style-type: square;\" class=\" list-paddingleft-2\">\n<li>\n<p>\u751f\u6210\u5668\uff1aAdam\uff08\u03b21=0.0\uff0c\u03b22=0.999\uff09<\/p>\n<\/li>\n<li>\n<p>\u5224\u522b\u5668\uff1aRMSprop\uff08\u5b66\u4e60\u73870.0002\uff09<\/p>\n<\/li>\n<\/ul>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u539f\u56e0<\/span>\uff1a<\/p>\n<\/li>\n<ul style=\"list-style-type: square;\" class=\" list-paddingleft-2\">\n<li>\n<p>\u751f\u6210\u5668\u9700\u4f4e\u03b21\u6291\u5236\u521d\u59cb\u68af\u5ea6\u9707\u8361\uff0c\u9ad8\u03b22\u7a33\u5b9a\u540e\u671f\u8bad\u7ec3\u3002<\/p>\n<\/li>\n<li>\n<p>\u5224\u522b\u5668\u4f7f\u7528RMSprop\u907f\u514d\u8fc7\u65e9\u6536\u655b\u5bfc\u81f4\u6a21\u5f0f\u5d29\u6e83\u3002<\/p>\n<\/li>\n<\/ul>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u4ee3\u7801\u793a\u4f8b<\/span>\uff1a<\/p>\n<pre style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin-top: 8px; margin-bottom: 8px; scrollbar-width: none; font-size: 1em; overflow: auto; position: relative;\">optimizer_G&nbsp;=&nbsp;torch.optim.Adam(generator.parameters(),&nbsp;lr=0.0002,&nbsp;betas=(0.0,&nbsp;0.999))\noptimizer_D&nbsp;=&nbsp;torch.optim.RMSprop(discriminator.parameters(),&nbsp;lr=0.0002,&nbsp;alpha=0.9)<\/pre>\n<\/li>\n<\/ul>\n<h3 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 20px; border: none; line-height: 1.7; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">2.3 \u9ad8\u7ea7\u8c03\u53c2\u6280\u5de7<\/span><\/h3>\n<ol style=\"box-sizing: border-box; padding: 0px 0px 0px 30px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style-position: outside; list-style-image: none; margin-top: 14px; margin-bottom: 1em; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\" class=\" list-paddingleft-2\">\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u68af\u5ea6\u88c1\u526a<\/span>\uff1a\u9632\u6b62RNN\u6216GAN\u4e2d\u68af\u5ea6\u7206\u70b8<\/p>\n<pre style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin-top: 8px; margin-bottom: 8px; scrollbar-width: none; font-size: 1em; overflow: auto; position: relative;\">torch.nn.utils.clip_grad_norm_(model.parameters(),&nbsp;max_norm=1.0)<\/pre>\n<\/li>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u5b66\u4e60\u7387\u9884\u70ed<\/span>\uff1a\u7ed3\u5408\u7ebf\u6027\u9884\u70ed\u4e0e\u4f59\u5f26\u9000\u706b<\/p>\n<pre style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin-top: 8px; margin-bottom: 8px; scrollbar-width: none; font-size: 1em; overflow: auto; position: relative;\">scheduler&nbsp;=&nbsp;torch.optim.lr_scheduler.SequentialLR(\n&nbsp;&nbsp;&nbsp;&nbsp;optimizer,\n&nbsp;&nbsp;&nbsp;&nbsp;schedulers=[\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;torch.optim.lr_scheduler.LinearLR(optimizer,&nbsp;start_factor=0.01,&nbsp;end_factor=1.0,&nbsp;total_iters=1000),\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;torch.optim.lr_scheduler.CosineAnnealingLR(optimizer,&nbsp;T_max=89000,&nbsp;eta_min=0)\n&nbsp;&nbsp;&nbsp;&nbsp;],\n&nbsp;&nbsp;&nbsp;&nbsp;milestones=[1000]\n)<\/pre>\n<\/li>\n<li>\n<p><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none; font-weight: 600;\">\u81ea\u9002\u5e94\u6279\u91cf\u5f52\u4e00\u5316<\/span>\uff1a\u5bf9\u5c0f\u6279\u91cf\u6570\u636e\uff08\u5982\u533b\u5b66\u56fe\u50cf\uff09\u4f7f\u7528<code class=\" inline\" style=\"box-sizing: border-box; padding: 1px 4px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin: 0px 2px; scrollbar-width: none; font-size: 12.75px; tab-size: 4; background-color: rgba(27, 31, 35, 0.05); border-radius: 3px;\">SyncBatchNorm<\/code>\u591aGPU\u540c\u6b65\u7edf\u8ba1\u91cf<\/p>\n<pre style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: SFMono-Regular, Consolas, &quot;Liberation Mono&quot;, Menlo, Courier, monospace; list-style: none; margin-top: 8px; margin-bottom: 8px; scrollbar-width: none; font-size: 1em; overflow: auto; position: relative;\">model&nbsp;=&nbsp;nn.SyncBatchNorm.convert_sync_batchnorm(model)<\/pre>\n<\/li>\n<\/ol>\n<hr style=\"box-sizing: content-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 20px 0px; scrollbar-width: none; height: 0px; overflow: visible; border-right: 0px; border-bottom: 0px; border-left: 0px; border-image: initial; border-top-style: solid; border-top-color: rgb(230, 230, 235); color: rgb(6, 7, 31); font-size: 15px; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"\/>\n<h2 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 24px; border: none; line-height: 1.6; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><span style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: -apple-system, BlinkMacSystemFont, &quot;Segoe UI&quot;, Roboto, Ubuntu, &quot;Helvetica Neue&quot;, Helvetica, Arial, &quot;PingFang SC&quot;, &quot;Hiragino Sans GB&quot;, &quot;Microsoft YaHei UI&quot;, &quot;Microsoft YaHei&quot;, &quot;Source Han Sans CN&quot;, sans-serif, &quot;Apple Color Emoji&quot;, &quot;Segoe UI Emoji&quot;; list-style: none; margin: 0px; scrollbar-width: none;\">\u4e09\u3001\u5b8c\u6574\u8bad\u7ec3\u6d41\u7a0b\u793a\u4f8b<\/span><\/h2>\n<p id=\"\" style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFang-SC-Regular; list-style: none; margin-top: 0px; margin-bottom: 0px; scrollbar-width: none; font-size: 15px; line-height: 1.7; color: rgb(6, 7, 31); text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\">\u4ee5CIFAR-10\u5206\u7c7b\u4efb\u52a1\u4e3a\u4f8b\uff0c\u6574\u5408\u81ea\u5b9a\u4e49\u6a21\u578b\u4e0e\u4f18\u5316\u5668\uff1a<\/p>\n<pre class=\"prism-highlight prism-language-python\">import&nbsp;torch\nimport&nbsp;torch.nn&nbsp;as&nbsp;nn\nimport&nbsp;torch.optim&nbsp;as&nbsp;optim\nfrom&nbsp;torchvision&nbsp;import&nbsp;datasets,&nbsp;transforms\n\n#&nbsp;1.&nbsp;\u5b9a\u4e49\u81ea\u5b9a\u4e49\u6a21\u578b\nclass&nbsp;CustomCNN(nn.Module):\n&nbsp;&nbsp;&nbsp;&nbsp;def&nbsp;__init__(self):\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;super().__init__()\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.conv1&nbsp;=&nbsp;nn.Conv2d(3,&nbsp;32,&nbsp;kernel_size=3,&nbsp;padding=1)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.conv2&nbsp;=&nbsp;nn.Conv2d(32,&nbsp;64,&nbsp;kernel_size=3,&nbsp;padding=1)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.pool&nbsp;=&nbsp;nn.MaxPool2d(2,&nbsp;2)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.fc1&nbsp;=&nbsp;nn.Linear(64&nbsp;*&nbsp;8&nbsp;*&nbsp;8,&nbsp;512)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;self.fc2&nbsp;=&nbsp;nn.Linear(512,&nbsp;10)\n&nbsp;&nbsp;&nbsp;&nbsp;\n&nbsp;&nbsp;&nbsp;&nbsp;def&nbsp;forward(self,&nbsp;x):\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=&nbsp;self.pool(torch.relu(self.conv1(x)))\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=&nbsp;self.pool(torch.relu(self.conv2(x)))\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=&nbsp;torch.flatten(x,&nbsp;1)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;x&nbsp;=&nbsp;torch.relu(self.fc1(x))\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;return&nbsp;self.fc2(x)\n\n#&nbsp;2.&nbsp;\u6570\u636e\u52a0\u8f7d\u4e0e\u9884\u5904\u7406\ntransform&nbsp;=&nbsp;transforms.Compose([\n&nbsp;&nbsp;&nbsp;&nbsp;transforms.ToTensor(),\n&nbsp;&nbsp;&nbsp;&nbsp;transforms.Normalize((0.5,&nbsp;0.5,&nbsp;0.5),&nbsp;(0.5,&nbsp;0.5,&nbsp;0.5))\n])\ntrain_set&nbsp;=&nbsp;datasets.CIFAR10(root=&#39;.\/data&#39;,&nbsp;train=True,&nbsp;download=True,&nbsp;transform=transform)\ntrain_loader&nbsp;=&nbsp;torch.utils.data.DataLoader(train_set,&nbsp;batch_size=64,&nbsp;shuffle=True)\n\n#&nbsp;3.&nbsp;\u521d\u59cb\u5316\u6a21\u578b\u3001\u635f\u5931\u51fd\u6570\u4e0e\u4f18\u5316\u5668\nmodel&nbsp;=&nbsp;CustomCNN()\ncriterion&nbsp;=&nbsp;nn.CrossEntropyLoss()\noptimizer&nbsp;=&nbsp;optim.SGD(model.parameters(),&nbsp;lr=0.01,&nbsp;momentum=0.9,&nbsp;weight_decay=5e-4)\nscheduler&nbsp;=&nbsp;optim.lr_scheduler.StepLR(optimizer,&nbsp;step_size=30,&nbsp;gamma=0.1)\n\n#&nbsp;4.&nbsp;\u8bad\u7ec3\u5faa\u73af\nfor&nbsp;epoch&nbsp;in&nbsp;range(100):\n&nbsp;&nbsp;&nbsp;&nbsp;for&nbsp;inputs,&nbsp;labels&nbsp;in&nbsp;train_loader:\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer.zero_grad()\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;outputs&nbsp;=&nbsp;model(inputs)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;loss&nbsp;=&nbsp;criterion(outputs,&nbsp;labels)\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;loss.backward()\n&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;optimizer.step()\n&nbsp;&nbsp;&nbsp;&nbsp;scheduler.step()\n&nbsp;&nbsp;&nbsp;&nbsp;print(f&#39;Epoch&nbsp;{epoch},&nbsp;Loss:&nbsp;{loss.item():.4f}&#39;)<\/pre>\n<p><\/p>\n","protected":false},"excerpt":{"rendered":"<h2 style=\"box-sizing: border-box; padding: 0px; -webkit-font-smoothing: antialiased; font-family: PingFangSC-Semibold; list-style: none; margin: 14px 0px 8px; scrollbar-width: none; color: rgb(5, 7, 59); font-size: 24px; border: none; line-height: 1.6; text-wrap-mode: wrap; background-color: rgb(253, 253, 254);\"><!--autointro--><\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[3],"tags":[],"class_list":["post-188","post","type-post","status-publish","format-standard","hentry","category-pythonbiji"],"_links":{"self":[{"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=\/wp\/v2\/posts\/188","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=188"}],"version-history":[{"count":0,"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=\/wp\/v2\/posts\/188\/revisions"}],"wp:attachment":[{"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=188"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=188"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.mianka.xyz\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=188"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}