{"id":1338,"date":"2026-01-20T10:13:46","date_gmt":"2026-01-20T01:13:46","guid":{"rendered":"https:\/\/rtlearner.com\/?p=1338"},"modified":"2026-01-20T10:13:48","modified_gmt":"2026-01-20T01:13:48","slug":"ai-architecture-12-resnet-skip-connection-bottleneck","status":"publish","type":"post","link":"https:\/\/rtlearner.com\/en\/ai-architecture-12-resnet-skip-connection-bottleneck\/","title":{"rendered":"AI Architecture 12. Skip Connection: ResNet and Bottlenecks"},"content":{"rendered":"\n<p class=\"wp-block-paragraph\">\uc9c0\ub09c <a href=\"https:\/\/rtlearner.com\/ai-architecture-7-mlp-layer-memory-wall\/\">MLP\uc640 \uba54\ubaa8\ub9ac \uc7a5\ubcbd<\/a>\uc5d0\uc11c \uc6b0\ub9ac\ub294 \uba54\ubaa8\ub9ac \ub300\uc5ed\ud3ed\uc774 \uc2dc\uc2a4\ud15c \uc131\ub2a5\uc744 \uc81c\ud55c\ud558\ub294 &#8216;Memory Wall&#8217; \ud604\uc0c1\uc5d0 \ub300\ud574 \ub2e4\ub8e8\uc5c8\uc2b5\ub2c8\ub2e4. \uadf8\ub9ac\uace0 <a href=\"https:\/\/rtlearner.com\/ai-architecture-8-cnn-locality-sram-data-reuse\/\">CNN\uacfc \uc9c0\uc5ed\uc131<\/a>\uc5d0\uc11c\ub294 CNN\uc774 \uc21c\ucc28\uc801\uc778 \ub370\uc774\ud130 \ucc98\ub9ac\ub97c \ud1b5\ud574 \uc774 \ubb38\uc81c\ub97c \uc6b0\uc544\ud558\uac8c \ud574\uacb0\ud588\ub294\uc9c0 \uc0b4\ud3b4\ubcf4\uc558\uc8e0.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">2015\ub144 \ub4f1\uc7a5\ud55c ResNet (Residual Network)\uc740 \ub525\ub7ec\ub2dd \uc5ed\uc0ac\uc0c1 \uac00\uc7a5 \uc704\ub300\ud55c \ubc1c\uba85\ud488 \uc911 \ud558\ub098\ub85c \uaf3d\ud799\ub2c8\ub2e4. &#8220;\uce35(Layer)\uc774 \uae4a\uc5b4\uc9c8\uc218\ub85d \ud559\uc2b5\uc774 \uc548 \ub41c\ub2e4&#8221;\ub294 \ub09c\uc81c\ub97c Skip Connection Y = F(X) + X\uc774\ub77c\ub294 \uc544\uc8fc \uac04\ub2e8\ud55c \uc544\uc774\ub514\uc5b4\ub85c \ud574\uacb0\ud588\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. \ud558\uc9c0\ub9cc \uc18c\ud504\ud2b8\uc6e8\uc5b4 \uc5d4\uc9c0\ub2c8\uc5b4\ub4e4\uc774 ResNet\uc758 \uc6b0\uc544\ud568\uc5d0 \ud658\ud638\ud560 \ub54c, \ud558\ub4dc\uc6e8\uc5b4 \uc544\ud0a4\ud14d\ud2b8\ub4e4\uc740 \uba38\ub9ac\ub97c \uac10\uc2f8 \uc950\uc5b4\uc57c \ud588\uc2b5\ub2c8\ub2e4.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\uc774 \uac04\ub2e8\ud574 \ubcf4\uc774\ub294 \ub367\uc148(+X) \ud558\ub098\uac00, \uadf8\ub3d9\uc548 \ud558\ub4dc\uc6e8\uc5b4\uac00 \uc9c0\ucf1c\uc624\ub358 \uc21c\ucc28\uc801 \uba54\ubaa8\ub9ac \uad00\ub9ac \uaddc\uce59\uc744 \uc644\uc804\ud788 \ubb34\ub108\ub728\ub838\uae30 \ub54c\ubb38\uc785\ub2c8\ub2e4. \uc774\ubc88 \uae00\uc5d0\uc11c\ub294 ResNet\uc774 \uce69 \ub0b4\ubd80\uc758 \uba54\ubaa8\ub9ac \ubc84\ud37c(Buffer)\uc640 \uc2a4\ucf00\uc904\ub7ec(Scheduler)\ub97c \uc5b4\ub5bb\uac8c \uad34\ub86d\ud788\ub294\uc9c0 \ubd84\uc11d\ud574 \ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n\n\n<style>.kb-table-of-content-nav.kb-table-of-content-id1338_9a08ff-9e .kb-table-of-content-wrap{padding-top:var(--global-kb-spacing-sm, 1.5rem);padding-right:var(--global-kb-spacing-sm, 1.5rem);padding-bottom:var(--global-kb-spacing-sm, 1.5rem);padding-left:var(--global-kb-spacing-sm, 1.5rem);box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}.kb-table-of-content-nav.kb-table-of-content-id1338_9a08ff-9e .kb-table-of-contents-title-wrap{padding-top:0px;padding-right:0px;padding-bottom:0px;padding-left:0px;}.kb-table-of-content-nav.kb-table-of-content-id1338_9a08ff-9e .kb-table-of-contents-title{font-weight:regular;font-style:normal;}.kb-table-of-content-nav.kb-table-of-content-id1338_9a08ff-9e .kb-table-of-content-wrap .kb-table-of-content-list{font-weight:regular;font-style:normal;margin-top:var(--global-kb-spacing-sm, 1.5rem);margin-right:0px;margin-bottom:0px;margin-left:0px;}@media all and (max-width: 767px){.kb-table-of-content-nav.kb-table-of-content-id1338_9a08ff-9e .kb-table-of-contents-title{font-size:var(--global-kb-font-size-md, 1.25rem);}.kb-table-of-content-nav.kb-table-of-content-id1338_9a08ff-9e .kb-table-of-content-wrap .kb-table-of-content-list{font-size:var(--global-kb-font-size-sm, 0.9rem);}}<\/style>\n\n<style>.kadence-column1338_fdce44-e3 > .kt-inside-inner-col{box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}.kadence-column1338_fdce44-e3 > .kt-inside-inner-col,.kadence-column1338_fdce44-e3 > .kt-inside-inner-col:before{border-top-left-radius:0px;border-top-right-radius:0px;border-bottom-right-radius:0px;border-bottom-left-radius:0px;}.kadence-column1338_fdce44-e3 > .kt-inside-inner-col{column-gap:var(--global-kb-gap-sm, 1rem);}.kadence-column1338_fdce44-e3 > .kt-inside-inner-col{flex-direction:column;}.kadence-column1338_fdce44-e3 > .kt-inside-inner-col > .aligncenter{width:100%;}.kadence-column1338_fdce44-e3 > .kt-inside-inner-col:before{opacity:0.3;}.kadence-column1338_fdce44-e3{position:relative;}@media all and (max-width: 1024px){.kadence-column1338_fdce44-e3 > .kt-inside-inner-col{flex-direction:column;justify-content:center;}}@media all and (max-width: 767px){.kadence-column1338_fdce44-e3 > .kt-inside-inner-col{flex-direction:column;justify-content:center;}}<\/style>\n<div class=\"wp-block-kadence-column kadence-column1338_fdce44-e3\"><div class=\"kt-inside-inner-col\">\n<p class=\"wp-block-paragraph\"><strong>\uad00\ub828 \uae00<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2705<a href=\"https:\/\/rtlearner.com\/ai-architecture-1-neuron-hardware-mac-analysis\/\" data-type=\"post\" data-id=\"1248\">AI Architecture 1. \uc778\uacf5 \ub274\ub7f0\uc758 \ud574\ubd80: silicon\uc5d0\uc11c Y=WX+B \uad6c\ud604<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2705<a href=\"https:\/\/rtlearner.com\/ai-architecture-2-activation-relu-vs-sigmoid\/\" data-type=\"post\" data-id=\"1255\">AI Architecture 2. \ud65c\uc131\ud654 \ud568\uc218\uc758 \ube44\uc6a9: ReLU vs Sigmoid<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2705<a href=\"https:\/\/rtlearner.com\/ai-architecture-3-matmul-simd-parallel-processing\/\" data-type=\"post\" data-id=\"1263\">AI Architecture 3. \ud589\ub82c\uacf1(MatMul)\uc758 \ubbf8\ud559: \ub525\ub7ec\ub2dd\uc774 GPU\/NPU\ub97c \uc120\ud0dd\ud55c \uc774\uc720<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2705<a href=\"https:\/\/rtlearner.com\/ai-architecture-4-training-vs-inference\/\" data-type=\"post\" data-id=\"1267\">AI Architecture 4. \ud559\uc2b5(Training) vs \ucd94\ub860(Inference)<\/a><\/p>\n<\/div><\/div>\n\n\n\n<h2 class=\"wp-block-heading\">1. \uc21c\ucc28\uc801 \ucc98\ub9ac (Sequentiality)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ResNet \uc774\uc804\uc758 \ubaa8\ub378\ub4e4(AlexNet, VGG)\uc740 \uc544\uc8fc \ub2e8\uc21c\ud55c \uc9c1\ub82c \uad6c\uc870(Chain Structure)\uc600\uc2b5\ub2c8\ub2e4.<\/p>\n\n\n\n<div class=\"wp-block-math\"><math display=\"block\"><semantics><mrow><mi>L<\/mi><mi>a<\/mi><mi>y<\/mi><mi>e<\/mi><mi>r<\/mi><mn>1<\/mn><mo stretchy=\"false\">\u2192<\/mo><mi>L<\/mi><mi>a<\/mi><mi>y<\/mi><mi>e<\/mi><mi>r<\/mi><mn>2<\/mn><mo stretchy=\"false\">\u2192<\/mo><mi>L<\/mi><mi>a<\/mi><mi>y<\/mi><mi>e<\/mi><mi>r<\/mi><mn>3<\/mn><mo stretchy=\"false\">\u2192<\/mo><mo>\u2026<\/mo><\/mrow><annotation encoding=\"application\/x-tex\">Layer 1 \\rightarrow Layer 2 \\rightarrow Layer 3 \\rightarrow \\dots<\/annotation><\/semantics><\/math><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">\uc774\ub294 \ud558\ub4dc\uc6e8\uc5b4 \uc785\uc7a5\uc5d0\uc11c \uba54\ubaa8\ub9ac \uad00\ub9ac\uac00 \ub108\ubb34\ub098 \uc26c\uc6b4 \uad6c\uc870\uc785\ub2c8\ub2e4.<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li>Layer 1\uc758 \ucd9c\ub825\uc744 \uba54\ubaa8\ub9ac\uc5d0 \uc501\ub2c8\ub2e4.<\/li>\n\n\n\n<li>\uadf8\uac78 \uc77d\uc5b4\uc11c Layer 2\ub97c \uacc4\uc0b0\ud569\ub2c8\ub2e4.<\/li>\n\n\n\n<li>Layer 2\uc758 \ucd9c\ub825\uc774 \ub098\uc624\ub294 \uc21c\uac04, Layer 1\uc758 \ub370\uc774\ud130\ub294 \ub36e\uc5b4\uc368\ub3c4 \ub429\ub2c8\ub2e4 (Overwrite).<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">\uc774\ub7f0 \uad6c\uc870\ub294 \ub370\uc774\ud130\uc758 \uc0dd\uba85\uc8fc\uae30(Lifetime)\uac00 \ub9e4\uc6b0 \uc9e7\uc2b5\ub2c8\ub2e4. \uc989, \uc791\uc740 \uc628\uce69 \ubc84\ud37c(SRAM) \ub450 \uac1c\ub9cc \uac00\uc9c0\uace0 \ud551\ud401(Ping-Pong) \uce58\ub4ef\uc774 \ub370\uc774\ud130\ub97c \uc8fc\uace0\ubc1b\uc73c\uba74 \uac70\ub300\ud55c \ubaa8\ub378\ub3c4 \ubb38\uc81c\uc5c6\uc774 \ub3cc\ub9b4 \uc218 \uc788\uc5c8\uc2b5\ub2c8\ub2e4.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">2. Skip Connection<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">\ud558\uc9c0\ub9cc ResNet\uc758 Skip Connection (Shortcut)\uc740 \uc774 \uaddc\uce59\uc744 \uae79\ub2c8\ub2e4.<\/p>\n\n\n\n<div class=\"wp-block-math\"><math display=\"block\"><semantics><mrow><mi>Y<\/mi><mo>=<\/mo><mi>C<\/mi><mi>o<\/mi><mi>n<\/mi><mi>v<\/mi><mo form=\"prefix\" stretchy=\"false\">(<\/mo><mi>X<\/mi><mo form=\"postfix\" stretchy=\"false\">)<\/mo><mo>+<\/mo><mi>X<\/mi><\/mrow><annotation encoding=\"application\/x-tex\">Y = Conv(X) + X<\/annotation><\/semantics><\/math><\/div>\n\n\n<style>.kb-image1338_843e44-97.kb-image-is-ratio-size, .kb-image1338_843e44-97 .kb-image-is-ratio-size{max-width:500px;width:100%;}.wp-block-kadence-column > .kt-inside-inner-col > .kb-image1338_843e44-97.kb-image-is-ratio-size, .wp-block-kadence-column > .kt-inside-inner-col > .kb-image1338_843e44-97 .kb-image-is-ratio-size{align-self:unset;}.kb-image1338_843e44-97 figure{max-width:500px;}.kb-image1338_843e44-97 .image-is-svg, .kb-image1338_843e44-97 .image-is-svg img{width:100%;}.kb-image1338_843e44-97 .kb-image-has-overlay:after{opacity:0.3;}@media all and (max-width: 767px){.kb-image1338_843e44-97.kb-image-is-ratio-size, .kb-image1338_843e44-97 .kb-image-is-ratio-size{max-width:290px;width:100%;}.kb-image1338_843e44-97 figure{max-width:290px;}}<\/style>\n<div class=\"wp-block-kadence-image kb-image1338_843e44-97\"><figure class=\"aligncenter size-full\"><img data-dominant-color=\"f1f1f1\" data-has-transparency=\"false\" style=\"--dominant-color: #f1f1f1;\" loading=\"lazy\" decoding=\"async\" width=\"771\" height=\"433\" src=\"https:\/\/rtlearner.com\/wp-content\/uploads\/2026\/01\/image-3-6.jpg\" alt=\"Skip connection\" class=\"kb-img wp-image-1339 not-transparent\" srcset=\"https:\/\/rtlearner.com\/wp-content\/uploads\/2026\/01\/image-3-6.jpg 771w, https:\/\/rtlearner.com\/wp-content\/uploads\/2026\/01\/image-3-6-300x168.jpg 300w, https:\/\/rtlearner.com\/wp-content\/uploads\/2026\/01\/image-3-6-768x431.jpg 768w, https:\/\/rtlearner.com\/wp-content\/uploads\/2026\/01\/image-3-6-18x10.jpg 18w\" sizes=\"auto, (max-width: 771px) 100vw, 771px\" \/><figcaption>Skip connection<\/figcaption><\/figure><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">\uc785\ub825 \ub370\uc774\ud130 X\ub294 Conv \uc5f0\uc0b0(\ubcf5\uc7a1\ud55c 3 * 3 \ud569\uc131\uacf1 \ub4f1)\uc744 \ud558\uae30 \uc704\ud574 \uc0ac\uc6a9\ub429\ub2c8\ub2e4. \ub3d9\uc2dc\uc5d0 X\ub294 \ub098\uc911\uc5d0 \uacb0\uacfc\uac12\uacfc \ub354\ud574\uc9c0\uae30 \uc704\ud574 \uadf8\ub300\ub85c \ub0a8\uc544 \uc788\uc5b4\uc57c \ud569\ub2c8\ub2e4. \ubb38\uc81c\ub294 Conv(X) \uc5f0\uc0b0\uc774 \uc218\ud589\ub418\ub294 \ub3d9\uc548(Latency), X\ub97c \uc5b4\ub514\uc5d4\uac00 \ubcf4\uad00\ud574\uc57c \ud55c\ub2e4\ub294 \uc810\uc785\ub2c8\ub2e4.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li>Conv(X)\uac00 \ub05d\ub0a0 \ub54c\uae4c\uc9c0 X\uc758 \uba54\ubaa8\ub9ac \uacf5\uac04\uc744 \ud574\uc81c\ud560 \uc218 \uc5c6\uc2b5\ub2c8\ub2e4.<\/li>\n\n\n\n<li>\ub370\uc774\ud130 X\uc758 Lifetime\uc774 \uac15\uc81c\ub85c \uc5f0\uc7a5\ub429\ub2c8\ub2e4.<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\uc774\ub294 \ud55c\uc815\ub41c \uc628\uce69 \uba54\ubaa8\ub9ac(SRAM) \uc790\uc6d0\uc744 \uc7a5\uc2dc\uac04 \uc810\uc720\ud558\uac8c \ub9cc\ub4e4\uc5b4, \ub2e4\ub978 \uc5f0\uc0b0\ub4e4\uc774 \uc0ac\uc6a9\ud560 \ubc84\ud37c \uacf5\uac04\uc744 \ubd80\uc871\ud558\uac8c \ub9cc\ub4ed\ub2c8\ub2e4.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">3. \uba54\ubaa8\ub9ac \uacc4\uce35\uc758 \ub51c\ub808\ub9c8: SRAM vs DRAM<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">\ub9cc\uc57d Residual Block \ub0b4\ubd80\uc758 \uc5f0\uc0b0\ub7c9\uc774 \ub9ce\uc544\uc11c X\ub97c \uc624\ub7ab\ub3d9\uc548 \ub4e4\uace0 \uc788\uc5b4\uc57c \ud558\ub294\ub370, \uce69 \ub0b4\ubd80\uc758 SRAM \uc6a9\ub7c9\uc774 \ubd80\uc871\ud558\ub2e4\uba74 \uc5b4\ub5bb\uac8c \ub420\uae4c\uc694? Architecture\ub294 \uc5b4\uca54 \uc218 \uc5c6\uc774 X\ub97c \uce69 \ubc16\uc758 <strong>DRAM\uc73c\ub85c \ucad3\uc544\ub0c8\ub2e4\uac00(Spill), \ub098\uc911\uc5d0 \ub2e4\uc2dc \uac00\uc838\uc640\uc57c(Fill)<\/strong> \ud569\ub2c8\ub2e4.<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><strong>Read X:<\/strong> Conv \uc5f0\uc0b0\uc744 \uc704\ud574 \uc77d\uc74c.<\/li>\n\n\n\n<li><strong>Spill X:<\/strong> X\ub97c \ub098\uc911\uc5d0 \ub354\ud558\uae30 \uc704\ud574 DRAM\uc5d0 \uc800\uc7a5 (SRAM \ubd80\uc871 \uc2dc).<\/li>\n\n\n\n<li><strong>Compute F(X):<\/strong> \uc5f4\uc2ec\ud788 \ud569\uc131\uacf1 \uc5f0\uc0b0 \uc218\ud589.<\/li>\n\n\n\n<li><strong>Fill X:<\/strong> \ub367\uc148(F(X)+X)\uc744 \uc704\ud574 DRAM\uc5d0\uc11c \ub2e4\uc2dc \uc77d\uc5b4\uc634.<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">\uc774 \uacfc\uc815\uc5d0\uc11c \ubd88\ud544\uc694\ud55c DRAM \ud2b8\ub798\ud53d(\ub300\uc5ed\ud3ed \uc18c\ubaa8)\uc774 \ubc1c\uc0dd\ud558\ub294 \uac83\uc785\ub2c8\ub2e4.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">4. Streaming Architecture: \ub3d9\uae30\ud654(Synchronization)<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">\ub370\uc774\ud130\ub97c \ud55c \ubc88\uc5d0 \ucc98\ub9ac\ud558\uc9c0 \uc54a\uace0 \ud30c\uc774\ud504\ub77c\uc778\uc73c\ub85c \ud758\ub824\ubcf4\ub0b4\ub294 <strong>Streaming Architecture<\/strong>\ub098 FPGA \uc124\uacc4\uc5d0\uc11c\ub294 \ub354 \ud070 \ubb38\uc81c\uac00 \ubc1c\uc0dd\ud569\ub2c8\ub2e4.<\/p>\n\n\n\n<ol start=\"1\" class=\"wp-block-list\">\n<li><strong>Main Path:<\/strong> Conv -&gt; ReLU -&gt; Conv (\uc5f0\uc0b0\uc774 \ub9ce\uc544\uc11c \ub290\ub9bc)<\/li>\n\n\n\n<li><strong>Skip Path:<\/strong> \uadf8\ub0e5 wire\ub85c \uc5f0\uacb0<\/li>\n<\/ol>\n\n\n\n<p class=\"wp-block-paragraph\">\ub450 \ub370\uc774\ud130\uac00 \ub9c8\uc9c0\ub9c9 \ub367\uc148\uae30(Adder)\uc5d0\uc11c \ub9cc\ub098\uc57c \ud558\ub294\ub370, \ub3c4\ucc29 \uc2dc\uac04\uc774 \ub2e4\ub985\ub2c8\ub2e4. Skip Path\ub85c \uc628 \ub370\uc774\ud130 X\ub294 Main Path\uc758 \uc5f0\uc0b0\uc774 \ub05d\ub0a0 \ub54c\uae4c\uc9c0 \uae30\ub2e4\ub824\uc57c \ud569\ub2c8\ub2e4.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\uc774\ub97c \uc704\ud574 \ud558\ub4dc\uc6e8\uc5b4\uc5d0\ub294 \ub370\uc774\ud130\ub97c \uc7a0\uc2dc \uac00\ub46c\ub450\ub294 <strong>FIFO (First-In-First-Out) \ubc84\ud37c<\/strong>\uac00 \ucd94\uac00\ub85c \ud544\uc694\ud569\ub2c8\ub2e4. \ubaa8\ub378\uc774 \uae4a\uc744\uc218\ub85d, \uc774\ubbf8\uc9c0 \ud574\uc0c1\ub3c4\uac00 \ud074\uc218\ub85d \uc774 FIFO\uc758 \ud06c\uae30\ub294 \uc218 \ud0ac\ub85c\ubc14\uc774\ud2b8(KB)\uc5d0\uc11c \uba54\uac00\ubc14\uc774\ud2b8(MB) \ub2e8\uc704\ub85c \ucee4\uc9c0\uba70 \uce69\uc758 \uba74\uc801\uc744 \uac09\uc544\uba39\uc2b5\ub2c8\ub2e4.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\">5. Element-wise Addition<\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">\ub9c8\uc9c0\ub9c9\uc73c\ub85c, F(X) + X\ub77c\ub294 \uc694\uc18c\ubcc4 \ub367\uc148(Element-wise Addition) \uc790\uccb4\ub3c4 \ubb38\uc81c\uc785\ub2c8\ub2e4. \uc6b0\ub9ac\ub294 \ubcf4\ud1b5 \uacf1\uc148(MAC) \ube44\uc6a9\ub9cc \ub530\uc9c0\uc9c0\ub9cc, \ub367\uc148\uc740 \uc804\ud615\uc801\uc778 Memory-Bound \uc5f0\uc0b0\uc785\ub2c8\ub2e4.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>\uc5f0\uc0b0:<\/strong> \ub367\uc148 1\ud68c<\/li>\n\n\n\n<li><strong>\uba54\ubaa8\ub9ac \uc811\uadfc:<\/strong> \uc77d\uae30 2\ud68c(F(X), X), \uc4f0\uae30 1\ud68c(Y)<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\ubcf4\uc2dc\ub2e4\uc2dc\ud53c \uc5f0\uc0b0 \uac15\ub3c4(Arithmetic Intensity)\uac00 \ub9e4\uc6b0 \ub0ae\uc2b5\ub2c8\ub2e4. ResNet \uad6c\uc870\ub294 \uc8fc\uae30\uc801\uc73c\ub85c \uc774 Memory-Bound \uc5f0\uc0b0\uc744 \uc218\ud589\ud574\uc57c \ud558\ubbc0\ub85c, NPU\uc758 \uc5f0\uc0b0 \uc720\ub2db\ub4e4\uc774 \ub367\uc148 \ub370\uc774\ud130\uac00 \ub85c\ub529\ub418\uae30\ub97c \uae30\ub2e4\ub9ac\uba70 \uba48\ucd94\ub294(Stall) \ud604\uc0c1\uc744 \uc720\ubc1c\ud569\ub2c8\ub2e4.<\/p>\n\n\n\n<h2 class=\"wp-block-heading\"><strong>6. \uacb0\ub860: \uc720\uc5f0\uc131(Flexibility)\uc744 \uc704\ud55c \ube44\uc6a9, \uadf8\ub9ac\uace0 \ub2e4\uc74c \ub2e8\uacc4<\/strong><\/h2>\n\n\n\n<p class=\"wp-block-paragraph\">ResNet\uc758 Skip Connection\uc740 \ub525\ub7ec\ub2dd\uc758 \uc815\ud655\ub3c4\ub97c \ud601\uba85\uc801\uc73c\ub85c \ub192\uc5ec\uc8fc\uc5c8\uc9c0\ub9cc, \ud558\ub4dc\uc6e8\uc5b4 \uc5d4\uc9c0\ub2c8\uc5b4\uc5d0\uac8c\ub294 &#8220;\ube44\uc21c\ucc28\uc801 \ub370\uc774\ud130 \uad00\ub9ac&#8221;\ub77c\ub294 \uae4c\ub2e4\ub85c\uc6b4 \uc219\uc81c\ub97c \uc548\uaca8\uc8fc\uc5c8\uc2b5\ub2c8\ub2e4. \uc774 \ubb38\uc81c\ub97c \ud574\uacb0\ud558\uae30 \uc704\ud574 \ud604\ub300\uc758 NPU \ucef4\ud30c\uc77c\ub7ec\ub4e4\uc740 \uace0\ub3c4\uc758 <strong>\uba54\ubaa8\ub9ac \ud560\ub2f9(Memory Allocation) \uc54c\uace0\ub9ac\uc998<\/strong>\uc744 \uc0ac\uc6a9\ud558\uac70\ub098, \ud558\ub4dc\uc6e8\uc5b4\uc801\uc73c\ub85c <strong>Skip Connection \uc804\uc6a9 \uc555\ucd95\uae30<\/strong>\ub97c \ud0d1\uc7ac\ud558\uae30\ub3c4 \ud569\ub2c8\ub2e4.<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\uc774\uac83\uc73c\ub85c [Category 1. AI &amp; HW Fundamentals] \uc2dc\ub9ac\uc988\ub97c \ub9c8\uce69\ub2c8\ub2e4. \uc6b0\ub9ac\ub294 \uc9c0\uae08\uae4c\uc9c0 12\ud3b8\uc758 \uae00\uc744 \ud1b5\ud574 \ub2e4\uc591\ud55c \ubcd1\ubaa9 \ud604\uc0c1\uc744 \ub9c8\uc8fc\ud588\uc2b5\ub2c8\ub2e4.<\/p>\n\n\n\n<ul class=\"wp-block-list\">\n<li><strong>MLP:<\/strong> \uba54\ubaa8\ub9ac \ub300\uc5ed\ud3ed\uc774 \ubd80\uc871\ud574\uc11c \ub290\ub838\uace0 (Memory-Bound),<\/li>\n\n\n\n<li><strong>MobileNet:<\/strong> \uc5f0\uc0b0\uae30\ub294 \ub180\uace0 \uc788\ub294\ub370 \uad6c\uc870\uac00 \ubcf5\uc7a1\ud574\uc11c \ub290\ub838\uc73c\uba70 (Utilization Issue),<\/li>\n\n\n\n<li><strong>ResNet:<\/strong> \uba54\ubaa8\ub9ac \uad00\ub9ac\uc640 \ubc84\ud37c\ub9c1 \ub54c\ubb38\uc5d0 \uc2dc\uc2a4\ud15c\uc774 \uba48\uce6b\uac70\ub838\uc2b5\ub2c8\ub2e4 (Buffer Management).<\/li>\n<\/ul>\n\n\n\n<p class=\"wp-block-paragraph\">\uadf8\ub807\ub2e4\uba74, \ub0b4\uac00 \uc124\uacc4\ud55c(\ud639\uc740 \ubd84\uc11d \uc911\uc778) NPU\uac00 \ub290\ub9ac\ub2e4\uba74 \ub3c4\ub300\uccb4 \ub204\uad6c \ud0d3\uc77c\uae4c\uc694? \uc5f0\uc0b0\uae30\uc77c\uae4c\uc694, \uba54\ubaa8\ub9ac\uc77c\uae4c\uc694?<\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\ub2e4\uc74c \uae00\ubd80\ud130 \uc2dc\uc791\ub418\ub294 [Category 2. NPU Design &amp; Optimization]\uc5d0\uc11c\ub294 \uc774 \ubcf5\uc7a1\ud55c \ubcd1\ubaa9 \ud604\uc0c1\ub4e4\uc744 \ub2e8 \ud55c \uc7a5\uc758 \uadf8\ub798\ud504\ub85c \uba85\ucf8c\ud558\uac8c \uc9c4\ub2e8\ud558\ub294 \uc2dc\uc2a4\ud15c Architect \ucd5c\uace0\uc758 \ubd84\uc11d \ub3c4\uad6c, Roofline Model\uc5d0 \ub300\ud574 \uc54c\uc544\ubcf4\uaca0\uc2b5\ub2c8\ub2e4.<\/p>\n\n\n<style>.kadence-column1338_20ae35-f9 > .kt-inside-inner-col{box-shadow:0px 0px 14px 0px rgba(0, 0, 0, 0.2);}.kadence-column1338_20ae35-f9 > .kt-inside-inner-col,.kadence-column1338_20ae35-f9 > .kt-inside-inner-col:before{border-top-left-radius:0px;border-top-right-radius:0px;border-bottom-right-radius:0px;border-bottom-left-radius:0px;}.kadence-column1338_20ae35-f9 > .kt-inside-inner-col{column-gap:var(--global-kb-gap-sm, 1rem);}.kadence-column1338_20ae35-f9 > .kt-inside-inner-col{flex-direction:column;}.kadence-column1338_20ae35-f9 > .kt-inside-inner-col > .aligncenter{width:100%;}.kadence-column1338_20ae35-f9 > .kt-inside-inner-col:before{opacity:0.3;}.kadence-column1338_20ae35-f9{position:relative;}@media all and (max-width: 1024px){.kadence-column1338_20ae35-f9 > .kt-inside-inner-col{flex-direction:column;justify-content:center;}}@media all and (max-width: 767px){.kadence-column1338_20ae35-f9 > .kt-inside-inner-col{flex-direction:column;justify-content:center;}}<\/style>\n<div class=\"wp-block-kadence-column kadence-column1338_20ae35-f9\"><div class=\"kt-inside-inner-col\">\n<p class=\"wp-block-paragraph\"><strong>\uad00\ub828 \uae00<\/strong><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2705<a href=\"https:\/\/rtlearner.com\/ai-architecture-1-neuron-hardware-mac-analysis\/\" data-type=\"post\" data-id=\"1248\">AI Architecture 1. \uc778\uacf5 \ub274\ub7f0\uc758 \ud574\ubd80: silicon\uc5d0\uc11c Y=WX+B \uad6c\ud604<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2705<a href=\"https:\/\/rtlearner.com\/ai-architecture-2-activation-relu-vs-sigmoid\/\" data-type=\"post\" data-id=\"1255\">AI Architecture 2. \ud65c\uc131\ud654 \ud568\uc218\uc758 \ube44\uc6a9: ReLU vs Sigmoid<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2705<a href=\"https:\/\/rtlearner.com\/ai-architecture-3-matmul-simd-parallel-processing\/\" data-type=\"post\" data-id=\"1263\">AI Architecture 3. \ud589\ub82c\uacf1(MatMul)\uc758 \ubbf8\ud559: \ub525\ub7ec\ub2dd\uc774 GPU\/NPU\ub97c \uc120\ud0dd\ud55c \uc774\uc720<\/a><\/p>\n\n\n\n<p class=\"wp-block-paragraph\">\u2705<a href=\"https:\/\/rtlearner.com\/ai-architecture-4-training-vs-inference\/\" data-type=\"post\" data-id=\"1267\">AI Architecture 4. \ud559\uc2b5(Training) vs \ucd94\ub860(Inference)<\/a><\/p>\n<\/div><\/div>\n\n\n\n<p class=\"wp-block-paragraph\">\ucc38\uace0: <em><a href=\"https:\/\/arxiv.org\/abs\/1512.03385\" target=\"_blank\" rel=\"noopener\">Deep Residual Learning for Image Recognition<\/a><\/em><\/p>\n","protected":false},"excerpt":{"rendered":"<p>In the previous MLP and Memory Wall, we discussed the \"memory wall\" phenomenon, where memory bandwidth limits system performance. In CNN and Locality,<\/p>","protected":false},"author":1,"featured_media":1339,"comment_status":"open","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_kadence_starter_templates_imported_post":false,"_kad_post_transparent":"","_kad_post_title":"","_kad_post_layout":"","_kad_post_sidebar_id":"","_kad_post_content_style":"","_kad_post_vertical_padding":"","_kad_post_feature":"","_kad_post_feature_position":"","_kad_post_header":false,"_kad_post_footer":false,"_kad_post_classname":"","footnotes":""},"categories":[116],"tags":[117,118],"class_list":["post-1338","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-and-hw-fundamentals","tag-ai","tag-architecture"],"_links":{"self":[{"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/posts\/1338","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/comments?post=1338"}],"version-history":[{"count":4,"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/posts\/1338\/revisions"}],"predecessor-version":[{"id":1376,"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/posts\/1338\/revisions\/1376"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/media\/1339"}],"wp:attachment":[{"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/media?parent=1338"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/categories?post=1338"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/rtlearner.com\/en\/wp-json\/wp\/v2\/tags?post=1338"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}<!-- This website is optimized by Airlift. Learn more: https://airlift.net. Template:. Learn more: https://airlift.net. Template: 6a588749d36f73a90a601ae0. Config Timestamp: 2026-07-16 07:24:54 UTC, Cached Timestamp: 2026-08-15 14:09:40 UTC, Optimization Time: 4.02ms -->